Student-Model übertrifft Lehrer: Neue Distillationsmethode für LLMs

The current state of the art in the field of computer vision is that the best performing models are based on deep neural networks. These models are trained on large datasets and are able to learn complex features from the data. However, they are also very computationally expensive and require a lot of memory to store the model parameters. This makes them difficult to deploy on resource-constrained devices such as mobile phones or embedded systems. In addition, the models are often overfitted to the training data and do not generalize well to new data. This is a major limitation of the current state of the art.