Model Soups: Averaging Fine-Tuned Weights Beats Ensembles' Cost

Click a step to highlight it. Every run starts from the same pre-trained weights.

k models: an ensemble needs k forward passes and k copies in memory. A soup is one network.

Loss plane through weight space Illustrative schematic

Sort by validation accuracy, add a checkpoint, keep it only if validation accuracy doesn't drop. Accuracies below are made up to show the logic.

Each cell is one image out of 10,000. Highlighted count is the reported gap; cell positions are Illustrative