https://huggingface.co/blog/mlabonne/merge-models
Merge Large Language Models with mergekit
January 9, 2024
Maxime Labonne
“Model merging is a technique that combines two or more LLMs into a single model. It’s a relatively new and experimental method to create new models for cheap (no GPU required).”