Halving LLM Training Costs: One Month of BottleCap AI
Halving LLM Training Costs: One Month of BottleCap AI
One month into BottleCap AI, our mission is to build a European research hub that makes LLMs radically more efficient, with a long-term goal of a 100x efficiency gain over today's most prominent models.
The first result:
- New training algorithm cuts LLM training costs by up to 50%
- Applies to cutting-edge models that would otherwise cost tens of millions to train
- Not a rehash of Mixture of Experts — a new approach focused on speed
- Early signs it may also improve generalization
- Core dev team, led by Tomas Mikolov, still forming
Early days, but a promising start toward radically more efficient LLMs.
One month ago, we started BottleCap AI with the mission to build a European research hub to make LLMs (today's AI models) radically more efficient with the ambitious goal to increase efficiency of most prominent models by 100x.
While we're still forming the core dev team led by Tomas Mikolov we've already built a new algorithm that cuts LLM's training costs up to 50%.
To put that in perspective: training a cutting-edge large language model can cost tens of millions of dollars. With our new approach, that number can drop by nearly half as of today. Promising start!
We're not just rehashing trendy ideas like Mixture of Experts. Our new algorithms focus on speed, and in some cases have potential to even improve generalization, which can bring completely new capability to current AI models.
Lot of work ahead, but we are very optimistic about the future of LLMs!