We are announcing the Supra3 family with four core SLM models: - Supra3 Flash Lite: 25M parameters, ~60B pretraining tokens - Supra3 Flash: 50M parameters, ~100B pretraining tokens - Supra3 Pro: 75M parameters, ~150B pretraining tokens - Supra3 Ultra: 100M parameters, ~200B pretraining tokens
For Supra3 Pro and Ultra, we search for sponsors who give us free access to compute like RTX 5090 32GB or so.
We estimate the total cost of the pro and ultra models at around $600.
For Supra3 Flash Lite and Flash, we do not need sponsors.
If anyone would apply for helping us, we would be really thankful and this person would get early access to new modele, insider information, credit and more!
I don't know if it was us or one of you guys or maybe all of us at once but lately we have seen a finetuning/pretraining explosion of models below 200m params and we can't be more happy about it keep coming tinkerers all of this is possible because of you!