NVIDIA Support
Congratulations to the development team (this is really remarkable!). Unfortunately I don't have MLX (so I can't run this), but if you plan on making this CUDA compatible someday istg my life shall be yours.
I agree, MLX and whole Apple platform is very niche and small audience very restricted by soldered RAM(very strict ceiling for any work). Many models here including in Trillion sizes like Longcat2.0 have only MLX offerings and no demand (300 downloads max), any models without support in llama.cpp model list or at least custom fork of it are doomed to never be tested, like huge Yuan3.0-Ultra.
And i'm sure restriction of sideloading anything not from Apple Store incoming to Macs (on iPhones/iPads its forbidden forever) because of serious factors-1)iPhone Apps Store is now Desktop store on Macs and 2)launch of new Siri Ai, which remake is finished and preview ver quite good, it can work with other API Ai models but only through Siri itself, so restriction on OS level is very near.
we are working on it, it is in our roadmap
very restricted by soldered RAM
Not sure if you're aware: NVIDIA solders their RAM as well.
A 96 GB RTX PRO 6000 costs $17,999 before the rest of the PC (at B&H Photo, actually in stock), has no NVLink, and youβd need six of themβ**$107,994 in GPUs alone**βto exceed two 256G M5 Ultra Mac Studios
... which would cost you $18,998 and gets you two whole computers with distributed MLX over Thunderbolt 5 RDMA
And i'm sure restriction of sideloading anything not from Apple Store incoming to Macs
You're sure? (You're not)
Apple did good lately, expect moar MLX
Apple did good lately, expect moar MLX
Only if you mean "Their chips consume less energy on average" or "MLX fits bigger models better", which is true.
But inference? training? speed? support? everything else? (compared to CUDA)?
Still got a little work ahead of you before you start bragging about it lil homie.
@modemthug i've anticipated such reply, but i've worked with Apple during iPod era and in all their history they never really allow you to do anything on their platform without their control and models from China in direct competition with new Siri Ai. History with Mac clones and Cydia on iPhone only confirming that. When Siri gets out from Beta - the side-loading on desktops will be closed very fast and easily. Asahi Linux project also stalled because drivers on all Apple platform was always proprietary and Hackintosh mostly failed by this, its always was a mess, yes Mac OS can be loaded on PC but Wifi work not guaranteed, 50% of everything else not working usually.
This Mlx thing reminds me again early iPhone years, if anyone forgot people unlocked the modem firmware there in just a year to allow its use outside of USA contract & locked bootloader, and Apple wasnt happy, closed by next models and OS versions. Apple is "early adopters" always, again they'll allow "let all flowers blooms" to remove it all after with massive dose of reality. I'll advice not to invest many in their hardware by risks of inevitable lock or at least sit on old firmware or OS as long as possible (same situation in iPhones with firmwares on used market today).