Update: Sorry for the audio sync issue ?
In this video, we talk about Petals. A new project combines old-ish technology with large language models to allow you to run even the largest models in a distributed fashion on any device. This incredible new implementation truly decentralizes LLMs (LLaMA, Bloom, MPT, etc) and allows consumer-grade computers to run any large model.
Enjoy 🙂
Become a Patron ? – https://patreon.com/MatthewBerman
Join the Discord ? – https://discord.gg/xxysSXBxFW
Follow me on Twitter ? – https://twitter.com/matthewberman
Subscribe to my Substack ?? – https://matthewberman.substack.com
Links:
Petals – https://petals.dev/
Research – https://research.yandex.com/blog/petals-decentralized-inference-and-finetuning-of-large-language-models
Petals Google Colab – https://colab.research.google.com/drive/1uCphNY7gfAUkdDrTx21dZZwCOUDCMPw8?usp=sharing
