(image generated with Bing)
Notes on trying out generative AI.
Google has a free online learning path on generative AI. Finished first few activities. Nothing technical and just high level concepts.
---
Has been trying to run the Facebook's LLaMA models locally. There is a project llama.cpp to run the models with c/c++ program. Support both CPU and GPU acceleration. So far only tried the 7B and 13B models. Not impressed. Should give the 65B model a try. But even with quantization, it will need almost 40GB just for loading the model. Will need to get more RAM first.
BTW, to download the LLaMA models, just google "download llama". There is a shell script to help to pull the files quickly
---
There is a new open source generative Falcon that clams to be better than LLaMA (at least for now).
No comments:
Post a Comment