openbmb

/

MiniCPM-V-2_6-int4

like 54

For the time being, it's just bnb.
Other quantization frameworks have difficulty supporting multimodal models, and we don't have enough energy to develop them yet.
Or you can use our version of gguf, which is also int4.
here: https://huggingface.co/openbmb/MiniCPM-V-2_6-gguf

Thanks for the reply. 🙏 Sadly, maybe I will try to do AWQ quantization of this model if my supervisor accept it.

@CCRss I'm am also interested in AWQ/MLC quantization for running on Jetson devices. Maybe we can join effort to some point

@Aniel99 I will be happy to. You can contact me via twitter. https://x.com/VladimirAlbrek1