aifoundry-org/FLUX.1-schnell-Quantized

Quantized versions of https://huggingface.co/black-forest-labs/FLUX.1-schnell

Tools used for quantization: modded stable-diffusion.cpp, LlamaQuantizer

Work in progress, use at your own risk

How to:

[WIP]

1. Dowload and build stable-diffusion.cpp
2. Download one of the models from this repo and
   • Autoencoder https://huggingface.co/black-forest-labs/FLUX.1-schnell/resolve/main/ae.safetensors
   • CLIP_L https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/clip_l.safetensors
   • T5XXL https://huggingface.co/comfyanonymous/flux_text_encoders/blob/main/t5xxl_fp16.safetensors
3. Enter your stable-diffusion.cpp dir
4. Run the following command:

   ./build/bin/sd --diffusion-model [path to gguf] --vae [path to ae.safetensors] --clip_l [path to clip_l.safetensors] --t5xxl [path to t5xxl_fp16.safetensors] -p "a frog holding a sign saying 'hi' " -o ../frog.png -v --cfg-scale 1.0 --sampling-method euler -v --seed 42 --steps 4
   

Results:

Quant type	Size	Result (x0.5)	Download link
default	23.8 GB		flux1-schnell.safetensors.gguf
FP16	23.8 GB		flux1-schnell-F16.gguf
Q8_0	12.6 GB		flux1-schnell-Q8_0.gguf
Q5_0	8.18 GB		flux1-schnell-Q5_0.gguf
Q5_1	8.92 GB		flux1-schnell-Q5_1.gguf
Q4_0	6.69 GB		flux1-schnell-Q4_0.gguf
Q4_1	7.43 GB		flux1-schnell-Q4_1.gguf
Q4_K	6.69 GB		flux1-schnell-Q4_K.gguf
Q2_K	3.9 GB		flux1-schnell-Q2_K.gguf