What is the best way to finetune this (with a LORA?)
#15
by
PaulTheHuman
- opened
Anyone know a good way to fine tune this model perhaps using a LORA or something similar? Specifically the int4 ONNX version.
You can use this example script.
You will have to train it in bf16 or fp16 first and then convert it to int4 onnx afterwards
You can also use Olive to fine-tune with LoRA and then generate the resulting int4 ONNX model.
kvaishnavi
changed discussion status to
closed