How can I submit fine-tuned falcon model to LLM Leaderboard?

#92
by Sandiago21 - opened

How can I submit my fine-tuned falcon model to the LLM Leaderboard? I get error that the model needs to be launched with trust_remote_code=True and that for safety reason, these models are not allowed to be automatically submitted to the leaderboard.

Sandiago21 changed discussion title from Submit fine-tuned falcon model to LLM Leaderboard to How can I submit fine-tuned falcon model to LLM Leaderboard?
Open LLM Leaderboard org

At the moment, you cannot, as launching a model which needs trust_remote_code=True requires us to manually inspect the code to make sure it does not contain anything malicious and it's too time consuming for our small team of 2 people.
However, we are working on adding a better system in the next few months, we'll do our best and keep the community posted 🤗

clefourrier changed discussion status to closed

I see, thank you for your reply @clefourrier !

Sign up or log in to comment