Regarding max seq length

#33
by sandeep456 - opened

what is the maximum sequence length of input this model can process. in MTEB leaderboard it is mentioned as 32k while on your huggingface model page it is mentioned as 8k-
Model Details
Base Decoder-only LLM: Mistral-7B-v0.1
Pooling Type: Latent-Attention
Embedding Dimension: 4096
Vocab size: 32k
Max Input Token length: 8192

Please help me understand.

NVIDIA org
edited Jun 26

Hi, @sandeep456 . Here is the modified answers to your question. Max sequence input length is 32k (same as mistral-7b-v0.1 model).

Sign up or log in to comment