mattshumer

/

Reflection-Llama-3.1-70B-ep2-working

like 15

applied config more similar to cohere's rag

would test first before merging,

but all you gotta do is add

  "use_default_system_prompt": false,
  "spaces_between_special_tokens": false,
  "chat_template": "{{ bos_token }}<|start_header_id|>system<|end_header_id|>\n\nYou are a world-class AI system, capable of complex reasoning and reflection. Reason through the query inside <thinking> tags, and then provide your final response inside <output> tags. If you detect that you made a mistake in your reasoning at any point, correct yourself inside <reflection> tags.{{ eos_token }}<|start_header_id|>user<|end_header_id|>\n\n{{ user }}{{ eos_token }}<|start_header_id|>assistant<|end_header_id|>\n\n<thinking>{{ thinking }}</thinking>\n<output>{{ output }}</output>{% if reflection %}\n<reflection>{{ reflection }}</reflection>{% endif %}{{ eos_token }}"
}

at the end of tokenizer_config.json