Technonia

/

mistral-7b-cot-neftune

like 3

Below is the formatting function that I used for training:

BASE_FSTR = """{0}

{1}
"""


def formatting_prompts_func(example):
    output_texts = []
    for i in range(len(example['source'])):
        source = example['source'][i]
        target = example['target'][i]
        rationale = example['rationale'][i]

        text = BASE_FSTR.format(source, target)
        output_texts.append(text)
    return output_texts

not sure if I get it right, but did you include the rationale?
also my interpretation of this code is there is no token sequence indicating the alternation between instruction and response.
How do you prompt your model?
I did a similar model that only used a few records of the cot dataset, I formatted it like this:

### Instruction:
In this task, you are given a sentence with a missing word that can be an object, a person, and/or an action. Fill in the blank with a plausible word. Although each sentence has many correct answers, you only have to write one answer. PersonX puts ___ on PersonY's car 
### Reasoning:
Since PersonX is putting water on PersonY's car, it could be assumed that the car is on fire. 
### Response:
water

If I use that model, I prompt it for reasoning.

First, I did not include the rationale (sorry for the confusion).

For testing, I used the code below:

def get_completion(query: str, model, tokenizer, max_new_tokens=50, temperature=0.5, top_k=2) -> str:
    device = "cuda:0"

    prompt = f"""{query}

    """

    encodeds = tokenizer(query, return_tensors="pt", add_special_tokens=True)
    model_inputs = encodeds.to(device)

    with torch.no_grad():
        outputs = model.generate(
            **model_inputs,
            max_new_tokens=max_new_tokens,
            do_sample=True,
            temperature=temperature,
            top_k=top_k,
            return_dict_in_generate=True,
            output_scores=True,
            pad_token_id=tokenizer.eos_token_id,
            repetition_penalty=1.0,
        )
    output = tokenizer.decode(outputs.sequences[0], skip_special_tokens=True)
    return output

Initially, I also made some templates just like what you suggested.
However, the average length of text data in the dataset that I used for training exceeds 5000 tokens (I used 4096 for context length).
That is the main reason that I used an extremely simple prompt template.

Thanks for clarifying :)