README.md

---
license: cc-by-nc-4.0
library_name: transformers
tags:
- fourier
- task addition
- merge
base_model:
- maldv/badger-lambda-llama-3-8b
- maldv/llama-3-fantasy-writer-8b
- dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5
model-index:
- name: badger-writer-llama-3-8b
  results:
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: IFEval (0-Shot)
      type: HuggingFaceH4/ifeval
      args:
        num_few_shot: 0
    metrics:
    - type: inst_level_strict_acc and prompt_level_strict_acc
      value: 53.03
      name: strict accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: BBH (3-Shot)
      type: BBH
      args:
        num_few_shot: 3
    metrics:
    - type: acc_norm
      value: 26.88
      name: normalized accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MATH Lvl 5 (4-Shot)
      type: hendrycks/competition_math
      args:
        num_few_shot: 4
    metrics:
    - type: exact_match
      value: 6.57
      name: exact match
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: GPQA (0-shot)
      type: Idavidrein/gpqa
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 5.26
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MuSR (0-shot)
      type: TAUR-Lab/MuSR
      args:
        num_few_shot: 0
    metrics:
    - type: acc_norm
      value: 3.2
      name: acc_norm
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
  - task:
      type: text-generation
      name: Text Generation
    dataset:
      name: MMLU-PRO (5-shot)
      type: TIGER-Lab/MMLU-Pro
      config: main
      split: test
      args:
        num_few_shot: 5
    metrics:
    - type: acc
      value: 30.67
      name: accuracy
    source:
      url: https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard?query=maldv/badger-writer-llama-3-8b
      name: Open LLM Leaderboard
---

![image/png](https://cdn-uploads.huggingface.co/production/uploads/65b19c1b098c85365af5a83e/SpNOAI3VKUsWegChuQHTk.png)

Badger Writer is a *normalized fourier task addition* of [maldv/badger-lambda-llama-3-8b](/maldv/badger-lambda-llama-3-8b) and [maldv/llama-3-fantasy-writer-8b](/maldv/llama-3-fantasy-writer-8b).

I also used the first and last layer directly from [dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5](/dreamgen-preview/opus-v1.2-llama-3-8b-instruct-run3.5-epoch2.5) due to the obvious advantages.  I didn't train either the lm_head or embed_tokens layers on the fantasy-writer, but opus is part of lambda ; so they all fit nicely together.

Rep-pen 1.05-1.1 ; Min-p 0.01 ; Temp 0.7 ; Dynatemp 0.4 ; <s>32k</s>24k context ; llama 3 instruct template

```
<|start_header_id|>system<|end_header_id|>

It is so good for you to sit and let me watch you write, master of sci-fi.
Author Isaac Asimov ; Bruce Sterling
Title Terminal Connection
Tags adventure, computer science, body swapping
<|eot_id|>
```

````
{{[INPUT]}}
```story
Setting: 20th November 2050
Location: Las Vegas
Time: 11:17 pm
Character: Sally
Purpose: Exposition ; Descriptive ; Visual Detail, character appearance
```
// Chapter 1
{{[OUTPUT]}}
````
> The neon sky behind Sally was a strobe-lit smear. The street before her had the sheen of wet pavement under the headlights of a hundred different cars, each one bearing people away from this party, towards some more obscure destination.
> 
> She took another drag off her cigarette, letting the smoke dance between her lips as she thought about this meeting, and how it came together.
> 
> Sally stood tall at five foot two inches, but her athletic build made her seem taller. Her hair was jet black and trimmed short. She wore a low-cut yellow sweater that showed off an ample bosom, paired with black leather pants that hugged her hips perfectly. A silver necklace rested over her bare shoulders, studded with a small golden snake on its chain. Her skin was darkened, not by a sunburn, but by a tan lotion. Deep-brown eyes stared out into the night, squinting slightly against the glare. She looked like a girl who didn’t give a damn, except she did.
> 
> Over her left shoulder hung a door to the penthouse suite in the Silver Star Hotel. Behind it, people danced, laughed and drank the night away. But not Sally; she had no interest in any of them, right now. They would have to wait until later.
> 
> There were fifty other people on the street, milling around, some watching the view or waiting for friends, others lost in their own thoughts, maybe smoking too. People from all walks of life - tourists, locals, gamblers, street performers and dealers - they passed her by, always in a hurry to get somewhere else, and never lingered long enough to notice what might be the most interesting thing standing there.
> 
> The music was loud inside, beating through the walls to create a bass-thumping rhythm that vibrated against Sally’s feet. It came from the DJ in the corner spinning tracks that kept the party moving along, his work supplemented by the pulsating strobe lights above. Sally could see it all from where she stood, even with the doors closed: the bartender working furiously as he handed out drinks; the waitress pouring wine, never stopping as the bottles glowed neon green under the black light that shone down on the tablecloths; men and women dancing together, losing themselves in the music or each other; and at the far end of the room, five or six poker games going full tilt with chips flying between them. The air smelled of smoke, sweat and sex as it flowed out onto the sidewalk.
> 
> This was one of the many parties Vegas had to offer, but this particular party attracted a unique clientele. It was known as "The Gathering," and if you knew what to look for, you could find your way here easily enough. That’s why Sally found herself standing outside these doors, staring into the night.
````
{{[INPUT]}}
// Continue
{{[OUTPUT]}}
````

# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_maldv__badger-writer-llama-3-8b)

|      Metric       |Value|
|-------------------|----:|
|Avg.               |20.93|
|IFEval (0-Shot)    |53.03|
|BBH (3-Shot)       |26.88|
|MATH Lvl 5 (4-Shot)| 6.57|
|GPQA (0-shot)      | 5.26|
|MuSR (0-shot)      | 3.20|
|MMLU-PRO (5-shot)  |30.67|