BramVanroy commited on
Commit
146a18e
β€’
1 Parent(s): 3ebf539

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +12 -12
README.md CHANGED
@@ -1,6 +1,6 @@
1
  ---
2
  license: mit
3
- base_model: BramVanroy/fietje-2b-instruct
4
  tags:
5
  - trl
6
  - fietje
@@ -10,7 +10,7 @@ datasets:
10
  - BramVanroy/ultra_feedback_dutch_cleaned
11
  - BramVanroy/orca_dpo_pairs_dutch_cleaned
12
  model-index:
13
- - name: fietje-2b-chat
14
  results: []
15
  pipeline_tag: text-generation
16
  inference: false
@@ -19,27 +19,27 @@ language:
19
  ---
20
 
21
  <p align="center" style="margin:0;padding:0">
22
- <img src="https://huggingface.co/BramVanroy/fietje-2b-chat/resolve/main/img/fietje-2b-banner-rounded.png" alt="Fietje banner" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
23
  </p>
24
 
25
  <div style="margin:auto; text-align:center">
26
- <h1 style="margin-bottom: 0">Fietje 2B Chat</h1>
27
  <em>An open and efficient LLM for Dutch</em>
28
  </div>
29
 
30
  <blockquote class="tip" style="padding: 1.5em; border: 0">
31
  <p align="center" style="text-align: center; margin: 0">
32
- <a href="https://huggingface.co/BramVanroy/fietje-2b">πŸ‘±β€β™€οΈ Base version</a> -
33
- <a href="https://huggingface.co/BramVanroy/fietje-2b-instruct">πŸ€– Instruct version</a> -
34
- <a href="https://huggingface.co/BramVanroy/fietje-2b-chat">πŸ’¬ Chat version</a> (this one) -
35
- <a href="https://huggingface.co/BramVanroy/fietje-2b-chat-GGUF">πŸš€ GGUF of Chat</a>
36
  </p>
37
  <p align="center" style="text-align: center; margin: 0">
38
- <a href="https://huggingface.co/spaces/BramVanroy/fietje-2b"><strong>Chat with Fietje here!</strong></a>
39
  </p>
40
  </blockquote>
41
 
42
- This is the chat version of Fietje, a DPO-tuned (aligned) continuation on [the instruct version](https://huggingface.co/BramVanroy/fietje-2b-instruct). Fietje is an adapated version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2), tailored to Dutch text generation by training on 28B tokens. It is small and efficient with a size of 2.7 billion parameters while performing almost on par with more powerful Dutch LLMs of twice its size like [GEITje 7B Ultra](https://huggingface.co/BramVanroy/GEITje-7B-ultra).
43
 
44
  A thorough description of the creation and evaluation of Fietje as well as usage examples are available in [this Github repository](https://github.com/BramVanroy/fietje).
45
 
@@ -49,12 +49,12 @@ The same limitations as [phi-2](https://huggingface.co/microsoft/phi-2#limitatio
49
 
50
  ## Training and evaluation data
51
 
52
- Fietje 2B instruct was finetuned from [the instruct model](https://huggingface.co/BramVanroy/fietje-2b-instruct) on the following datasets. Number of training samples per dataset given in brackets, totalling 18,653 samples.
53
 
54
  - [BramVanroy/ultra_feedback_dutch_cleaned](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned) subset `dpo_hq`: a cleaned version of [BramVanroy/ultra_feedback_dutch](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch) (9186)
55
  - [BramVanroy/orca_dpo_pairs_dutch_cleaned](https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch_cleaned) subset `dpo_all`: a cleaned version of [BramVanroy/orca_dpo_pairs_dutch](https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch) (9467)
56
 
57
- A lot of different learning rates, beta, en batch sizes were investigated in search of a converging combination. You can find them all in [the W&B runs](https://wandb.ai/bramvanroy/dpo-fietje-2b).
58
 
59
  ## Training procedure
60
 
 
1
  ---
2
  license: mit
3
+ base_model: BramVanroy/fietje-2-instruct
4
  tags:
5
  - trl
6
  - fietje
 
10
  - BramVanroy/ultra_feedback_dutch_cleaned
11
  - BramVanroy/orca_dpo_pairs_dutch_cleaned
12
  model-index:
13
+ - name: fietje-2-chat
14
  results: []
15
  pipeline_tag: text-generation
16
  inference: false
 
19
  ---
20
 
21
  <p align="center" style="margin:0;padding:0">
22
+ <img src="https://huggingface.co/BramVanroy/fietje-2-chat/resolve/main/img/fietje-2b-banner-rounded.png" alt="Fietje banner" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
23
  </p>
24
 
25
  <div style="margin:auto; text-align:center">
26
+ <h1 style="margin-bottom: 0">Fietje 2 Chat</h1>
27
  <em>An open and efficient LLM for Dutch</em>
28
  </div>
29
 
30
  <blockquote class="tip" style="padding: 1.5em; border: 0">
31
  <p align="center" style="text-align: center; margin: 0">
32
+ <a href="https://huggingface.co/BramVanroy/fietje-2">πŸ‘±β€β™€οΈ Base version</a> -
33
+ <a href="https://huggingface.co/BramVanroy/fietje-2-instruct">πŸ€– Instruct version</a> -
34
+ <a href="https://huggingface.co/BramVanroy/fietje-2-chat">πŸ’¬ Chat version</a> (this one) -
35
+ <a href="https://huggingface.co/BramVanroy/fietje-2-chat-GGUF">πŸš€ GGUF of Chat</a>
36
  </p>
37
  <p align="center" style="text-align: center; margin: 0">
38
+ <a href="https://huggingface.co/spaces/BramVanroy/fietje-2"><strong>Chat with Fietje here!</strong></a>
39
  </p>
40
  </blockquote>
41
 
42
+ This is the chat version of Fietje, a DPO-tuned (aligned) continuation on [the instruct version](https://huggingface.co/BramVanroy/fietje-2-instruct). Fietje is an adapated version of [microsoft/phi-2](https://huggingface.co/microsoft/phi-2), tailored to Dutch text generation by training on 28B tokens. It is small and efficient with a size of 2.7 billion parameters while performing almost on par with more powerful Dutch LLMs of twice its size like [GEITje 7B Ultra](https://huggingface.co/BramVanroy/GEITje-7B-ultra).
43
 
44
  A thorough description of the creation and evaluation of Fietje as well as usage examples are available in [this Github repository](https://github.com/BramVanroy/fietje).
45
 
 
49
 
50
  ## Training and evaluation data
51
 
52
+ Fietje 2 Chat was finetuned from [the instruct model](https://huggingface.co/BramVanroy/fietje-2-instruct) on the following datasets. Number of training samples per dataset given in brackets, totalling 18,653 samples.
53
 
54
  - [BramVanroy/ultra_feedback_dutch_cleaned](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch_cleaned) subset `dpo_hq`: a cleaned version of [BramVanroy/ultra_feedback_dutch](https://huggingface.co/datasets/BramVanroy/ultra_feedback_dutch) (9186)
55
  - [BramVanroy/orca_dpo_pairs_dutch_cleaned](https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch_cleaned) subset `dpo_all`: a cleaned version of [BramVanroy/orca_dpo_pairs_dutch](https://huggingface.co/datasets/BramVanroy/orca_dpo_pairs_dutch) (9467)
56
 
57
+ A lot of different learning rates, beta, en batch sizes were investigated in search of a converging combination. You can find them all in [the W&B runs](https://wandb.ai/bramvanroy/dpo-fietje-2).
58
 
59
  ## Training procedure
60