Hugging Face's logo

imdatta0
/
mistral_7b_v_Magiccoder_evol_10k

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
imdatta0 commited on
Commit
b96c7bc
•
1 Parent(s): 8eabcd0

End of training

Browse files
Files changed (2) hide show
  1. README.md +42 -42
  2. adapter_model.safetensors +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.1499
21
 
22
  ## Model description
23
 
@@ -37,10 +37,10 @@ More information needed
37
 
38
  The following hyperparameters were used during training:
39
  - learning_rate: 0.0001
40
- - train_batch_size: 8
41
- - eval_batch_size: 8
42
  - seed: 42
43
- - gradient_accumulation_steps: 8
44
  - total_train_batch_size: 64
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: cosine
@@ -51,44 +51,44 @@ The following hyperparameters were used during training:
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:------:|:----:|:---------------:|
54
- | 1.1885 | 0.0262 | 4 | 1.1900 |
55
- | 1.0966 | 0.0523 | 8 | 1.1442 |
56
- | 1.1468 | 0.0785 | 12 | 1.1514 |
57
- | 1.0845 | 0.1047 | 16 | 1.1671 |
58
- | 1.1413 | 0.1308 | 20 | 1.1635 |
59
- | 1.0557 | 0.1570 | 24 | 1.1689 |
60
- | 1.1949 | 0.1832 | 28 | 1.1682 |
61
- | 1.149 | 0.2093 | 32 | 1.1674 |
62
- | 1.0952 | 0.2355 | 36 | 1.1541 |
63
- | 1.1551 | 0.2617 | 40 | 1.1687 |
64
- | 1.1864 | 0.2878 | 44 | 1.1547 |
65
- | 1.119 | 0.3140 | 48 | 1.1576 |
66
- | 1.141 | 0.3401 | 52 | 1.1748 |
67
- | 1.118 | 0.3663 | 56 | 1.1625 |
68
- | 1.1186 | 0.3925 | 60 | 1.1571 |
69
- | 1.1766 | 0.4186 | 64 | 1.1620 |
70
- | 1.0801 | 0.4448 | 68 | 1.1534 |
71
- | 1.0816 | 0.4710 | 72 | 1.1579 |
72
- | 1.087 | 0.4971 | 76 | 1.1575 |
73
- | 1.1822 | 0.5233 | 80 | 1.1619 |
74
- | 1.0812 | 0.5495 | 84 | 1.1607 |
75
- | 1.1626 | 0.5756 | 88 | 1.1611 |
76
- | 1.21 | 0.6018 | 92 | 1.1624 |
77
- | 1.1947 | 0.6280 | 96 | 1.1555 |
78
- | 1.1154 | 0.6541 | 100 | 1.1518 |
79
- | 1.1488 | 0.6803 | 104 | 1.1587 |
80
- | 1.1402 | 0.7065 | 108 | 1.1595 |
81
- | 1.0249 | 0.7326 | 112 | 1.1574 |
82
- | 1.1102 | 0.7588 | 116 | 1.1472 |
83
- | 1.1072 | 0.7850 | 120 | 1.1464 |
84
- | 1.1382 | 0.8111 | 124 | 1.1473 |
85
- | 1.1457 | 0.8373 | 128 | 1.1477 |
86
- | 1.156 | 0.8635 | 132 | 1.1483 |
87
- | 1.1037 | 0.8896 | 136 | 1.1488 |
88
- | 1.2025 | 0.9158 | 140 | 1.1492 |
89
- | 1.0551 | 0.9419 | 144 | 1.1496 |
90
- | 1.0823 | 0.9681 | 148 | 1.1499 |
91
- | 1.2344 | 0.9943 | 152 | 1.1499 |
92
 
93
 
94
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.1309
21
 
22
  ## Model description
23
 
 
37
 
38
  The following hyperparameters were used during training:
39
  - learning_rate: 0.0001
40
+ - train_batch_size: 16
41
+ - eval_batch_size: 16
42
  - seed: 42
43
+ - gradient_accumulation_steps: 4
44
  - total_train_batch_size: 64
45
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
  - lr_scheduler_type: cosine
 
51
 
52
  | Training Loss | Epoch | Step | Validation Loss |
53
  |:-------------:|:------:|:----:|:---------------:|
54
+ | 1.1655 | 0.0261 | 4 | 1.1621 |
55
+ | 1.0912 | 0.0523 | 8 | 1.1461 |
56
+ | 1.1696 | 0.0784 | 12 | 1.1691 |
57
+ | 1.0845 | 0.1046 | 16 | 1.1686 |
58
+ | 1.1548 | 0.1307 | 20 | 1.1627 |
59
+ | 1.0495 | 0.1569 | 24 | 1.1600 |
60
+ | 1.192 | 0.1830 | 28 | 1.1692 |
61
+ | 1.1397 | 0.2092 | 32 | 1.1666 |
62
+ | 1.0956 | 0.2353 | 36 | 1.1564 |
63
+ | 1.1748 | 0.2614 | 40 | 1.1647 |
64
+ | 1.1924 | 0.2876 | 44 | 1.1664 |
65
+ | 1.1258 | 0.3137 | 48 | 1.1596 |
66
+ | 1.1319 | 0.3399 | 52 | 1.1619 |
67
+ | 1.1099 | 0.3660 | 56 | 1.1577 |
68
+ | 1.122 | 0.3922 | 60 | 1.1573 |
69
+ | 1.1749 | 0.4183 | 64 | 1.1538 |
70
+ | 1.0708 | 0.4444 | 68 | 1.1579 |
71
+ | 1.0763 | 0.4706 | 72 | 1.1419 |
72
+ | 1.0635 | 0.4967 | 76 | 1.1494 |
73
+ | 1.1717 | 0.5229 | 80 | 1.1519 |
74
+ | 1.0674 | 0.5490 | 84 | 1.1404 |
75
+ | 1.1492 | 0.5752 | 88 | 1.1620 |
76
+ | 1.2029 | 0.6013 | 92 | 1.1477 |
77
+ | 1.1744 | 0.6275 | 96 | 1.1346 |
78
+ | 1.104 | 0.6536 | 100 | 1.1438 |
79
+ | 1.1398 | 0.6797 | 104 | 1.1436 |
80
+ | 1.1296 | 0.7059 | 108 | 1.1409 |
81
+ | 1.0167 | 0.7320 | 112 | 1.1469 |
82
+ | 1.1048 | 0.7582 | 116 | 1.1396 |
83
+ | 1.1004 | 0.7843 | 120 | 1.1358 |
84
+ | 1.1283 | 0.8105 | 124 | 1.1333 |
85
+ | 1.1287 | 0.8366 | 128 | 1.1322 |
86
+ | 1.1421 | 0.8627 | 132 | 1.1315 |
87
+ | 1.0848 | 0.8889 | 136 | 1.1303 |
88
+ | 1.184 | 0.9150 | 140 | 1.1300 |
89
+ | 1.0453 | 0.9412 | 144 | 1.1304 |
90
+ | 1.0604 | 0.9673 | 148 | 1.1307 |
91
+ | 1.2116 | 0.9935 | 152 | 1.1309 |
92
 
93
 
94
  ### Framework versions
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:cb9c595232822d46998f8b828ef93e9250fcef2cd06432352ccdfeab6c127c0c
3
  size 83945296
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:35fe02a9cfdf5a3ca67aa1af5060ddc9f81269a41fba3a89151b0bdd8446372d
3
  size 83945296