Add Model Evals (#1)
Browse files- Add Model Evals (976a1d31df5617e1288ee7cacd53fdfa24b51de7)
- Add Lambada OpenAI to evals (f7d8d88576e15c03220895f0fe65666b2f9618c7)
Co-authored-by: USVSN Sai Prashanth <[email protected]>
README.md
CHANGED
|
@@ -1 +1,18 @@
|
|
| 1 |
-
wandb run: https://wandb.ai/eleutherai/pythia-rlhf/runs/e0drjcsz?workspace=user-yongzx
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
wandb run: https://wandb.ai/eleutherai/pythia-rlhf/runs/e0drjcsz?workspace=user-yongzx
|
| 2 |
+
|
| 3 |
+
Model Evals:
|
| 4 |
+
| Task |Version|Filter| Metric |Value | |Stderr|
|
| 5 |
+
|-------------|-------|------|--------|-----:|---|-----:|
|
| 6 |
+
|arc_challenge|Yaml |none |acc |0.1877|± |0.0114|
|
| 7 |
+
| | |none |acc_norm|0.2372|± |0.0124|
|
| 8 |
+
|arc_easy |Yaml |none |acc |0.4390|± |0.0102|
|
| 9 |
+
| | |none |acc_norm|0.4082|± |0.0101|
|
| 10 |
+
|logiqa |Yaml |none |acc |0.1889|± |0.0154|
|
| 11 |
+
| | |none |acc_norm|0.2473|± |0.0169|
|
| 12 |
+
|piqa |Yaml |none |acc |0.6213|± |0.0113|
|
| 13 |
+
| | |none |acc_norm|0.6279|± |0.0113|
|
| 14 |
+
|sciq |Yaml |none |acc |0.7230|± |0.0142|
|
| 15 |
+
| | |none |acc_norm|0.6840|± |0.0147|
|
| 16 |
+
|winogrande |Yaml |none |acc |0.5162|± |0.0140|
|
| 17 |
+
|lambada_openai|Yaml |none |perplexity|58.9478|± |2.7662|
|
| 18 |
+
| | |none |acc | 0.2602|± |0.0061|
|