p-tuning-roberta-large-with-mrpc

This model is a fine-tuned version of roberta-large on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5925
• Accuracy: 0.6957
• F1: 0.7992

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	F1
No log	1.0	58	0.6199	0.6754	0.8003
No log	2.0	116	0.6082	0.6939	0.8066
No log	3.0	174	0.6217	0.6725	0.8013
No log	4.0	232	0.6317	0.6667	0.7993
No log	5.0	290	0.6402	0.6499	0.7703
No log	6.0	348	0.6219	0.6678	0.7994
No log	7.0	406	0.6287	0.6684	0.7556
No log	8.0	464	0.5914	0.6991	0.8051
0.6347	9.0	522	0.5826	0.6875	0.8022
0.6347	10.0	580	0.6129	0.6875	0.7764
0.6347	11.0	638	0.5942	0.6922	0.8036
0.6347	12.0	696	0.5975	0.6887	0.7822
0.6347	13.0	754	0.5943	0.6899	0.7963
0.6347	14.0	812	0.5863	0.6968	0.8039
0.6347	15.0	870	0.5880	0.6945	0.7998
0.6347	16.0	928	0.5967	0.6858	0.7982
0.6347	17.0	986	0.5951	0.6968	0.8020
0.5717	18.0	1044	0.5931	0.7043	0.7981
0.5717	19.0	1102	0.5936	0.6974	0.8006
0.5717	20.0	1160	0.5925	0.6957	0.7992

Framework versions

• PEFT 0.7.1
• Transformers 4.36.2
• Pytorch 2.0.0
• Datasets 2.15.0
• Tokenizers 0.15.0