Edit model card

Model Card for Model ID

This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct on tatsu-lab/alpaca dataset.

Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instruction better.

Model Details

Model Description

This is a fine-tuned version of the meta-llama/Meta-Llama-3-8B-Instruct model using Parameter Efficient Fine Tuning (PEFT) with Low Rank Adaptation (LoRA) on the Intel Gaudi 2 AI accelerator. This model can be used for various text generation tasks including chatbots, content creation, and other NLP applications.

Uses

Direct Use

This model can be used for text generation tasks such as:

  • Chatbots
  • Automated content creation
  • Text completion and augmentation

Out-of-Scope Use

  • Use in real-time applications where latency is critical
  • Use in highly sensitive domains without thorough evaluation and testing

Recommendations

Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.

Training Details

Training Hyperparameters

  • Training regime: Mixed precision training using bf16
  • Number of epochs: 3
  • Learning rate: 1e-4
  • Batch size: 16
  • Seq length: 512

Technical Specifications

Compute Infrastructure

Hardware

  • Intel Gaudi 2 AI Accelerator
  • Intel(R) Xeon(R) Platinum 8380 CPU @ 2.30GHz

Software

  • Transformers library
  • Optimum Habana library

Environmental Impact

Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).

  • Hardware Type: Intel Gaudi 2 AI Accelerator
  • Hours used: < 1 hour
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference API
Unable to determine this model’s pipeline type. Check the docs .

Dataset used to train migaraa/Alpaca_Llama-3-8B-Instruct_Gaudi