notebooks Documentation

Notebooks using the Hugging Face libraries 🤗

4,651 stars Jupyter Notebook 1 file · ~26,747 tokens
RAW Doc

File: README.md

notebooks

Notebooks using the Hugging Face libraries 🤗

File: diffusers/README.md

Diffusers Notebooks

Notebooks for the HuggingFace Diffusers library

📓 Examples

Notebook Colab Description
Getting started with Diffusers First look at the library with a detailed model inference example.
Training a DDPM model An example of training a DDPM model from scratch on a dataset of butterfly images.
Stable Diffusion with Diffusers How to use Stable Diffusion model for text-to-image generation using diffusers library.

File: examples/idefics/README.md

IDEFICS Demos/examples

Inference

Finetuning

The following demos use the Image captioning task:

File: examples/idefics/idefics_zero3_finetuning/requirements.txt

torch
deepspeed
transformers>4.33.3
pillow
sentencepiece
protobuf
datasets
accelerate
mpi4py

File: sagemaker/README.md

Hugging Face Transformers Amazon SageMaker Examples

Example Jupyter notebooks that demonstrate how to build, train, and deploy Hugging Face Transformers using Amazon SageMaker and the Amazon SageMaker Python SDK.

🛠️ Setup

The quickest setup to run example notebooks includes:

📓 Examples

Notebook Type Description
01 Getting started with PyTorch Training Getting started end-to-end example on how to fine-tune a pre-trained Hugging Face Transformer for Text-Classification using PyTorch
02 getting started with TensorFlow Training Getting started end-to-end example on how to fine-tune a pre-trained Hugging Face Transformer for Text-Classification using TensorFlow
03 Distributed Training: Data Parallelism Training End-to-end example on how to use distributed training with data-parallelism strategy for fine-tuning a pre-trained Hugging Face Transformer for Question-Answering using Amazon SageMaker Data Parallelism
04 Distributed Training: Model Parallelism Training End-to-end example on how to use distributed training with model-parallelism strategy to pre-trained Hugging Face Transformer using Amazon SageMaker Model Parallelism
05 How to use Spot Instances & Checkpointing Training End-to-end example on how to use Spot Instances and Checkpointing to reduce training cost
06 Experiment Tracking with SageMaker Metrics Training End-to-end example on how to use SageMaker metrics to track your experiments and training jobs
07 Distributed Training: Data Parallelism Training End-to-end example on how to use Amazon SageMaker Data Parallelism with TensorFlow
08 Distributed Training: Summarization with T5/BART Training End-to-end example on how to fine-tune BART/T5 for Summarization using Amazon SageMaker Data Parallelism
09 Vision: Fine-tune ViT Training End-to-end example on how to fine-tune Vision Transformer for Image-Classification
10 Deploy HF Transformer from Amazon S3 Inference End-to-end example on how to deploy a model from Amazon S3
11 Deploy HF Transformer from Hugging Face Hub Inference End-to-end example on how to deploy a model from the Hugging Face Hub
12 Batch Processing with Amazon SageMaker Batch Transform Inference End-to-end example on how to do batch processing with Amazon SageMaker Batch Transform
13 Autoscaling SageMaker Endpoints Inference End-to-end example on how to use autoscaling for a HF Endpoint
14 Fine-tune and push to Hub Training End-to-end example on how to use the Hugging Face Hub as MLOps backend for saving checkpoints during training
15 Training Compiler Training End-to-end example on how to use Amazon SageMaker Training Compiler to speed up training time
16 Asynchronous Inference Inference End-to-end example on how to use Amazon SageMaker Asynchronous Inference endpoints with Hugging Face Transformers
17 Custom inference.py script Inference End-to-end example on how to create a custom inference.py for Sentence Transformers and sentence embeddings
18 AWS Inferentia Inference End-to-end example on how to AWS Inferentia to speed up inference time
19 Serverless Inference Inference Serverless Inference example to save cost
20 Automatic Speech Recognition Inference Example how to do speech recognition with wav2vec2
21 Image Segmentation Inference Example how to do image segmentation with segformer
22 Accelerate AWS SageMaker Integration examples Training End-to-end examples on how to use AWS SageMaker integration of Accelerate
23 Stable Diffusion Inference Example how to generate images with stable diffusion
24 Train BLOOM with PEFT Training Example how to train BLOOM on a single GPU using PEFT & LoRA
25 PyTorch FSDP model parallelism Training Example how to train LLMs on multi-node multi GPU with PyTorch FSDP
26 Document AI Donut Training In this tutorial, you will learn how to fine-tune and deploy Donut-base for document-understand/document-parsing using Hugging Face Transformers and Amazon SageMaker.
27 Deploy Large Language Models Inference Learn how to deploy LLMs with the Hugging Face LLM DLC
28 Train LLMs with QLora Training Example on how to fine-tune LLMs using Q-Lora
29 Deploy LLMs with Inferentia2 Inference Learn how to deploy LLMs using AWS Inferentia2
30 Evaluate LLMs with ligtheval Inference Learn how to evaluate LLMs using Hugging Face LightEval
31 Deploy Embedding Models with TEI Inference Learn how to deploy Embedding models for RAG applications with Hugging Face TEI
32 Train and deploy Embedding Models Train & Inference Learn how to train and deploy embedding models with Sentence Transformers and TEI

File: sagemaker/14_train_and_push_to_hub/README.md

SageMaker push to hf.co/models example

File: sagemaker/22_accelerate_sagemaker_examples/README.md

accelerate-aws-sagemaker

Examples showcasing AWS SageMaker integration of 🤗 Accelerate. Just give the accelerate config and do accelerate launch 🚀. It's as simple as that!

  1. Set up the accelerate config by running accelerate config --config_file accelerate_config.yaml and answer the SageMaker questions.

  2. Below is a sample config which is using aws profile to launch training job using 🤗 SageMaker estimator. It also has the iam_role_name which has the needed SageMaker permissions specified. In this config it is replaced xxxxx as user needs to specify it based on their corresponding AWS setup.

yaml
base_job_name: accelerate-sagemaker-1
compute_environment: AMAZON_SAGEMAKER
distributed_type: DATA_PARALLEL
ec2_instance_type: ml.p3.16xlarge
iam_role_name: xxxxx
image_uri: null
mixed_precision: fp16
num_machines: 1
profile: xxxxx
py_version: py38
pytorch_version: 1.10.2
region: us-east-1
transformers_version: 4.17.0
use_cpu: false

  1. One can specify a custom docker image instead of Official 🤗 DLCs through the accelerate config questionnaire. When this isn't provided, the latest Official 🤗 DLC will be used.

  2. Support for input channels pointing to S3 data locations via TSV file, e.g., below are the contents of sagemaker_inputs.tsv whose location is given as part of accelerate config setup.

tsv
channel_name	data_location
train	s3://sagemaker-sample/samples/datasets/imdb/train
test	s3://sagemaker-sample/samples/datasets/imdb/test
  1. Support for SageMaker metrics logging via TSV file, e.g., below are the contents of the sagemaker_metrics_definition.tsv whose location is given as part of accelerate config setup.
tsv
metric_name	metric_regex
accuracy	'accuracy': ([0-9.]+)
f1	'f1': ([0-9.]+)
  1. Example of accelerate config with above features setup [XXXXX values are AWS account specific]:
yaml
base_job_name: accelerate-sagemaker-1
compute_environment: AMAZON_SAGEMAKER
distributed_type: DATA_PARALLEL
ec2_instance_type: ml.p3.16xlarge
iam_role_name: XXXXX
image_uri: 763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-pytorch-training:1.8.1-transformers4.10.2-gpu-py36-cu111-ubuntu18.04
mixed_precision: fp16
num_machines: 1
profile: XXXXX
py_version: py38
pytorch_version: 1.10.2
region: us-east-1
sagemaker_inputs_file: sagemaker_inputs.tsv
sagemaker_metrics_file: sagemaker_metrics_definition.tsv
transformers_version: 4.17.0
use_cpu: false

  1. Put requirements.txt with all the needed libraries for running the training script.

  2. Running text-classification example using s3 datasets (from the root directory):

bash
cd src/text-classification
bash launch.sh

The contents of launch.sh

bash
accelerate launch  --config_file accelerate_config.yaml train_using_s3_data.py \
    --mixed_precision "fp16"

Output logs:

bash
...

[1,mpirank:0,algo-1]<stdout>:algo-1:79:1300 [0] NCCL INFO Launch mode Parallel
[1,mpirank:0,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:3,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:1,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:2,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:6,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:5,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:7,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:4,algo-1]<stderr>:INFO:root:Reducer buckets have been rebuilt in this iteration.
[1,mpirank:0,algo-1]<stdout>:epoch 0: {'accuracy': 0.6838235294117647, 'f1': 0.8122270742358079}
[1,mpirank:0,algo-1]<stdout>:epoch 1: {'accuracy': 0.7205882352941176, 'f1': 0.8256880733944955}
[1,mpirank:0,algo-1]<stdout>:epoch 2: {'accuracy': 0.75, 'f1': 0.838095238095238}
2022-09-21 13:21:05,187 sagemaker-training-toolkit INFO     Waiting for the process to finish and give a return code.
2022-09-21 13:21:05,188 sagemaker-training-toolkit INFO     Done waiting for a return code. Received 0 from exiting process.
2022-09-21 13:21:05,188 sagemaker-training-toolkit INFO     Reporting training SUCCESS
  1. Running seq2seq example:
bash
cd src/seq2seq
bash launch.sh

The contents of launch.sh

bash
accelerate launch --config_file accelerate_config.yaml run_seq2seq_no_trainer.py \
    --dataset_name "smangrul/MuDoConv" \
    --max_source_length 128 \
    --source_prefix "chatbot: " \
    --max_target_length 64 \
    --val_max_target_length 64 \
    --val_min_target_length 20 \
    --n_val_batch_generations 5 \
    --n_train 10000 \
    --n_val 1000 \
    --pad_to_max_length True\
    --num_beams 10 \
    --model_name_or_path "facebook/blenderbot-400M-distill" \
    --per_device_train_batch_size 16 \
    --per_device_eval_batch_size 16 \
    --learning_rate 1e-6 \
    --weight_decay 0.0 \
    --num_train_epochs 1 \
    --gradient_accumulation_steps 1 \
    --num_warmup_steps 100 \
    --output_dir "/opt/ml/model" \
    --seed 25 \
    --logging_steps 100 \
    --report_name "blenderbot_400M_finetuning"

Output logs:

text
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:37:39 - INFO - __main__ - Distributed environment: MULTI_GPU  Backend: smddp
[1,mpirank:0,algo-1]<stderr>:Num processes: 8
[1,mpirank:0,algo-1]<stderr>:Process index: 0
[1,mpirank:0,algo-1]<stderr>:Local process index: 0
[1,mpirank:0,algo-1]<stderr>:Device: cuda:0
[1,mpirank:0,algo-1]<stderr>:

...

[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ - ***** Running training *****
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Num examples = 10000
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Num Epochs = 1
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Instantaneous batch size per device = 16
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Total train batch size (w. parallel, distributed & accumulation) = 128
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Gradient Accumulation steps = 1
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:31 - INFO - __main__ -   Total optimization steps = 79

...

[1,mpirank:0,algo-1]<stderr>:#015100%|██████████| 79/79 [00:19<00:00,  4.79it/s]
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:50 - INFO - __main__ - Epoch 0 training took 19.50162172317505 seconds
[1,mpirank:0,algo-1]<stdout>:starting evaluation
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:55 - INFO - __main__ - printing few sample generations and corresponding labels from eval set
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:55 - INFO - __main__ - prompt | generated | label
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:55 - INFO - __main__ - chatbot:your persona: i work as an electrician. i always sleep 8 hours a day. </s>  <s> Which level are you at?</s> |  I'm at the top of the ladder. I work for an electrical company. |  I received on-the-job training when i first started
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:38:55 - INFO - __main__ - chatbot:your persona: i work as an electrician. i always sleep 8 hours a day. </s>  <s> Which level are you at? </s> <s> I received on-the-job training when i first started </s> <s> Thats great! How long have you been doing this work? </s> |  I've been working as an Electrician for about 5 years now. It's a great job. |  For a good number of years now.

... 

[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:39:02 - INFO - __main__ - chatbot:your persona: i'm a painter and love to create art. i'm a talented singer and have won several competitions. </s>  <s> I love the Doors! They have such a unique sound. Do you have a favorite Doors song?</s> |  My favorite song of theirs is "When I Was Your Man". What's yours? |  It's a tie between People are strange and Love me two times. What's your favorite?
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:39:02 - INFO - __main__ - chatbot:your persona: i'm a painter and love to create art. i'm a talented singer and have won several competitions. </s>  <s> I think my favorite is Love Street. It has such a haunting melody. Have you heard that one?</s> |  No, I haven't. I'll have to check it out. What genre is it? |  Yeah, it's a pretty great song, Jim Morrison was just an amazing songwriter, I aspire to make songs as good as his one day.

...

[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:39:17 - INFO - __main__ - {'bleu': 1.7067114414104911}
[1,mpirank:0,algo-1]<stdout>:evaluation completed
[1,mpirank:0,algo-1]<stderr>:09/21/2022 13:39:17 - INFO - __main__ - Epoch 0 evaluation took 24.294514417648315 seconds
[1,mpirank:0,algo-1]<stderr>:Configuration saved in /opt/ml/model/config.json
[1,mpirank:0,algo-1]<stderr>:Model weights saved in /opt/ml/model/pytorch_model.bin
[1,mpirank:0,algo-1]<stderr>:tokenizer config file saved in /opt/ml/model/tokenizer_config.json
[1,mpirank:0,algo-1]<stderr>:Special tokens file saved in /opt/ml/model/special_tokens_map.json
[1,mpirank:0,algo-1]<stderr>:#015100%|██████████| 79/79 [00:47<00:00,  1.65it/s]
2022-09-21 13:39:27,753 sagemaker-training-toolkit INFO     Waiting for the process to finish and give a return code.
2022-09-21 13:39:27,753 sagemaker-training-toolkit INFO     Done waiting for a return code. Received 0 from exiting process.
2022-09-21 13:39:27,754 sagemaker-training-toolkit INFO     Reporting training SUCCESS

File: sagemaker/22_accelerate_sagemaker_examples/src/seq2seq/accelerate_config.yaml

base_job_name: accelerate-sagemaker-1
compute_environment: AMAZON_SAGEMAKER
distributed_type: DATA_PARALLEL
ec2_instance_type: ml.p3.16xlarge
iam_role_name: xxxxx
image_uri: null
mixed_precision: fp16
num_machines: 1
profile: xxxxx
py_version: py38
pytorch_version: 1.10.2
region: us-east-1
transformers_version: 4.17.0
use_cpu: false

File: sagemaker/22_accelerate_sagemaker_examples/src/seq2seq/requirements.txt

accelerate # used to be installed in Amazon SageMaker environment
evaluate
datasets==2.3.2
nltk
deepspeed
sentencepiece != 0.1.92
protobuf
sacrebleu >= 1.4.12
nltk
py7zr
torch >= 1.3
transformers==4.22.1

File: sagemaker/22_accelerate_sagemaker_examples/src/text-classification/accelerate_config.yaml

base_job_name: accelerate-sagemaker-1
compute_environment: AMAZON_SAGEMAKER
distributed_type: DATA_PARALLEL
ec2_instance_type: ml.p3.16xlarge
iam_role_name: xxxxx
image_uri: null
mixed_precision: fp16
num_machines: 1
profile: xxxxx
py_version: py38
pytorch_version: 1.10.2
region: us-east-1
sagemaker_inputs_file: sagemaker_inputs.tsv
sagemaker_metrics_file: sagemaker_metrics_definition.tsv
transformers_version: 4.17.0
use_cpu: false

File: sagemaker/22_accelerate_sagemaker_examples/src/text-classification/requirements.txt

accelerate # used to be installed in Amazon SageMaker environment
evaluate
datasets==2.3.2
transformers

File: sagemaker/23_stable_diffusion_inference/code/requirements.txt

diffusers==0.6.0
transformers==4.23.1

File: sagemaker/24_train_bloom_peft_lora/scripts/requirements.txt

peft==0.3.0
transformers==4.27.1
accelerate==0.17.1
bitsandbytes==0.37.1

File: sagemaker/25_pytorch_fsdp_model_parallelism/scripts/requirements.txt

transformers==4.28.1

File: sagemaker/28_train_llms_with_qlora/scripts/requirements.txt

git+https://github.com/huggingface/transformers.git@2ab75add4b30c2fc44a8bf575156d448d9ed87a7

transformers==4.31.0

peft==0.4.0
accelerate==0.21.0
bitsandbytes==0.40.2
safetensors>=0.3.1
tokenizers>=0.13.3

File: sagemaker/32_train_deploy_embedding_models/scripts/requirements.txt

transformers==4.41.2
accelerate==0.31.0
sentence-transformers==3.0.0
datasets==2.18.0
torch==2.1.2

File: transformers_doc/README.md

🤗 Transformers doc notebooks

These notebooks are automatically generated from the 🤗 Transformers documentation
so you should not make any direct modification here. If there is a typo to fix or a sentence to add, open a pull
request in the 🤗 Transformers repo and fix the corresponding file in
the docs/source/ folder.

If there is something that seems weirdly converted from the original doc file, open an issue in this repo and we will
try to fix the conversion script.

Note: If you are looking for a particular notebook and can't find it, it may have moved inside the en subfolder.

File: transformers_doc/en/tensorflow/README.md

[!WARNING]
TensorFlow is deprecated for Transformers and no longer actively maintained. It will be removed in v5. We recommend pinning or installing a version of Transformers that supports TensorFlow to continue using these notebooks.

File: .github/pull_request_template.md

What does this PR do?

Fixes # (issue)

Who can review?

Feel free to tag members/contributors who may be interested in your PR.

Explore Related Technologies & Topics

View All 9,111+ Repositories →

Discover similar high-velocity repositories, agent skills, and OpenAPI specifications across the ecosystem.

⚡ MCP Servers Model Context Protocol → 🧠 Agent Skills Prompt & rule files → 🔌 OpenAPI Specs REST & Swagger schemas →