Assignment 3: Fine-tuning a language model
Fine-tune BERT for text classification — first on Yelp reviews, then on your own State of the Union sentiment labels — and benchmark it against the simple baselines
The task
Lecture 9’s route 1, for real: adapt a pre-trained transformer to a classification task and find out what it buys you over simpler methods.
Exercise 1 — Yelp reviews. Load the yelp_review_full dataset
(five-star review ratings), fine-tune a pre-trained bert-base-cased (or an
alternative) for 5-class classification, evaluate, and reflect on strengths
and weaknesses.
Exercise 2 — Your own sentiment data. Reuse the sentiment labels your group produced in assignment 1 (manual and/or dictionary-scored State of the Union paragraphs): build a custom dataset, fine-tune BERT for sentiment classification, and evaluate with the same metrics. Then the real question: benchmark against the simple approaches — VADER, Boolean search, a logistic regression — on the same data. Where does the transformer earn its compute, and where do the baselines hold up?
Practical notes
- Training on a CPU takes ~30 minutes on the demonstration subset (1,000 samples); the course server is available for bigger runs — access was covered on practical day 2
- The model (~440 MB) and dataset download from Hugging Face on first run
Materials
- Notebook: assignment3.ipynb — a complete fine-tuning loop for Exercise 1; Exercise 2’s dataset and baselines are yours to build
- Your assignment-1 sentiment scores
Deliverable
A report with training results, evaluation metrics, the baseline comparison, and your discussion of when fine-tuning is worth it. Groups of 1–3; part of the collected end-of-semester submission. Format details on Itslearning.
Builds on
Lectures 8–9; the Hugging Face chapters from the asynchronous work; practical day 2’s pre-trained-models workstream.