Practical Day 2 – GRA 4164

<p class="kicker">GRA 4164 · Lecture 10 · Fall 2026</p>

## Practical Day 2

### Leif Anders Thorsrud & Vegard H. Larsen — BI Norwegian Business School

---

## How today works

- Three **workstreams** — pick your order, we circulate
- Everything runs on a **laptop**; the course server handles the bigger models
- Second half: **assignments 2–4** with your group

---

## Workstream 1: embeddings from scratch

- The **counting route**, end to end: SOTU corpus → co-occurrence → PPMI → SVD
- 50-dim vectors that know `war → enemy`, `gold → silver`, `railroad → canal`
- Pure **numpy** — the lecture-6 slides, executable

---

## Workstream 2: pre-trained models

```python
from transformers import pipeline
clf = pipeline("sentiment-analysis")
clf("The market reaction was surprisingly muted.")
# [{'label': 'NEGATIVE', 'score': 0.97}]
```

- Hugging Face `pipeline`: lecture 9's **route 1**, one line
- Then: **embeddings as features** — model vectors into your own classifier

---

## Workstream 3: the generation knobs

- Load a small generative model; sweep **temperature** 0.2 → 2.0 on one prompt
- Try **top-k** vs **top-p** on the same distribution
- Watch lecture 9's figure happen in your own output

---

## The course server

**Access details and credentials: on Itslearning today.** Open-weight models (Llama-class and friends) for workstream 2 and the assignments — nothing to install beyond an SSH client.

</div>

---

## Assignments 2–4

**Work block: your group, your assignment.** Assignment 2 leans on workstream 1 (embeddings); assignment 3 on workstream 2 (fine-tuning, incl. the course server); assignment 4 is lecture 11 material — start with its part 2 reflection. Presentations are next Tuesday — today is the time to get unstuck.

</div>

---

## In sum

- **Embeddings**: built from counts, by you, in numpy
- **Pre-trained models**: one-line classifiers, embeddings as features
- **Sampling knobs**: temperature and top-k/p, felt in your own output

</div>

---

## Questions

---

## What's next

- **Lecture 11 (Mon):** prompt engineering, alignment — and the **course summary**
- **Lecture 12 (Tue): presentations** — 6 minutes per group, allocation on Itslearning
- Reading: J&M chapters 8–10 continue to apply