Assignment 2: Word embeddings

The task

Still on the State of the Union corpus, you build word embeddings several ways and ask what they know — the lecture 6 toolkit end to end.

Part 1 — Training embeddings. Train word2vec on the corpus (justify vector size, window, and frequency cutoffs); re-train on the unpreprocessed corpus and compare cosine-similarity tables for five common words; repeat the comparison with pre-trained GloVe vectors; and finally build embeddings the counting way — a PMI matrix + SVD — and compare once more.

Part 2 — Vector arithmetic. Perform analogy tasks (king − man + woman ≈ queen) with at least three analogies relevant to the corpus, and explore nearest neighbours of chosen seed words — including the unexpected ones, with an explanation.

Part 3 — Visualisation. Reduce embeddings to 2D (PCA or t-SNE) and plot annotated clusters of semantically related words.

Part 4 — Semantic change over time (flexible: code or conceptual). Compare word meanings across periods (e.g. 1800–1900 vs 1900–2000) and quantify or describe the shifts. Closing question: as an advisor to a presidential campaign, how would you use these findings to inform language strategy?

Materials

• Notebook: assignment2.ipynb — reference code; rearrange and supplement it, or write your own
• Data: sotu.zip (same corpus as assignment 1)
• The practical-day-2 notebook Embeddings_from_scratch.ipynb is a working template for the PMI+SVD route

Deliverable

A report with your comparisons, visualisations, and reflections, organised by part. Groups of 1–3; part of the collected end-of-semester submission. Format details on Itslearning.

Builds on

Lecture 6; practical day 2’s embeddings workstream is the supervised session for this assignment.