Back to case studies

Case study

BardMatch: searching Shakespeare by meaning

A self-initiated AI project: give it any line, for example a song lyric, and it finds the closest lines in Shakespeare’s plays and sonnets by meaning. A later feature, the Canon Map, does the same for themes, with an LLM at two points in the search. This study focuses on two decisions about how far to trust what the models return.

Client

[N/A]

Stack

PythonVueOpenAI APIChromaDB
BardMatch home page

Problem

BardMatch started as a way to put vector embeddings to work on something interesting: matching a line from elsewhere, such as a song lyric, to the line in Shakespeare closest to it in meaning. Keyword search can’t do that, because the two lines rarely share words. Asking an LLM directly isn’t reliable either. It can’t check every line of the canon, so it misses things, and it can misquote or invent lines. Embeddings compare meaning across every stored passage and only return lines that really exist, so a modern lyric about heartbreak can land on a speech that never uses the word.

The Canon Map extended the same idea from single lines to themes. You type a theme such as jealousy, and it shows where that theme gathers across the plays, act by act. Themes turned out to be the harder case, and that’s where the decisions in this study come from.

Challenge

To give each query more to go on, BardMatch has an LLM expand it into several related phrases before searching, which also adds a little variety to the results. The catch is that the model’s choice of wording shapes what comes back. That mattered more once the Canon Map added theme searches. With a single word like “jealousy”, the expansion is most of what the search has to work with, so I tested how much its wording affected the results.

The second difficulty only affected the Canon Map. When you’re searching for a theme, being close in meaning isn’t the same as being about it. A passage can carry the language of a theme without the theme being the business of the scene. Any extra model step to sort that out would add cost and time to every search, so it had to stay light.

Solution

The plays and sonnets are cut into overlapping five-line passages, de-duplicated and embedded with OpenAI’s API. Each passage is stored in ChromaDB with its play, act, scene, speaker and line numbers. At search time the expanded phrases are embedded and compared against every stored passage. For the Canon Map, the closest matches have near-duplicates removed and are cut to the top 150 play passages, then drawn as a grid of plays against acts, shaded by how many strong matches land in each cell.

From the start, the expansion had a “Skew by” setting with three options. Emotion leans it towards the feeling of a line, Meaning towards what the line actually says, and Neutral sits in between. So a sad lyric on Emotion can match any grief-stricken speech, while on Meaning it looks for one about the same situation. I tested each setting on eight themes in the Canon Map, against the plays you’d expect them to find. For these theme searches, the Emotion setting never put an expected play at the top. It was turning a query like “jealousy” into generic poetic language, and the theme got lost along the way. The Meaning setting did best, so I made it the Canon Map’s default. The test also showed that single-word themes are the hardest kind of query for the index, even on the best setting, which is worth knowing when reading the Canon Map.

In the Canon Map, a proverb, a metaphor or a passing remark can show up as a match because it uses the language of jealousy, in a scene that’s concerned with something else. To catch those, I added an optional check. An LLM reads each matched passage, along with its play, act, scene and speaker, and labels it as central (the theme is a real concern of the moment), related (shared language or a nearby theme) or incidental (a throwaway or figure of speech). The labels appear alongside the results. Because this adds model calls to every search, passages go in batches of 40 to a small, low-cost model, with a progress indicator in the UI and size limits on the server.

Outcome

BardMatch is live at bardmatch.com, where you can try both line matching and the Canon Map.

The context check has one known limit. It judges each passage on its own, with its play, act, scene and speaker, but not the lines around it. Every passage already stores its line numbers, so giving the check the surrounding lines is a modest change, and it’s the first improvement planned for version two.