3 ms·

Show HN: Building Table Canon, an AI Campaign Memory Engine for TTRPGs

Hey HN! I built Table Canon to solve a problem my playgroup kept running into: 3-4 hour tabletop gaming sessions leave behind massive audio recordings, but standard meeting note-takers treat every session as an isolated island, butcher fantasy terms, and don't know who is speaking. I wanted an engine that tracks long-term state across months of games, so I built a pipeline to extract entity updates, open quest hooks, and character promises across sessions. The Tech Stack: * Transcription: whisper-large-v3-turbo * Diarization: pyannote for speaker embeddings & voice profile matching Extraction & Memory: OpenAI API with Structured Outputs (JSON Schema enforcement for state updates) * TTS & Audio Recaps: Kokoro / Chatterbox Turbo Music Generation: ACE-Step-v1.5-XL-Turbo for rendering session summaries into lyrics/ballads A Few Engineering Lessons & Challenges: * State Delta Extraction vs. Context Explosions: Feeding 20 prior session transcripts into context windows quickly becomes cost-prohibitive and noisy. Instead of re-reading raw history, each session outputs an atomic state delta (updates to NPC dossiers, new locations, resolved promises) to a database. Keeping context bounded as campaigns stretch past session 30+ has been one of the trickiest architectural hurdles. * Custom Pre-Lexicons: General STT models struggle with homebrew proper nouns (turning fantasy names into standard dictionary words). Injecting a pre-pass fantasy term dictionary into prompt context significantly improved first-pass spelling. * VAD & Audio Chunking: Passing a 4-hour raw audio file directly to Pyannote/Whisper leads to memory leaks and process timeouts. Pre-processing with Voice Activity Detection (VAD) and deterministic chunking was necessary before touching the models. Current Limitations & Active Hard Problems: * Entity Alias Resolution: Matching entities across sessions when players use varying aliases or informal shorthand (e.g., matching "The Red Bishop" to "Arthur" or "that cult leader guy") without accidentally merging distinct NPCs. I address this, partially, but allowing the user to Edit aliases, merge or split entities after-the-fact. * Quest & Hook Resolution Logic: Fine-tuning the LLM to reliably determine whether a promise, open mystery, or quest has actually been resolved versus remaining open or implicitly abandoned. I'd love feedback on how others are handling these sorts of issues - or any notes for folks who try it out! No initial login required with 6 hours of upload available to try.

jermaustin1 1mo ago
How good does it filter out the "out of context" audio? My party's sessions are 4-6 hours once a month, an I would say only around 45-90 minutes are the actual campaign, but spread across that entire time.
- schillingderek 1mo ago
  Great question! If you have a recorded session, you can try the website with up to 6 hours of audio, no login required, I’d love to hear how it goes! In my own testing I haven’t used audio with quite that high a “table talk” ratio, but in general I’ve found the system is quite good at filtering out anything that’s not “ttrpg-ish” in the transcript. I’d add, there is basically a built in aspect that tries to identify table talk, and Ive played around with features that will attempt to look for “amusing” bits within that to call out. Not currently live, but the structure to filter things that way is still embedded.
ipotapov 1mo ago
[dead]

jermaustin1 1mo ago

How good does it filter out the "out of context" audio? My party's sessions are 4-6 hours once a month, an I would say only around 45-90 minutes are the actual campaign, but spread across that entire time.

schillingderek 1mo ago
Great question! If you have a recorded session, you can try the website with up to 6 hours of audio, no login required, I’d love to hear how it goes! In my own testing I haven’t used audio with quite that high a “table talk” ratio, but in general I’ve found the system is quite good at filtering out anything that’s not “ttrpg-ish” in the transcript. I’d add, there is basically a built in aspect that tries to identify table talk, and Ive played around with features that will attempt to look for “amusing” bits within that to call out. Not currently live, but the structure to filter things that way is still embedded.

ipotapov 1mo ago

[dead]