Iris: a local-first meeting notetaker that is growing into a second brain

This is a submission for the Hacktoberfest Open-Source AI Challenge Week 3: Slow and Steady
Iris is a local-first meeting notetaker that is growing into a second brain. It records when you tell it to, transcribes on your own machine with open-weight Whisper, writes the notes with a local open-weight model, and keeps everything as plain markdown on your disk.
The effort it targets is the hour after every meeting – writing up what was decided, copying action items somewhere, filing the notes where you will find them later. Iris does that part:
Meetings become notes on their own. A watch daemon notices when Zoom/Meet/Teams/Webex opens and nudges you; recording starts only with your tap. After the meeting, notes with decisions, action items and owners are written locally. Recordings auto-delete after 7 days (RETENTION_DAYS in the store); transcripts and notes stay.

Notes connect into a personal wiki. Write [[Roadmap]] in a note and it becomes a real link: the other note gets a “Linked from” backlink, and a graph view maps every note and link – the Notion/Obsidian pattern, over local markdown files.

Read your notes anywhere, keep them yours. An optional hosted viewer syncs notes between devices and a web page. Sync is off until you link a device; it moves notes only by default (transcripts are a separate opt-in) and never audio. Any synced note can then be encrypted in the browser with a passphrase the site never sees – WebCrypto AES-GCM, PBKDF2-SHA256 at 600,000 iterations. After that the database holds only ciphertext for the body, title and transcript; the graph and backlinks stop reading the note. The page states the caveats plainly: encryption starts from the moment you turn it on (the note synced as plaintext first, and overwriting the stored row cannot promise that every older backup or log copy is gone), the note’s existence and timestamps stay visible, and a lost passphrase means the note cannot be recovered.

Every claim has a receipt. Each decision and action item must quote the exact transcript lines it came from, with a timestamp. A verifier checks every claim before the note is saved; a claim the model cannot ground is demoted to an “unverified” section instead of silently becoming a to-do.

Code-mixed audio is handled. Per-segment language detection transcribes a meeting that switches between English and Hindi mid-sentence. Tested on synthetic code-mixed audio, with small-model word errors on names documented in TESTED.md; real-meeting reliability is not verified yet. Speaker labels and voice enrollment also run locally – voiceprints are 192-d embeddings on disk, no audio kept.

Demo

Live cloud viewer: https://iris-cloud-one.vercel.app (sign up, or try the recorder PWA). Local app: install, then iris watch and iris ui.
https://github.com/h55n/iris (local app + self-host relay) https://github.com/h55n/iris-cloud (the hosted viewer)
The default pipeline is open-weight models running where the audio already is: whisper.cpp (tiny/base/small) for transcription with silero VAD sentence timestamps, and Gemma 3 4B via Ollama as the default notes model. Speaker diarization is pyannote segmentation plus ERes2Net speaker embeddings through sherpa-onnx – local, no account.
The local web app is a zero-dependency Python server (stdlib http.server) on 127.0.0.1. The hosted viewer is FastAPI on Vercel’s free tier with a free-tier Postgres; its notes pipeline uses a hosted open-weight model as an explicit fallback path. Both repos ship test suites (261 local tests, 47 hosted tests) and the hosted side is verified with browser end-to-end runs against the real deployment.
Meeting audio is among the most private data people produce: client names, salaries, health, unreleased plans. A hosted notetaker means that audio leaves your machine. Iris is possible because the models are open: Whisper’s weights are MIT and Gemma’s are open, so record, transcribe and summarize can all run where the audio already is. On the default path there is no vendor in the loop, no per-minute bill, and it works offline. The optional hosted pieces stay explicit and inspectable – a sync you turn on, a fallback model that is logged per run – because open weights are what make that honesty cheap.
Best Use of Gemma – notes are written by gemma3:4b running locally via Ollama
Best Use of Ultramarine – the local app’s hosted fallback notes backend (UltramarineBackend in iris/backends.py), with usage logged locally per run