Personal knowledge & retrieval
Second Memory
Personal projectDesign & engineering
Self-hosted application
A useful memory for the web.
Second Memory is my personal application for saving articles, videos, posts, and repositories, then finding them again. It brings a FastAPI backend and PostgreSQL/pgvector retrieval together with a Flutter web and locally configured Android client.
View source on GitHubProduct behavior
Saving useful context, not just links
Saving a URL extracts public metadata/descriptions and produces editable titles, summaries, and tags. Tags can be reused, listed, renamed, or removed. The Flutter client supports web and locally configured Android share intents; saved items can be filtered by type and tags, with dates supported by the list API.
JSON and CSV import/export skip duplicates. Current ingestion works with metadata and summaries: it does not transcribe videos or index complete article bodies.
Hybrid search
Combining semantic and keyword retrieval
The backend uses PostgreSQL 16 and pgvector. Gemini gemini-embedding-001 produces 768-dimensional embeddings with RETRIEVAL_DOCUMENT for saved titles/summaries and RETRIEVAL_QUERY for questions. Cosine-similarity candidates and PostgreSQL full-text results are fused with Reciprocal Rank Fusion, using constant 60.
Account ownership and tag/type filters are applied inside database queries before candidate limits. Specified embedding-provider failures fall back to keyword search; database errors still propagate. Search is over saved metadata/summaries, without a claim of passage-level document QA.
Tool-using assistant
References grounded in authorized results
A LangGraph/LangChain assistant uses search and listing tools, with hosted Groq inference and the current documented default openai/gpt-oss-120b. SSE streams answers to Flutter. The server resolves saved-item cards from authorized tool results, discarding invented IDs and duplicate references.
Backend work includes Pydantic schemas/settings, SQLAlchemy asynchronous access, asyncpg, and Alembic migrations. The Flutter client uses Dart, Riverpod, go_router, and Freezed. Tests cover ranking, filters, ownership, provider fallback, streaming, and reference validation.
Personal deployment / evaluation
Delivery and a small retrieval check
Docker Compose and migrations support self-hosting, with a documented VPS deployment path. GitHub Actions is documented for backend/Flutter tests, analysis, and the release web build. The application and database are self-hosted; Groq inference and Google embeddings remain external dependencies.
A recorded September 11, 2026 smoke run used eight manually labeled queries over eight synthetic notes: Recall@5 and MRR@5 were both 1.0, with median API latency 324.7 ms. This was one sequential pass with real Google embeddings after warmup/import, not a general accuracy, production-scale, or baseline-superiority result.
- FlutterWeb + Android client
URL saving / share intents - FastAPIMetadata & summaries
Tags, import / export - PostgreSQL + pgvectorVector + full-text retrieval
Reciprocal Rank Fusion - LangGraphSearch & listing tools
Streaming answers + references
Application deployed with Docker Compose. Hosted Groq inference and Gemini embeddings supply the model APIs.
Watch the 48-second walkthrough & see the assistant
A recorded walkthrough of the running application: search, note details, and an assistant response with retrieved sources. Uses eight synthetic sample notes; pauses are trimmed. Silent video, with the steps described below.
- Browse the saved-note library.
- Search with a paraphrased question and open the matching note.
- Ask the assistant to compare two notes and inspect its source cards.
