RAG Pipeline#

Blinko's Retrieval-Augmented Generation (RAG) pipeline lets the AI assistant answer questions grounded in the user's own notes and attachments. When a chat query arrives with withRAG: true (the default), the pipeline retrieves semantically relevant notes, injects their content into the model's system prompt, and streams the response back to the client.

Key entry points:

• AiService.completions() — the main chat handler; calls queryVector to build context before constructing the agent prompt.
• AiModelFactory.queryVector() — the retrieval core; embeds the query, searches the vector store, and returns matching notes.
• AiService.embeddingUpsert() — indexes note content into the vector store.
• AiService.embeddingInsertAttachments() — indexes attachment content into the vector store.

Vector Store#

Embeddings are stored in a local LibSQLVector database (@mastra/libsql) persisted at file:.blinko/vector/embeddings.db 1. The store is lazy-initialized on first access in AiUtilities.VectorStore(), which also creates the blinko index if it does not exist.

The index is always named blinko and uses cosine similarity. Dimensions are determined dynamically from the configured embedding model 2:

For non-standard models, users must set the dimension explicitly in AI Settings → Embed Settings → Advanced Settings 3.

Indexing: Notes and Attachments#

Note content#

embeddingUpsert() handles note indexing:

1. Tag exclusion: If excludeEmbeddingTagId is configured, notes containing that tag are skipped 4.
2. Chunking: Content is chunked via MDocument.fromMarkdown(content).chunk() (from @mastra/rag) 5.
3. Embedding: Each chunk is embedded with embedMany(), and the timestamp (createTime, updatedAt) is appended to the chunk text before embedding 6.
4. Upsert: Vectors are stored in the blinko index with metadata: { text, id, noteId, createTime, updatedAt } 7.
5. Status flag: On success, note.metadata.isIndexed is set to true 8.

On updates, the old vectors are deleted first via queryAndDeleteVectorById(), which runs a raw SQL DELETE against the LibSQLVector Turso client 9.

Attachment content#

embeddingInsertAttachments() handles file attachments:

• Images: Described via AiModelFactory.describeImage(), which sends the image to a vision-capable LLM and returns a textual description for embedding.
• PDFs: Loaded with PDFLoader (LangChain community).
• DOCX / DOC: Loaded with DocxLoader.
• TXT: Loaded with TextLoader.
• CSV: Loaded with CSVLoader.
• Other formats: Fall back to UnstructuredLoader 10.

The extracted text is chunked with MDocument.fromText(content).chunk(), embedded, and upserted with metadata { text, id, noteId, isAttachment: true, updatedAt } 11. A successful run sets note.metadata.isAttachmentsIndexed = true 12.

Retrieval and Context Construction#

queryVector(query, accountId) is the retrieval function:

1. Embed the query: Calls embed({ value: query, model: Embeddings }).
2. ANN search: Queries the blinko index with topK (default: embeddingTopK from global config, fallback 3) 13.
3. Score filter: Drops results below embeddingScore (default: 0.4) 14.
4. Account isolation: Fetches full note records from Postgres via Prisma, filtering by accountId and the IDs returned by the vector store — ensuring cross-user data leakage is impossible at the DB layer 15.
5. Returns: { notes, aiContext } — notes is the full note array with tags, attachments, references, and comment counts; aiContext is their content joined by newlines 16.

In completions(), the RAG context is assembled into a single system prompt string 17:

<prior system messages>
Current user name: <name>
<custom systemPrompt>
This is the note content <note bodies> <aiContext>

All prior system messages are merged into one to avoid provider-specific "multiple system messages" errors (e.g., Qwen/DashScope) 18.

Rebuilding the Embedding Index#

RebuildEmbeddingJob (at server/jobs/rebuildEmbeddingJob.ts) handles full and incremental index rebuilds:

• Full rebuild: Deletes and recreates the blinko index, then re-embeds all non-recycled notes and their non-image attachments in batches of 5 19.
• Incremental rebuild: Resumes from saved progress, skipping already-processed note IDs 20.
• Retry logic: Each note and attachment is retried up to 3 times on failure 21.
• Progress tracking: Progress is persisted to a cache table (key: "rebuild-embedding-progress") and updated after every note, enabling resume across restarts 22.
• Cancellation: A forceStopFlag is checked at the batch and note level to allow graceful shutdown 23.

Trigger via AiService.rebuildEmbeddingIndex() 24, which delegates to RebuildEmbeddingJob.ForceRebuild(force).

Note on images during rebuild: Image attachments are explicitly skipped (recorded as skip) during the rebuild job; they are only embedded on initial attachment insertion via describeImage().