SIFT Toolbox User Guide#

The SIFT Toolbox is an AI-powered fact-checking assistant built around the SIFT methodology. This guide covers the core methodology, report formats, prompt system, source assessment, and chat commands.

1. The SIFT Methodology#

The Four Steps#

SIFT is a four-step framework for fact-checking and evaluating information 1. Each step builds on the previous to create a systematic approach to information verification:

How the Toolbox Implements Each Step#

The SIFT Toolbox operationalizes each step through dedicated features:

Stop & Investigate (S, I)

• The Assessment of Source Reliability section in Full Check reports evaluates source usefulness, expertise, and credibility 6
• The source assessment sidebar provides at-a-glance reliability ratings for every cited source 7

Find (F)

• The Another Round command finds additional sources—including conflicting perspectives—to surface broader viewpoints 8
• The Web Search command supports lateral reading by searching for coverage across multiple trusted sources 9

Trace (T)

• The Trace Claim command uses gathered sources to track claims back to their earliest identifiable origin 9
• Reports enforce structured citation practices that point back to original sources 10

2. Report Formats#

The SIFT Toolbox provides three distinct report formats 11, each suited to a different fact-checking scenario. The Full Check is the default for new analyses 12; the other two are triggered via chat commands or buttons in the interface 13.

2.1 Full Check#

The Full Check is the most comprehensive format for systematic fact verification 14.

Best for:

• In-depth, claim-by-claim verification of complex topics
• Detailed fact-checking of images or artifacts
• Situations requiring authoritative source credibility assessment

Report structure — eight sections in strict order 6:

Notable features:

• Evidence is categorized by type—Documentation, Personal Testimony, Statistics, Analysis, Reporting, or Common Knowledge—and evaluated on a 1–5 credibility scale 15
• Claims are assessed using the Toulmin Analysis Framework for systematic argumentation analysis 16
• The AI reasons internally in a <think> block before producing the final structured output 17

2.2 Context Report#

The Context Report provides structured contextual analysis for artifacts, images, or claims that circulate online with misleading framing 18.

Best for:

• Explaining viral artifacts or memes
• Providing background on quotes or images stripped of their original context
• Creating educational summaries about commonly misunderstood topics

Report structure — seven question-based sections 19:

Notable features:

• Carefully distinguishes "commonly presented as" (requires multiple citations) from "has been presented as" (single example sufficient) 27
• All URLs must be direct final links—no redirect links 28
• No additional sections beyond the specified format are permitted 29

2.3 Community Note#

The Community Note produces a concise, Twitter/X Community Notes–style annotation focused on the critical context without which an artifact would be seriously misunderstood 30.

Best for:

• Quick public corrections on social media
• Shareable annotations for viral content
• Communicating essential context in character-limited environments

Report structure — two components 31:

Notable features:

• The AI internally applies the full Context Report structure but surfaces only the essential note 30
• Strictly excludes finer details to stay within the 700-character limit 34
• Uses bare URLs rather than Markdown-formatted links 35

Format Comparison#

3. Prompt Engineering System#

The SIFT Toolbox guides AI behavior through a layered prompt engineering system: specialized prompt templates, a dynamic construction pipeline, intelligent context management, and provider-specific delivery.

3.1 System Prompt Architecture#

The toolbox maintains six named prompt constants 36:

Conversational system prompts — set the AI's persona and behavioral defaults for chat sessions:

The default system instruction exposed to non-Gemini providers is: "You are a SIFT (Stop, Investigate, Find, Trace) methodology assistant. You help users fact-check claims, understand context, and analyze information." 40

Report generation prompts — injected as the initial user-turn for structured report requests:

3.2 Prompt Construction Pipeline#

Every initial analysis query passes through a five-stage assembly pipeline before reaching the AI provider:

Stage 1 — Prompt Selection

The constructFullPrompt function 44 picks the appropriate template based on ReportType:

• ReportType.FULL_CHECK → SIFT_FULL_CHECK_PROMPT
• ReportType.CONTEXT_REPORT → SIFT_CONTEXT_REPORT_PROMPT
• ReportType.COMMUNITY_NOTE → SIFT_COMMUNITY_NOTE_PROMPT

Stage 2 — Date Injection

The current date is substituted into the template so the AI can situate claims in time 45:

new Date().toLocaleDateString('en-US', {
  year: 'numeric',
  month: 'long',
  day: 'numeric'
})

Stage 3 — Query Appending

The user's initial query is appended at the end of the prompt 46:

"User's initial query: " + userQuery

Stage 4 — System Prompt Resolution

The getSystemPromptForSelectedModel function resolves which system prompt to use, following a four-level precedence order 47:

1. Custom prompt override — user-supplied custom prompt takes highest priority 48
2. Model-specific default — the model's configured defaultSystemPrompt, if set 49
3. Provider-specific fallback — for OpenAI, OpenRouter, and Mistral, a concise summarized prompt 50
4. Base default — SIFT_CHAT_SYSTEM_PROMPT 51

Stage 5 — Command Processing

When a user invokes a command, the message is augmented before dispatch 52:

• Built-in commands: the command name is prefixed as [COMMAND: command_name] 53
• Custom commands: the command's prompt instructions are prepended to the user input, and any command-specific model parameters override the defaults 54

3.3 Chat History and Context Management#

To keep context relevant without overloading the context window, the getTruncatedHistoryForApi function applies an intelligent retention strategy 55:

• Always retains the first user query and first AI SIFT report as a permanent anchor 56
• Keeps up to 5 recent user/AI message pairs (MAX_RECENT_TURNS = 5) for ongoing conversation continuity 57
• Strips out loading indicators and error messages before sending history to the API 58

History is then serialized into provider-specific formats:

• Gemini: Content[] with "model" role mapping 59
• OpenAI-compatible providers: ChatCompletionMessageParam[] with the system message prepended 60

3.4 AI Provider Delivery#

The toolbox supports five AI providers through a unified factory pattern 61:

Gemini uses a direct client and builds a content array with any file attachments as inlineData parts before the text prompt 70.

OpenAI, Anthropic, Mistral, and OpenRouter are handled through the Vercel AI SDK's streamText, which converts conversation history to CoreMessage format and conditionally appends image content for vision-capable models 71.

3.5 Dynamic Source Injection#

When using Gemini with Google Search grounding enabled, the system automatically tracks and corrects citation URLs:

• Source extraction: Grounding chunks are extracted from candidate.groundingMetadata?.groundingChunks and emitted as stream events for the UI to display 72
• Redirect correction: The correctRedirectLinksInMarkdown function replaces Vertex AI Search redirect URLs with direct final URLs. It builds a normalized title-to-URL map from grounding chunks 73, detects vertexaisearch.cloud.google.com redirect patterns 74, and resolves them using exact title matching or fuzzy matching with a 0.7 similarity threshold 75

4. Source Assessment System#

After every Full Check report, the SIFT Toolbox automatically extracts source citations, validates their accessibility, and surfaces reliability ratings in the UI—without any manual action from the user.

4.1 How Sources Are Extracted#

The parseSourceAssessmentsFromMarkdown function parses the generated Markdown report to pull structured source data 76:

1. Locates the section: Uses a regex to find the 🔴 Assessment of Source Reliability section header 77
2. Parses the table: Splits the Markdown table by rows and columns 78
3. Extracts link data: Parses [name](url) Markdown link syntax from the source column 79
4. Returns structured records: Each record contains the source's name, URL, assessment, notes, and rating 80

Each source is represented by a SourceAssessment object 81:

interface SourceAssessment {
  index: number; // Position in the report (starting at 1)
  name: string; // Source title
  url: string; // Web address
  assessment: string; // AI's usefulness evaluation
  notes: string; // AI's analysis notes
  rating: string; // Reliability score (e.g. "4–5")
  linkValidationStatus?: LinkValidationStatus;
}

4.2 URL Validation#

For each extracted source, checkLinkStatus runs a multi-stage accessibility check 82:

The five possible validation statuses are 86:

4.3 Reliability Ratings#

The AI assigns each source a numeric reliability rating on a 1–5 scale, which may be expressed as a range (e.g., "4–5") 81. The RatingDisplay component converts this to a star visualization using the lower bound of the range 87.

The sidebar color-codes each rating for quick scanning 88:

4.4 The Assessment Pipeline#

The full pipeline from report generation to UI display 89:

4.5 Reading Source Assessments in the UI#

Right Sidebar 90

The sidebar lists all extracted sources with:

• Color-coded reliability rating
• Brief assessment snippet
• Link status dot (LinkStatusDot component) 91
• Collapsible/expandable view for each entry

Source Assessment Modal 92

Click any source in the sidebar or any superscript citation index in the report text to open the modal, which shows:

• Full source name and direct URL 93
• Link validation status with plain-English messaging 94
• Star rating visualization 95
• Full usefulness assessment and AI analysis notes 96

Inline citation indices — When the AI cites a source in its report text, the toolbox automatically attaches a superscript index to the Markdown link. Clicking the index opens the Source Assessment Modal for that source .

Link status icons 97:

5. Chat Commands#

Chat commands let you guide the AI's analysis beyond the initial report. Eight built-in commands are available as buttons in the chat interface; you can also create your own custom commands in Settings. All commands are disabled while a response is loading 98.

5.1 Built-in Commands#

Another Round 🔁#

Command string: another round 99

Runs an additional search pass on the same topic to surface new or conflicting information. After results appear, the AI summarizes what new information has emerged and reassesses the overall picture 8.

Tips:

• Run with an affirmative framing ("MMT true"), then again with a contrary framing ("MMT false") to systematically surface opposing viewpoints 100
• Maps directly to the Find step of SIFT

Read the Room 🧐#

Command string: read the room 101

Analyzes the structure of expert and public opinion on the topic. The AI characterizes the information environment using one of five opinion-structure categories 102:

Best for: Understanding controversial topics, assessing whether a claim is mainstream or fringe, and identifying where genuine expert disagreement lies.

Trace Claim 👣#

Command string: trace_claim 103

Uses gathered sources to track a claim or piece of media back to its earliest identifiable origin, revealing how the claim may have been transformed or stripped of context.

Best for: Investigating viral quotes or images, verifying attribution, and implementing the Trace step of SIFT directly.

Web Search 🌐#

Command string: web_search 104

Triggers a live web search to find current information on the topic. Results are integrated into the ongoing analysis.

Note: This command requires a model with web search support. It is automatically disabled for models that lack search tool integration 64. Google Gemini models with the Google Search tool enabled support this command.

Context Report 📊#

Command string: generate_context_report 105

Generates a full Context Report for the artifact or subject currently under discussion, covering its online presentation, audience meaning, actual background, and topical context.

Best for: Creating educational summaries, explaining viral artifacts, and preparing contextual background before drafting a correction.

Community Note 📝#

Command string: generate_community_note 106

Generates a Community Note—a plain-text annotation of 700 characters or fewer with 2–5 supporting links—focused on the context without which the artifact would be seriously misunderstood 30.

Best for: Social media corrections, shareable annotations for viral content, and platform-limited environments.

Discourse Map 🗺️#

Command string: discourse_map 107

Produces an interactive d3.js visualization that maps the claims, evidence, issues, and participants in the discourse around the topic. The core claim sits at the center of the graph, with related nodes clustered around it 108.

Best for: Understanding complex debates with many interrelated claims, visualizing argument structure, and identifying who the key participants are and what positions they hold.

Explain Simply 🧑‍🏫#

Command string: explain_like_im_in_high_school 109

Reformulates the current analysis in plain language accessible to a general audience, stripping technical jargon and simplifying complex concepts.

Best for: Making fact-check findings accessible, educational contexts, and sanity-checking your own understanding by seeing a simpler version.

5.2 Custom Commands#

Users can define custom commands in Settings → Custom Commands 110. Each custom command has the following properties:

Custom commands are saved in the application store and can be added, updated, or deleted at any time 113.

When a custom command is invoked, its prompt instructions are prepended to the user input and its parameter overrides replace the active model settings for that request 54.

When to use custom commands:

• Repetitive analysis patterns specific to your domain or workflow
• Specialized investigation techniques not covered by built-in commands
• Standardized prompt templates for team-wide consistency

5.3 Command Quick Reference#

5.4 Effective Command Sequences#

Comprehensive investigation:

1. Start with the default Full Check for an initial analysis
2. Run Another Round (with an opposing search term) to stress-test the findings
3. Run Read the Room to understand the broader expert consensus
4. Run Trace Claim to verify the origin of any key claims

From analysis to publication:

1. Run Full Check for in-depth research notes
2. Run Context Report to build a structured educational summary
3. Run Community Note to distill findings into a shareable annotation