open-notebook/docs/3-USER-GUIDE/search.md

Search Effectively - Finding What You Need

Search is your gateway into your research. This guide covers two search modes and when to use each.

---

Quick-Start: Find Something

Simple Search

1. Go to your notebook
2. Type in search box
3. See results (both sources and notes)
4. Click result to view source/note
5. Done!

That works for basic searches.
But you can do much better...

---

Two Search Modes Explained

Open Notebook has two fundamentally different search approaches.

Search Type 1: TEXT SEARCH (Keyword Matching)

How it works:

• You search for words: "transformer"
• System finds chunks containing "transformer"
• Ranked by relevance: frequency, position, context

Speed: Very fast (instant)

When to use:

• You remember exact words or phrases
• You're looking for specific terms
• You want precise keyword matches
• You need exact quotes

Example:

Search: "attention mechanism"
Results:
  1. "The attention mechanism allows..." (perfect match)
  2. "Attention and other mechanisms..." (partial match)
  3. "How mechanisms work in attention..." (includes words separately)

All contain "attention" AND "mechanism"
Ranked by how close together they are

What it finds:

• Exact phrases: "transformer model"
• Individual words: transformer OR model (too broad)
• Names: "Vaswani et al."
• Numbers: "1994", "GPT-4"
• Technical terms: "LSTM", "convolution"

What it doesn't find:

• Similar words: searching "attention" won't find "focus"
• Synonyms: searching "large" won't find "big"
• Concepts: searching "similarity" won't find "likeness"

---

Search Type 2: VECTOR SEARCH (Semantic/Concept Matching)

How it works:

• Your search converted to embedding (vector)
• All chunks converted to embeddings
• System finds most similar embeddings
• Ranked by semantic similarity

Speed: A bit slower (1-2 seconds)

When to use:

• You're exploring a concept
• You don't know exact words
• You want semantically similar content
• You're discovering, not searching

Example:

Search: "What's the mechanism for understanding in models?"
(Notice: No chunk likely says exactly that)

Results:
  1. "Mechanistic interpretability allows understanding..." (semantic match)
  2. "Feature attribution reveals how models work..." (conceptually similar)
  3. "Attention visualization shows model decisions..." (same topic)

None contain your exact words
But all are semantically related

What it finds:

• Similar concepts: "understanding" + "interpretation" + "explainability" (all related)
• Paraphrases: "big" and "large" (same meaning)
• Related ideas: "safety" relates to "alignment" (connected concepts)
• Analogies: content about biological learning when searching "learning"

What it doesn't find:

• Exact keywords: if you search a rare word, vector search might miss it
• Specific numbers: "1994" vs "1993" are semantically different
• Technical jargon: "LSTM" and "RNN" are different even if related

---

Decision: Text Search vs. Vector Search?

Question: "Do I remember the exact words?"

→ YES: Use TEXT SEARCH
   Example: "I remember the paper said 'attention is all you need'"

→ NO: Use VECTOR SEARCH
   Example: "I'm looking for content about how models process information"

→ UNSURE: Try TEXT SEARCH first (faster)
         If no results, try VECTOR SEARCH

Text search: "I know what I'm looking for"
Vector search: "I'm exploring an idea"

---

Step-by-Step: Using Each Search

Text Search

1. Go to search box
2. Type your keywords: "transformer", "attention", "2017"
3. Press Enter
4. Results appear (usually instant)
5. Click result to see context

Results show:
  - Which source contains it
  - How many times it appears
  - Relevance score
  - Preview of surrounding text

Vector Search

1. Go to search box
2. Type your concept: "How do models understand language?"
3. Choose "Vector Search" from dropdown
4. Press Enter
5. Results appear (1-2 seconds)
6. Click result to see context

Results show:
  - Semantically related chunks
  - Similarity score (higher = more related)
  - Preview of surrounding text
  - Different sources mixed together

---

The Ask Feature (Automated Search)

Ask is different from simple search. It automatically searches, synthesizes, and answers.

How Ask Works

Stage 1: QUESTION UNDERSTANDING
  "Compare the approaches in my papers"
  → System: "This asks for comparison"

Stage 2: SEARCH STRATEGY
  → System: "I should search for each approach separately"

Stage 3: PARALLEL SEARCHES
  → Search 1: "Approach in paper A"
  → Search 2: "Approach in paper B"
  (Multiple searches happen at once)

Stage 4: ANALYSIS & SYNTHESIS
  → Per-result analysis: "Based on paper A, the approach is..."
  → Per-result analysis: "Based on paper B, the approach is..."
  → Final synthesis: "Comparing A and B: A differs from B in..."

Result: Comprehensive answer, not just search results

When to Use Ask vs. Simple Search

Task	Use	Why
"Find the quote about X"	TEXT SEARCH	Need exact words
"What does source A say about X?"	TEXT SEARCH	Direct, fast answer
"Find content about X"	VECTOR SEARCH	Semantic discovery
"Compare A and B"	ASK	Comprehensive synthesis
"What's the big picture?"	ASK	Full analysis needed
"How do these sources relate?"	ASK	Cross-source synthesis
"I remember something about X"	TEXT SEARCH	Recall memory
"I'm exploring the topic of X"	VECTOR SEARCH	Discovery mode

---

Advanced Search Strategies

Strategy 1: Simple Search with Follow-Up

1. Text search: "attention mechanism"
   Results: 50 matches

2. Too many. Follow up with vector search:
   "Why is attention useful?" (concept search)
   Results: Most relevant papers/notes

3. Better results with less noise

Strategy 2: Ask for Comprehensive, Then Search for Details

1. Ask: "What are the main approaches to X?"
   Result: Comprehensive answer about A, B, C

2. Use that to identify specific sources

3. Text search in those specific sources:
   "Why did they choose method X?"
   Result: Detailed information

Strategy 3: Vector Search for Discovery, Text for Verification

1. Vector search: "How do transformers generalize?"
   Results: Related conceptual papers

2. Skim to understand landscape

3. Text search in promising sources:
   "generalization", "extrapolation", "transfer"
   Results: Specific passages to read carefully

Strategy 4: Combine Search with Chat

1. Vector search: "What's new in AI 2026?"
   Results: Latest papers

2. Go to Chat
3. Add those papers to context
4. Ask detailed follow-up questions
5. Get deep analysis of results

---

Search Quality Issues & Fixes

Getting No Results

Problem	Cause	Solution
Text search: no results	Word doesn't appear	Try vector search instead
Vector search: no results	Concept not in content	Try broader search term
Both empty	Content not in notebook	Add sources to notebook
	Sources not processed	Wait for processing to complete

Getting Too Many Results

Problem	Cause	Solution
1000+ results	Search too broad	Be more specific
	All sources	Filter by source
	Keyword matches rare words	Use vector search instead

Getting Wrong Results

Problem	Cause	Solution
Results irrelevant	Search term has multiple meanings	Provide more context
	Using text search for concepts	Try vector search
Different meaning	Homonym (word means multiple things)	Add context (e.g., "attention mechanism")

Getting Low Quality Results

Problem	Cause	Solution
Results don't match intent	Vague search term	Be specific ("Who invented X?" vs "X")
	Concept not well-represented	Add more sources on that topic
	Vector embedding not trained on domain	Use text search as fallback

---

Tips for Better Searches

For Text Search

1. Be specific — "attention mechanism" not just "attention"
2. Use exact phrases — Put quotes around: "attention is all you need"
3. Include context — "LSTM vs attention" not just "attention"
4. Use technical terms — These are usually more precise
5. Try synonyms — If first search fails, try related terms

For Vector Search

1. Ask a question — "What's the best way to X?" is better than "best way"
2. Use natural language — Explain what you're looking for
3. Be specific about intent — "Compare X and Y" not "X and Y"
4. Include context — "In machine learning, how..." vs just "how..."
5. Think conceptually — What idea are you exploring?

General Tips

1. Start broad, then narrow — "AI papers" → "transformers" → "attention mechanism"
2. Try both search types — Each finds different things
3. Use Ask for complex questions — Don't just search
4. Save good results as notes — Create knowledge base
5. Filter by source if needed — "Search in Paper A only"

---

Search Examples

Example 1: Finding a Specific Fact

Goal: "Find the date the transformer was introduced"

Step 1: Text search
  "transformer 2017" (or year you remember)

If that works: Done!

If no results: Try
  "attention is all you need" (famous paper title)

Check result for exact date

Example 2: Exploring a Concept

Goal: "Find content about alignment interpretability"

Step 1: Vector search
  "How do we make AI interpretable?"

Results: Papers on interpretability, transparency, alignment

Step 2: Review results
  See which papers are most relevant

Step 3: Deep dive
  Go to Chat, add top 2-3 papers
  Ask detailed questions about alignment

Example 3: Comprehensive Answer

Goal: "How do different approaches to AI safety compare?"

Step 1: Ask
  "Compare the main approaches to AI safety in my sources"

Result: Comprehensive analysis comparing approaches

Step 2: Identify sources
  From answer, see which papers were most relevant

Step 3: Deep dive
  Text search in those papers:
  "limitations", "critiques", "open problems"

Step 4: Save as notes
  Create comparison note from Ask result

Example 4: Finding Pattern

Goal: "Find all papers mentioning transformers"

Step 1: Text search
  "transformer"

Results: All papers mentioning "transformer"

Step 2: Vector search
  "neural network architecture for sequence processing"

Results: Papers that don't say "transformer" but discuss similar concept

Step 3: Combine
  Union of text + vector results shows full landscape

Step 4: Analyze
  Go to Chat with all results
  Ask: "What's common across all these?"

---

Search in the Workflow

How search fits with other features:

SOURCES
  ↓
SEARCH (find what matters)
  ├─ Text search (precise)
  ├─ Vector search (exploration)
  └─ Ask (comprehensive)
  ↓
CHAT (explore with follow-ups)
  ↓
TRANSFORMATIONS (batch extract)
  ↓
NOTES (save insights)

Workflow Example

1. Add 10 papers to notebook

2. Search: "What's the state of the art?"
   (Vector search explores landscape)

3. Ask: "Compare these 3 approaches"
   (Comprehensive synthesis)

4. Chat: Deep questions about winner
   (Follow-up exploration)

5. Save best insights as notes
   (Knowledge capture)

6. Transform remaining papers
   (Batch extraction for later)

7. Create podcast from notes + sources
   (Share findings)

---

Summary: Know Your Search

TEXT SEARCH — "I know what I'm looking for"

• Fast, precise, keyword-based
• Use when you remember exact words/phrases
• Best for: Finding specific facts, quotes, technical terms
• Speed: Instant

VECTOR SEARCH — "I'm exploring an idea"

• Slow-ish, concept-based, semantic
• Use when you're discovering connections
• Best for: Concept exploration, related ideas, synonyms
• Speed: 1-2 seconds

ASK — "I want a comprehensive answer"

• Auto-searches, auto-analyzes, synthesizes
• Use for complex questions needing multiple sources
• Best for: Comparisons, big-picture questions, synthesis
• Speed: 10-30 seconds

Pick the right tool for your search goal, and you'll find what you need faster.