llm-checker/USAGE_GUIDE.md

📚 LLM Checker - Complete Usage Guide

This guide provides detailed examples and use cases for the LLM Checker intelligent model selector.

---

🎯 Common Use Cases

1. First-Time User - "What model should I use?"

Scenario: You're new to LLMs and want to know what works best on your system.

# Get intelligent recommendation for your hardware
llm-checker ai-check

# See what models are available locally
llm-checker list-models

# Get broader system analysis
llm-checker check

Expected Output:

🧠 INTELLIGENT MODEL SELECTION 
│ 🏆 Selected Model: llama2:7b
│ 📊 Confidence: 100%
│ 🔢 Intelligence Score: 99/100
│ 💡 AI Analysis: Excellent fit for your high hardware configuration...

What to do next: Run ollama pull llama2:7b to install the recommended model.

---

2. Developer - "I need a coding assistant"

Scenario: You want to find the best model for programming tasks.

# Compare coding-specific models
llm-checker ai-check --models codellama:7b deepseek-coder:6.7b starcoder:7b

# Browse all coding models
llm-checker list-models --category coding

# Quick coding session
llm-checker ai-run --prompt "Help me write a Python function to parse JSON"

Why it works: The intelligent selector considers:

• Model specialization in coding tasks
• Context length for large code files
• Performance on programming benchmarks

---

3. Researcher - "I need reasoning capabilities"

Scenario: You're doing research and need models good at complex reasoning.

# Compare reasoning models
llm-checker ai-check --models llama2:13b qwen2:7b mistral:7b

# Get category-specific recommendations
llm-checker recommend --category reasoning

# Check what large models you can run
llm-checker ai-check --models llama2:70b claude:20b gpt4:175b

Intelligence in action: Prioritizes models with:

• Higher parameter counts for complex reasoning
• Strong performance on reasoning benchmarks
• Sufficient context length for multi-step problems

---

4. Hardware Upgrader - "What can I run with better hardware?"

Scenario: You're considering a hardware upgrade and want to see potential.

# Check current capabilities
llm-checker check --detailed

# See what models require more resources
llm-checker list-models --full | grep "16.*GB\|32.*GB"

# Compare small vs large models
llm-checker ai-check --models phi3:mini llama2:7b llama2:13b llama2:70b

Insight: The hardware tier analysis shows exactly what upgrade would unlock new model tiers.

---

5. Performance Optimizer - "What's the fastest model for my needs?"

Scenario: You need the best performance/speed ratio.

# Find efficient models
llm-checker list-models --popular --limit 20

# Compare performance profiles
llm-checker ai-check --models phi3:mini mistral:7b gemma:7b

# Test multiple sizes of same family
llm-checker ai-check --models llama2:7b llama2:13b

Intelligence factors: Considers inference speed, memory efficiency, and quality balance.

---

🛠️ Advanced Usage Patterns

Command Chaining for Workflows

# Complete discovery workflow
llm-checker check && \
llm-checker list-models --category coding && \
llm-checker ai-check --models codellama:7b deepseek-coder:6.7b

# Automated model testing
for model in llama2:7b mistral:7b phi3:mini; do
  echo "Testing $model:"
  llm-checker ai-check --models $model
done

JSON Output for Integration

# Export model database
llm-checker list-models --json > my-models.json

# Integrate with scripts
MODEL=$(llm-checker ai-check --models llama2:7b mistral:7b --json | jq -r '.bestModel')
echo "Best model: $MODEL"

Custom Model Comparison

# Compare your installed models
MODELS=$(ollama list | tail -n +2 | awk '{print $1}' | tr '\n' ' ')
llm-checker ai-check --models $MODELS

---

🎨 Understanding the Output

Intelligence Score Breakdown

Score Range	Meaning	Action
90-100	Perfect match	Use immediately
70-89	Great fit	Recommended choice
50-69	Good option	Consider if you like the model
30-49	Suboptimal	Better alternatives exist
0-29	Poor fit	Likely won't work well

Confidence Levels

Confidence	Interpretation
95-100%	Algorithm is certain about hardware compatibility
80-94%	High confidence, good match expected
60-79%	Moderate confidence, should work well
40-59%	Lower confidence, may have limitations
<40%	Uncertain, proceed with caution

Hardware Tier Impact

Your Hardware Tier: HIGH
├── Can run: 7B-13B models optimally
├── Possible: 30B models with limitations  
├── Recommended: llama2:7b, mistral:7b, codellama:7b
└── Avoid: 70B+ models (insufficient memory)

---

🚨 Troubleshooting

"No models found"

# Check if Ollama is running
ollama list

# Install some models first
ollama pull llama2:7b
ollama pull phi3:mini

# Then rerun
llm-checker ai-check

"Low confidence scores"

This usually means:

• Limited hardware - Consider smaller models
• Unusual configuration - Algorithm less certain
• Memory constraints - Most models too large

Solution: Try with smaller models:

llm-checker ai-check --models phi3:mini gemma:2b qwen2:1.5b

"Model doesn't perform as expected"

The algorithm optimizes for compatibility, not task-specific performance. For specialized needs:

# Check task-specific models
llm-checker list-models --category coding  # For programming
llm-checker list-models --category creative  # For writing

---

🎓 Tips & Best Practices

1. Start Small, Scale Up

# Begin with efficient models
llm-checker ai-check --models phi3:mini gemma:2b

# Then try larger ones
llm-checker ai-check --models llama2:7b mistral:7b

2. Use Specific Comparisons

# Instead of checking everything
llm-checker list-models --limit 50

# Compare specific candidates  
llm-checker ai-check --models model1 model2 model3

3. Monitor Resource Usage

# Check system impact
llm-checker check --detailed

# Monitor during model execution
htop  # or Activity Monitor on macOS

4. Keep Database Updated

# Regular updates for new models
llm-checker update-db

# Force refresh if needed
llm-checker update-db --force

5. Leverage Categories

llm-checker list-models --category coding    # Programming
llm-checker list-models --category reasoning # Research  
llm-checker list-models --category creative  # Writing

---

🔬 Advanced Configuration

For Power Users

The intelligent selector can be customized by editing src/ai/intelligent-selector.js:

// Adjust scoring weights
this.performanceWeights = {
    memory_efficiency: 0.40,    // Increase memory priority
    performance_match: 0.30,    // Boost performance importance
    task_optimization: 0.15,    // Reduce task specificity  
    popularity_quality: 0.10,   // Less weight on popularity
    resource_efficiency: 0.05   // Keep efficiency low
};

Custom Model Profiles

Add your own models to the database:

'custom-model:7b': {
    name: 'Custom Model 7B',
    size_gb: 4.2,
    parameters: 7,
    memory_requirement: 8,
    cpu_cores_min: 4,
    cpu_intensive: 0.8,
    specialization: ['custom', 'specialized'],
    quality_score: 9.0,
    popularity_score: 5.0,
    context_length: 8192,
    quantization: 'Q4_0',
    inference_speed: 'medium'
}

---

📊 Performance Benchmarking

Optional ML Training Pipeline

For users who want to contribute benchmarking data:

# Collect performance data
npm run benchmark

# Process and label data  
cd ml-model && python python/dataset_aggregator.py

# Train TabTransformer model
npm run train-ai

# Use trained model for predictions
llm-checker ai-check --status

This creates an ONNX model that learns from real performance data across different hardware configurations.

---

💡 Pro Tip: Use llm-checker ai-check --help and llm-checker ai-run --help to see all available options for any command!