Ollama Setup for GrepAI

This skill covers installing and configuring Ollama as the local embedding provider for GrepAI. Ollama enables 100% private code search where your code never leaves your machine.

When to Use This Skill

• Setting up GrepAI with local, private embeddings

• Installing Ollama for the first time

• Choosing and downloading embedding models

• Troubleshooting Ollama connection issues

Why Ollama?

Benefit

Description

🔒 Privacy

Code never leaves your machine

💰 Free

No API costs

⚡ Fast

Local processing, no network latency

🔌 Offline

Works without internet

Installation

macOS (Homebrew)

# Install Ollama

brew install ollama

# Start the Ollama service

ollama serve

macOS (Direct Download)

• Download from ollama.com

• Open the .dmg and drag to Applications

• Launch Ollama from Applications

Linux

# One-line installer

curl -fsSL https://ollama.com/install.sh | sh

# Start the service

ollama serve

Windows

• Download installer from ollama.com

• Run the installer

• Ollama starts automatically as a service

Downloading Embedding Models

GrepAI requires an embedding model to convert code into vectors.

Recommended Model: nomic-embed-text

# Download the recommended model (768 dimensions)

ollama pull nomic-embed-text

Specifications:

• Dimensions: 768

• Size: ~274 MB

• Performance: Excellent for code search

• Language: English-optimized

Alternative Models

# Multilingual support (better for non-English code/comments)

ollama pull nomic-embed-text-v2-moe

# Larger, more accurate

ollama pull bge-m3

# Maximum quality

ollama pull mxbai-embed-large

Model

Dimensions

Size

Best For

nomic-embed-text

768

274 MB

General code search

nomic-embed-text-v2-moe

768

500 MB

Multilingual codebases

bge-m3

1024

1.2 GB

Large codebases

mxbai-embed-large

1024

670 MB

Maximum accuracy

Verifying Installation

Check Ollama is Running

# Check if Ollama server is responding

curl http://localhost:11434/api/tags

# Expected output: JSON with available models

List Downloaded Models

ollama list

# Output:

# NAME                     ID           SIZE    MODIFIED

# nomic-embed-text:latest  abc123...    274 MB  2 hours ago

Test Embedding Generation

# Quick test (should return embedding vector)

curl http://localhost:11434/api/embeddings -d '{

  "model": "nomic-embed-text",

  "prompt": "function hello() { return world; }"

}'

Configuring GrepAI for Ollama

After installing Ollama, configure GrepAI to use it:

# .grepai/config.yaml

embedder:

  provider: ollama

  model: nomic-embed-text

  endpoint: http://localhost:11434

This is the default configuration when you run grepai init, so no changes are needed if using nomic-embed-text.

Running Ollama

Foreground (Development)

# Run in current terminal (see logs)

ollama serve

Background (macOS/Linux)

# Using nohup

nohup ollama serve &

# Or as a systemd service (Linux)

sudo systemctl enable ollama

sudo systemctl start ollama

Check Status

# Check if running

pgrep -f ollama

# Or test the API

curl -s http://localhost:11434/api/tags | head -1

Resource Considerations

Memory Usage

Embedding models load into RAM:

• nomic-embed-text: ~500 MB RAM

• bge-m3: ~1.5 GB RAM

• mxbai-embed-large: ~1 GB RAM

CPU vs GPU

Ollama uses CPU by default. For faster embeddings:

• macOS: Uses Metal (Apple Silicon) automatically

• Linux/Windows: Install CUDA for NVIDIA GPU support

Common Issues

❌ Problem: connection refused to localhost:11434

✅ Solution: Start Ollama:

ollama serve

❌ Problem: Model not found

✅ Solution: Pull the model first:

ollama pull nomic-embed-text

❌ Problem: Slow embedding generation

✅ Solution:

• Use a smaller model

• Ensure Ollama is using GPU (check ollama ps)

• Close other memory-intensive applications

❌ Problem: Out of memory

✅ Solution: Use a smaller model or increase system RAM

Best Practices

• Start Ollama before GrepAI: Ensure ollama serve is running

• Use recommended model: nomic-embed-text offers best balance

• Keep Ollama running: Leave it as a background service

• Update periodically: ollama pull nomic-embed-text for updates

Output Format

After successful setup:

✅ Ollama Setup Complete

   Ollama Version: 0.1.x

   Endpoint: http://localhost:11434

   Model: nomic-embed-text (768 dimensions)

   Status: Running

   GrepAI is ready to use with local embeddings.

   Your code will never leave your machine.