GrepAI Embeddings with Ollama

This skill covers using Ollama as the embedding provider for GrepAI, enabling 100% private, local code search.

When to Use This Skill

• Setting up private, local embeddings

• Choosing the right Ollama model

• Optimizing Ollama performance

• Troubleshooting Ollama connection issues

Why Ollama?

Advantage

Description

🔒 Privacy

Code never leaves your machine

💰 Free

No API costs or usage limits

⚡ Speed

No network latency

🔌 Offline

Works without internet

🔧 Control

Choose your model

Prerequisites

• Ollama installed and running

• An embedding model downloaded

# Install Ollama

brew install ollama  # macOS

# or

curl -fsSL https://ollama.com/install.sh | sh  # Linux

# Start Ollama

ollama serve

# Download model

ollama pull nomic-embed-text

Configuration

Basic Configuration

# .grepai/config.yaml

embedder:

  provider: ollama

  model: nomic-embed-text

  endpoint: http://localhost:11434

With Custom Endpoint

embedder:

  provider: ollama

  model: nomic-embed-text

  endpoint: http://192.168.1.100:11434  # Remote Ollama server

With Explicit Dimensions

embedder:

  provider: ollama

  model: nomic-embed-text

  endpoint: http://localhost:11434

  dimensions: 768  # Usually auto-detected

Available Models

Recommended: nomic-embed-text

ollama pull nomic-embed-text

Property

Value

Dimensions

768

Size

~274 MB

Speed

Fast

Quality

Excellent for code

Language

English-optimized

Configuration:

embedder:

  provider: ollama

  model: nomic-embed-text

Multilingual: nomic-embed-text-v2-moe

ollama pull nomic-embed-text-v2-moe

Property

Value

Dimensions

768

Size

~500 MB

Speed

Medium

Quality

Excellent

Language

Multilingual

Best for codebases with non-English comments/documentation.

Configuration:

embedder:

  provider: ollama

  model: nomic-embed-text-v2-moe

High Quality: bge-m3

ollama pull bge-m3

Property

Value

Dimensions

1024

Size

~1.2 GB

Speed

Slower

Quality

Very high

Language

Multilingual

Best for large, complex codebases where accuracy is critical.

Configuration:

embedder:

  provider: ollama

  model: bge-m3

  dimensions: 1024

Maximum Quality: mxbai-embed-large

ollama pull mxbai-embed-large

Property

Value

Dimensions

1024

Size

~670 MB

Speed

Medium

Quality

Highest

Language

English

Configuration:

embedder:

  provider: ollama

  model: mxbai-embed-large

  dimensions: 1024

Model Comparison

Model

Dims

Size

Speed

Quality

Use Case

nomic-embed-text

768

274MB

⚡⚡⚡

⭐⭐⭐

General use

nomic-embed-text-v2-moe

768

500MB

⚡⚡

⭐⭐⭐⭐

Multilingual

bge-m3

1024

1.2GB

⚡

⭐⭐⭐⭐⭐

Large codebases

mxbai-embed-large

1024

670MB

⚡⚡

⭐⭐⭐⭐⭐

Maximum accuracy

Performance Optimization

Memory Management

Models load into RAM. Ensure sufficient memory:

Model

RAM Required

nomic-embed-text

~500 MB

nomic-embed-text-v2-moe

~800 MB

bge-m3

~1.5 GB

mxbai-embed-large

~1 GB

GPU Acceleration

Ollama automatically uses:

• macOS: Metal (Apple Silicon)

• Linux/Windows: CUDA (NVIDIA GPUs)

Check GPU usage:

ollama ps

Keeping Model Loaded

By default, Ollama unloads models after 5 minutes of inactivity. Keep loaded:

# Keep model loaded indefinitely

curl http://localhost:11434/api/generate -d '{

  "model": "nomic-embed-text",

  "keep_alive": -1

}'

Verifying Connection

Check Ollama is Running

curl http://localhost:11434/api/tags

List Available Models

ollama list

Test Embedding

curl http://localhost:11434/api/embeddings -d '{

  "model": "nomic-embed-text",

  "prompt": "function authenticate(user, password)"

}'

Running Ollama as a Service

macOS (launchd)

Ollama app runs automatically on login.

Linux (systemd)

# Enable service

sudo systemctl enable ollama

# Start service

sudo systemctl start ollama

# Check status

sudo systemctl status ollama

Manual Background

nohup ollama serve > /dev/null 2>&1 &

Remote Ollama Server

Run Ollama on a powerful server and connect remotely:

On the Server

# Allow remote connections

OLLAMA_HOST=0.0.0.0 ollama serve

On the Client

# .grepai/config.yaml

embedder:

  provider: ollama

  model: nomic-embed-text

  endpoint: http://server-ip:11434

Common Issues

❌ Problem: Connection refused

✅ Solution:

# Start Ollama

ollama serve

❌ Problem: Model not found

✅ Solution:

# Pull the model

ollama pull nomic-embed-text

❌ Problem: Slow embedding generation

✅ Solutions:

• Use a smaller model (nomic-embed-text)

• Ensure GPU is being used (ollama ps)

• Close memory-intensive applications

• Consider a remote server with better hardware

❌ Problem: Out of memory

✅ Solutions:

• Use a smaller model

• Close other applications

• Upgrade RAM

• Use remote Ollama server

❌ Problem: Embeddings differ after model update

✅ Solution: Re-index after model updates:

rm .grepai/index.gob

grepai watch

Best Practices

• **Start with nomic-embed-text:** Best balance of speed/quality

• Keep Ollama running: Background service recommended

• Match dimensions: Don't mix models with different dimensions

• Re-index on model change: Delete index and re-run watch

• Monitor memory: Embedding models use significant RAM

Output Format

Successful Ollama configuration:

✅ Ollama Embedding Provider Configured

   Provider: Ollama

   Model: nomic-embed-text

   Endpoint: http://localhost:11434

   Dimensions: 768 (auto-detected)

   Status: Connected

   Model Info:

   - Size: 274 MB

   - Loaded: Yes

   - GPU: Apple Metal