GrepAI Chunking Configuration

This skill covers how GrepAI splits code files into chunks for embedding, and how to optimize chunking for your codebase.

When to Use This Skill

• Optimizing search accuracy

• Adjusting for code style (verbose vs. concise)

• Troubleshooting search results

• Understanding how indexing works

What is Chunking?

Chunking is the process of splitting source files into smaller segments for embedding:

┌─────────────────────────────────────┐

│         Large Source File           │

│         (1000+ tokens)              │

└─────────────────────────────────────┘

                  ↓

┌─────────┐ ┌─────────┐ ┌─────────┐

│ Chunk 1 │ │ Chunk 2 │ │ Chunk 3 │

│ ~512    │ │ ~512    │ │ ~512    │

│ tokens  │ │ tokens  │ │ tokens  │

└─────────┘ └─────────┘ └─────────┘

                  ↓

          Each chunk gets

          its own embedding

Why Chunking Matters

Embedding models have optimal input sizes:

• Too large chunks: Less precise search results

• Too small chunks: Lost context, fragmented results

• Just right: Good balance of precision and context

Configuration

Basic Settings

# .grepai/config.yaml

chunking:

  size: 512      # Tokens per chunk

  overlap: 50    # Overlap between chunks

Understanding Parameters

#### Chunk Size

The target number of tokens per chunk.

Size

Effect

256

More precise, less context

512

Balanced (default)

1024

More context, less precise

#### Overlap

Tokens shared between adjacent chunks. Preserves context at boundaries.

Overlap

Effect

0

No overlap, may lose context at boundaries

50

Standard overlap (default)

100

More context, larger index

Visualization

With size=512 and overlap=50:

File: auth.go (1000 tokens)

Chunk 1: tokens 1-512

         ┌────────────────────────────────────┐

         │ func Login(user, pass)...          │

         └────────────────────────────────────┘

                                    ↘

                              50 token overlap

                                    ↙

Chunk 2: tokens 463-974

         ┌────────────────────────────────────┐

         │ ...validate credentials...         │

         └────────────────────────────────────┘

                                    ↘

                              50 token overlap

                                    ↙

Chunk 3: tokens 925-1000

         ┌──────────────┐

         │ ...return    │

         └──────────────┘

Recommended Settings by Language

Verbose Languages (Java, C#)

chunking:

  size: 768    # Larger to capture full methods

  overlap: 75

Concise Languages (Go, Python)

chunking:

  size: 512    # Standard size

  overlap: 50

Very Concise (Rust, Zig)

chunking:

  size: 384    # Smaller for precise results

  overlap: 40

Recommended Settings by Codebase

Small Functions (Microservices)

chunking:

  size: 384    # Capture individual functions

  overlap: 40

Large Classes (Monolith)

chunking:

  size: 768    # Capture more context

  overlap: 100

Mixed Codebase

chunking:

  size: 512    # Balanced default

  overlap: 50

How Tokens are Counted

GrepAI uses approximate token counting:

• ~4 characters = 1 token (for English text)

• Code varies based on identifiers and syntax

Example:

func calculateTotal(items []Item) float64 {

    total := 0.0

    for _, item := range items {

        total += item.Price * float64(item.Quantity)

    }

    return total

}

≈ 45 tokens

Impact on Index Size

Larger overlap = more chunks = larger index:

Size

Overlap

Chunks per 10K tokens

Index Impact

512

0

~20

Smallest

512

50

~22

Standard

512

100

~24

+10%

256

50

~44

+100%

Impact on Search Quality

Too Small Chunks (size: 128)

Query: "authentication middleware"

Result: "...c.AbortWithStatus(401)..."

        (Fragment, missing context)

Just Right (size: 512)

Query: "authentication middleware"

Result: "func AuthMiddleware() gin.HandlerFunc {

            return func(c *gin.Context) {

                token := c.GetHeader("Authorization")

                if token == "" {

                    c.AbortWithStatus(401)

                    return

                }

                // validate token...

            }

        }"

        (Complete function with context)

Too Large Chunks (size: 2048)

Query: "authentication middleware"

Result: "// Multiple unrelated functions...

        func AuthMiddleware()... (your match)

        func LoggingMiddleware()...

        func CORSMiddleware()..."

        (Too much noise)

Experimentation

Testing Different Settings

• Try smaller chunks for more precise results:

chunking:

  size: 384

  overlap: 40

• Re-index:

rm .grepai/index.gob

grepai watch

• Test with searches:

grepai search "your query"

• Adjust and repeat until satisfied.

Comparing Results

Before changing settings, save a search result:

grepai search "authentication" > before.txt

After changing settings and re-indexing:

grepai search "authentication" > after.txt

diff before.txt after.txt

Chunk Boundaries

GrepAI tries to split at logical boundaries:

• Empty lines (function/class boundaries)

• Closing braces

• Statement ends

This means actual chunk sizes may vary slightly from the target.

Best Practices

• Start with defaults: 512/50 works well for most codebases

• Adjust based on code style: Verbose = larger, concise = smaller

• Test with real queries: See what your searches return

• Re-index after changes: Must regenerate embeddings

• Consider overlap: Don't set to 0 unless index size is critical

Common Issues

❌ Problem: Search results are too fragmented

✅ Solution: Increase chunk size:

chunking:

  size: 768

❌ Problem: Search results have too much irrelevant context

✅ Solution: Decrease chunk size:

chunking:

  size: 384

❌ Problem: Results miss related code at function boundaries

✅ Solution: Increase overlap:

chunking:

  overlap: 100

❌ Problem: Index is too large

✅ Solutions:

• Decrease overlap

• Increase chunk size

• Add more ignore patterns

Output Format

Chunking status:

✅ Chunking Configuration

   Size: 512 tokens

   Overlap: 50 tokens

   Index Statistics:

   - Total files: 245

   - Total chunks: 1,234

   - Avg chunks/file: 5.0

   - Avg chunk size: 478 tokens

   Recommendations:

   - Current settings are balanced

   - Consider size: 384 for more precise results

   - Consider size: 768 for more context