Differential Fuzzer

Always load Debugging skill for reference

The differential fuzzer compares Turso results against SQLite for generated SQL statements to find correctness bugs.

Location

testing/differential-oracle/fuzzer/

Running the Fuzzer

Single Run

# Basic run (100 statements, random seed)

cargo run --bin differential_fuzzer

With specific seed for reproducibility

cargo run --bin differential_fuzzer -- --seed 12345

More statements with verbose output

cargo run --bin differential_fuzzer -- -n 1000 --verbose

Keep database files after run (for debugging)

cargo run --bin differential_fuzzer -- --seed 12345 --keep-files

All options

cargo run --bin differential_fuzzer --

--seed # Deterministic seed

-n # Number of statements (default: 100)

-t # Number of tables (default: 2)

-c # Columns per table (default: 5)

--verbose # Print each SQL statement

--keep-files # Persist .db files to disk

### Continuous Fuzzing (Loop Mode)

Run forever with random seeds

cargo run --bin differential_fuzzer -- loop

Run 50 iterations

cargo run --bin differential_fuzzer -- loop 50

### Docker Runner (CI/Production)

Build and run from repo root

docker build -f testing/differential-oracle/fuzzer/docker-runner/Dockerfile -t fuzzer .

docker run -e GITHUB_TOKEN=xxx -e SLACK_WEBHOOK_URL=xxx fuzzer

Environment variables for docker-runner:

- `TIME_LIMIT_MINUTES` - Total runtime (default: 1440 = 24h)

- `PER_RUN_TIMEOUT_SECONDS` - Per-run timeout (default: 1200 = 20min)

- `NUM_STATEMENTS` - Statements per run (default: 1000)

- `LOG_TO_STDOUT` - Print fuzzer output (default: false)

- `GITHUB_TOKEN` - For auto-filing issues

- `SLACK_WEBHOOK_URL` - For notifications

## Output Files

All output goes to `simulator-output/` directory:

File
Description

`test.sql`
All executed SQL statements. Failed statements prefixed with `-- FAILED:`, errors with `-- ERROR:`

`schema.json`
Database schema at end of run (or at failure)

`test.db`
Turso database file (only with `--keep-files`)

`test-sqlite.db`
SQLite database file (only with `--keep-files`)

## Reproducing Errors

Always follow these steps

-

**Find the seed** in the error output:

INFO: Starting differential_fuzzer with config: SimConfig { seed: 12345, ... }

-

**Re-run with that seed**:

cargo run --bin differential_fuzzer -- --seed 12345 --verbose --keep-files

-

**Check output files**:

- `simulator-output/test.sql` - Find the failing statement (look for `-- FAILED:`)

- `simulator-output/schema.json` - Check table structure at failure time

-

**Create a minimal reproducer**

- Create reproducer in `.sqltest` or in `.rs` always load [Debugging skill for reference](https://github.com/tursodatabase/turso/blob/HEAD/.claude/skills/differential-fuzzer/../debugging/)

-

**Compare behavior manually**:
If needed try to compare the behaviour and produce a report in the end.
Always write to a tmp file first with Edit tool to test the sql and then pass it to the binaries.

Run failing SQL against SQLite

sqlite3 :memory: < simulator-output/test.sql

Run against tursodb CLI

tursodb :memory: < simulator-output/test.sql

## Understanding Failures

### Oracle Failure Types

- **Row set mismatch** - Turso returned different rows than SQLite

- **Turso errored but SQLite succeeded** - Turso rejected valid SQL

- **SQLite errored but Turso succeeded** - Turso accepted invalid SQL

- **Schema mismatch** - Tables/columns differ after DDL

### Warning (non-fatal)

- **Unordered LIMIT mismatch** - LIMIT without ORDER BY may return different valid rows

## Key Source Files

File
Purpose

`main.rs`
CLI parsing, entry point

`runner.rs`
Main simulation loop, executes statements on both DBs

`oracle.rs`
Compares Turso vs SQLite results

`schema.rs`
Introspects schema from both databases

`memory/`
In-memory IO for deterministic simulation

## Tracing

Set `RUST_LOG` for more detailed output:

RUST_LOG=debug cargo run --bin differential_fuzzer -- --seed 12345