Altimate Data Engineering Skills

Skill by ara.so — Data Skills collection.

Altimate Data Engineering Skills is a collection of Claude Code skills that encode the workflows and best practices of experienced analytics engineers. These skills transform Claude from a code generator into a capable data engineering assistant by teaching how to approach tasks, not just what syntax to use.

The project demonstrates 53% accuracy on ADE-bench (43 real-world dbt tasks), 3x improvement on model creation tasks, and 84% pass rate on Snowflake query optimization.

What This Project Does

Data Engineering Skills provides:

• 7 dbt skills: Model creation, debugging, testing, documentation, migration, refactoring, incremental models

• 3 Snowflake skills: Cost analysis, query optimization by ID, query optimization by text

• 1 delegation skill: Hand off complex tasks to altimate-code CLI tool

• Workflow automation: Skills trigger automatically based on user intent

• Best practices: Encoded patterns from experienced analytics engineers

Skills are markdown files with YAML frontmatter that define trigger conditions and step-by-step workflows.

Installation

Installing Skills in Claude Code

# Add the marketplace plugin

/plugin marketplace add AltimateAI/data-engineering-skills

# Install all skills

/plugin install dbt-skills@data-engineering-skills

/plugin install snowflake-skills@data-engineering-skills

/plugin install altimate-code@data-engineering-skills

Installing Kits

Kits bundle skills, MCP servers, and instructions:

# Install altimate-code CLI (required for kits)

npm install -g altimate-code

# Install kit system

altimate-code kit install AltimateAI/data-engineering-skills

# Activate the dbt-snowflake kit

altimate-code kit activate dbt-snowflake

# Check status

altimate-code kit status

Manual Installation

Clone and reference skills directly:

git clone https://github.com/AltimateAI/data-engineering-skills.git

cd data-engineering-skills

Available Skills

dbt Skills

creating-dbt-models: Creates new dbt models following project conventions

• Discovers existing patterns before writing

• Runs dbt build after creation (not just compile)

• Verifies output with dbt show

• Handles staging, intermediate, and mart models

debugging-dbt-errors: Troubleshoots dbt compilation and runtime errors

• Reads full error messages carefully

• Checks upstream dependencies

• Applies fixes and rebuilds

• Stops after 3 failed attempts to reassess

testing-dbt-models: Adds schema tests to models

• Studies existing test patterns in project

• Matches project testing style

• Covers uniqueness, not_null, relationships, accepted_values

documenting-dbt-models: Generates dbt model documentation

• Analyzes model logic and columns

• Creates descriptions for models and fields

• Follows project documentation patterns

migrating-sql-to-dbt: Converts legacy SQL to dbt models

• Parses raw SQL queries

• Creates proper dbt model structure

• Handles CTEs, refs, and sources

refactoring-dbt-models: Safely restructures dbt models

• Tracks all dependencies before changes

• Applies refactoring

• Verifies downstream models still work

developing-incremental-models: Creates incremental dbt models

• Selects appropriate strategy (append, merge, delete+insert)

• Designs proper unique_key

• Handles late-arriving data and edge cases

Snowflake Skills

finding-expensive-queries: Identifies costly Snowflake queries

• Finds queries by cost, time, or data scanned

• Ranks by impact

• Provides query IDs for optimization

optimizing-query-by-id: Optimizes using Snowflake query history ID

• Retrieves query profile from ID

• Applies optimization patterns

• Validates semantic preservation

optimizing-query-text: Optimizes raw SQL query text

• Profiles query execution

• Identifies bottlenecks (scans, joins, aggregations)

• Applies anti-pattern fixes

• Tests performance improvement

Delegation Skill

altimate-code: Delegates complex data tasks to altimate-code CLI

• Verifies altimate-code installation

• Invokes altimate-code run --yolo non-interactively

• Reads output file and summarizes results

• Requires: npm install -g altimate-code (Node 20+)

Skill Structure

Skills are markdown files with YAML frontmatter:

---

name: creating-dbt-models

description: |

  Guide for creating dbt models. ALWAYS use this skill when:

  (1) Creating ANY new model (staging, intermediate, mart)

  (2) Task mentions "create", "build", "add" with model/table

  (3) Modifying model logic or columns

---

Followed by workflow instructions:

# dbt Model Development

**Read before you write. Build after you write. Verify your output.**

## Critical Rules

1. ALWAYS run `dbt build` after creating models - compile is NOT enough

2. ALWAYS verify output after build using `dbt show`

3. If build fails 3+ times, stop and reassess your approach

## Workflow

1. Discover Conventions

   - Check existing models in same layer (staging/intermediate/mart)

   - Note naming patterns, CTE style, column ordering

2. Write Model

   - Follow discovered patterns

   - Use proper refs and sources

3. Build and Verify

   ```bash

   dbt build --select model_name

   dbt show --select model_name --limit 10

## Usage Examples

### Creating a dbt Model

User request:

"Create a staging model for raw customers data"

Skill triggers: creating-dbt-models

AI workflow:

1. Checks models/staging/ for naming patterns

2. Creates models/staging/stg_customers.sql

3. Runs: dbt build --select stg_customers

4. Verifies: dbt show --select stg_customers

Example model created:

-- models/staging/stg_customers.sql

with source as (

select * from {{ source('jaffle_shop', 'customers') }}

),

renamed as (

select

id as customer_id,

first_name,

last_name,

email,

created_at

from source

)

select * from renamed

### Debugging dbt Errors

User request:

"Fix this error: Compilation Error in model customers (models/customers.sql)"

Skill triggers: debugging-dbt-errors

AI workflow:

1. Reads full error message

2. Checks upstream model dependencies

3. Identifies issue (e.g., missing ref)

4. Applies fix

5. Rebuilds: dbt build --select customers

### Optimizing Snowflake Query

User request:

"This query is taking 5 minutes, can you optimize it?"

Skill triggers: optimizing-query-text

AI workflow:

1. Profiles query (EXPLAIN, execution stats)

2. Identifies bottlenecks (table scans, inefficient joins)

3. Applies optimizations (clustering, filters, join order)

4. Tests and measures improvement

Example optimization:

-- Before (slow)

SELECT

o.order_id,

c.customer_name,

SUM(oi.amount) as total

FROM orders o

JOIN customers c ON o.customer_id = c.customer_id

JOIN order_items oi ON o.order_id = oi.order_id

WHERE o.order_date >= '2024-01-01'

GROUP BY 1, 2;

-- After (optimized)

SELECT

o.order_id,

c.customer_name,

SUM(oi.amount) as total

FROM orders o

INNER JOIN order_items oi ON o.order_id = oi.order_id

LEFT JOIN customers c ON o.customer_id = c.customer_id

WHERE o.order_date >= '2024-01-01'

AND o.order_date < '2024-12-31' -- Added upper bound for partition pruning

GROUP BY 1, 2;

-- Assumes orders table is clustered by order_date

### Using Kits

Activate dbt-snowflake kit for a project

cd /path/to/dbt-project

altimate-code kit activate dbt-snowflake

This configures:

- All dbt and Snowflake skills

- MCP server for dbt (live project access)

- Project-specific instructions

Deactivate when done

altimate-code kit deactivate

Kit configuration (`.altimate/kits/active.yaml`):

kit: dbt-snowflake

version: 1.0.0

skills:

- creating-dbt-models

- debugging-dbt-errors

- testing-dbt-models

- documenting-dbt-models

- migrating-sql-to-dbt

- refactoring-dbt-models

- developing-incremental-models

- finding-expensive-queries

- optimizing-query-by-id

- optimizing-query-text

mcp_servers:

- dbt

instructions: |

You are working on a dbt + Snowflake project.

Always check project conventions before creating models.

Run dbt build (not compile) after creating models.

## Configuration

### Skill Configuration

Skills are auto-triggered based on user requests. No explicit configuration needed.

### MCP Integration

Skills work best with Altimate MCP server for live project access:

// claude_desktop_config.json or similar

{

"mcpServers": {

"altimate-dbt": {

"command": "npx",

"args": ["-y", "@altimate/mcp-server-dbt"],

"env": {

"DBT_PROJECT_DIR": "/path/to/dbt/project",

"DBT_PROFILES_DIR": "/path/to/.dbt"

}

}

}

}

MCP tools available to skills:

- `dbt_project_info`: Project structure, model list, sources

- `dbt_model_details`: Column types, dependencies, compiled SQL

- `dbt_compile`: Compile models without CLI

- `snowflake_query_history`: Recent query executions and stats

- `snowflake_table_stats`: Row counts, clustering info

## Common Patterns

### Pattern 1: Multi-Step Model Creation

User: "Create a mart model for monthly revenue by customer"

Skills auto-chains:

1. creating-dbt-models: Creates initial model

2. testing-dbt-models: Adds schema tests

3. documenting-dbt-models: Adds documentation

### Pattern 2: Debug-Fix-Verify Loop

User: "My model won't compile"

Skill: debugging-dbt-errors

Workflow:

1. Read error

2. Check dependencies

3. Apply fix

4. Rebuild

5. If still fails, repeat up to 3x

6. If 3x fails, stop and reassess approach

### Pattern 3: Safe Refactoring

User: "Break this large model into smaller ones"

Skill: refactoring-dbt-models

Workflow:

1. Map all downstream dependencies

2. Create new intermediate models

3. Update refs in downstream models

4. Run dbt build on entire DAG

5. Verify no breakage

### Pattern 4: Incremental Model Development

-- Pattern: merge strategy with unique_key

{{ config(

materialized='incremental',

unique_key='event_id',

merge_update_columns=['status', 'updated_at']

) }}

select

event_id,

user_id,

event_type,

status,

created_at,

updated_at

from {{ source('events', 'raw_events') }}

{% if is_incremental() %}

where updated_at > (select max(updated_at) from {{ this }})

{% endif %}

### Pattern 5: Query Optimization Workflow

Skill: optimizing-query-text

Steps encoded:

1. Profile query

2. Identify bottleneck type:

- Full table scan → Add filters/clustering

- Inefficient join → Reorder, change type

- Large aggregation → Pre-aggregate or partition

3. Apply pattern-based fix

4. Validate semantics unchanged

5. Measure improvement

## Troubleshooting

### Skills Not Triggering

**Problem**: Skills don't activate for your request

**Solution**:

- Be explicit: "Create a dbt model" not "make a model"

- Check skill installation: `/plugin list`

- Review trigger phrases in skill YAML

- Use exact trigger phrases from skill description

### dbt Build Failures

**Problem**: Models fail to build after creation

**Solution**:

- Skill will auto-retry up to 3 times

- After 3 failures, skill stops to reassess

- Check `dbt debug` for connection issues

- Verify refs and sources exist

- Check for SQL syntax errors

### MCP Server Not Connected

**Problem**: Skills can't access live project data

**Solution**:

Check MCP server config

cat claude_desktop_config.json | grep altimate

Verify dbt project path

export DBT_PROJECT_DIR=/correct/path

export DBT_PROFILES_DIR=/correct/.dbt/path

Restart Claude Code

### altimate-code Delegation Fails

**Problem**: "altimate-code not found" error

**Solution**:

Install altimate-code CLI

npm install -g altimate-code

Verify installation

altimate-code --version

Check Node version (requires 20+)

node --version

### Query Optimization No Improvement

**Problem**: Optimized query performs the same

**Solution**:

- Skill checks for anti-patterns but can't fix all issues

- May need manual clustering/indexing setup

- Check if bottleneck is data volume (consider aggregation)

- Review Snowflake warehouse size

- Use `finding-expensive-queries` to compare before/after

### Kit Activation Issues

**Problem**: Kit won't activate

**Solution**:

Check kit is installed

altimate-code kit list

Reinstall if needed

altimate-code kit install AltimateAI/data-engineering-skills

Check for conflicting active kits

altimate-code kit status

Deactivate others first

altimate-code kit deactivate