AgentDB Learning Plugins

What This Skill Does

Provides access to 9 reinforcement learning algorithms via AgentDB's plugin system. Create, train, and deploy learning plugins for autonomous agents that improve through experience. Includes offline RL (Decision Transformer), value-based learning (Q-Learning), policy gradients (Actor-Critic), and advanced techniques.

Performance: Train models 10-100x faster with WASM-accelerated neural inference.

Prerequisites

• Node.js 18+

• AgentDB v1.0.7+ (via agentic-flow)

• Basic understanding of reinforcement learning (recommended)

Quick Start with CLI

Create Learning Plugin

# Interactive wizard

npx agentdb@latest create-plugin

# Use specific template

npx agentdb@latest create-plugin -t decision-transformer -n my-agent

# Preview without creating

npx agentdb@latest create-plugin -t q-learning --dry-run

# Custom output directory

npx agentdb@latest create-plugin -t actor-critic -o .$plugins

List Available Templates

# Show all plugin templates

npx agentdb@latest list-templates

# Available templates:

# - decision-transformer (sequence modeling RL - recommended)

# - q-learning (value-based learning)

# - sarsa (on-policy TD learning)

# - actor-critic (policy gradient with baseline)

# - curiosity-driven (exploration-based)

Manage Plugins

# List installed plugins

npx agentdb@latest list-plugins

# Get plugin information

npx agentdb@latest plugin-info my-agent

# Shows: algorithm, configuration, training status

Quick Start with API

import { createAgentDBAdapter } from 'agentic-flow$reasoningbank';

// Initialize with learning enabled

const adapter = await createAgentDBAdapter({

  dbPath: '.agentdb$learning.db',

  enableLearning: true,       // Enable learning plugins

  enableReasoning: true,

  cacheSize: 1000,

});

// Store training experience

await adapter.insertPattern({

  id: '',

  type: 'experience',

  domain: 'game-playing',

  pattern_data: JSON.stringify({

    embedding: await computeEmbedding('state-action-reward'),

    pattern: {

      state: [0.1, 0.2, 0.3],

      action: 2,

      reward: 1.0,

      next_state: [0.15, 0.25, 0.35],

      done: false

    }

  }),

  confidence: 0.9,

  usage_count: 1,

  success_count: 1,

  created_at: Date.now(),

  last_used: Date.now(),

});

// Train learning model

const metrics = await adapter.train({

  epochs: 50,

  batchSize: 32,

});

console.log('Training Loss:', metrics.loss);

console.log('Duration:', metrics.duration, 'ms');

Available Learning Algorithms (9 Total)

1. Decision Transformer (Recommended)

Type: Offline Reinforcement Learning

Best For: Learning from logged experiences, imitation learning

Strengths: No online interaction needed, stable training

npx agentdb@latest create-plugin -t decision-transformer -n dt-agent

Use Cases:

• Learn from historical data

• Imitation learning from expert demonstrations

• Safe learning without environment interaction

• Sequence modeling tasks

Configuration:

{

  "algorithm": "decision-transformer",

  "model_size": "base",

  "context_length": 20,

  "embed_dim": 128,

  "n_heads": 8,

  "n_layers": 6

}

2. Q-Learning

Type: Value-Based RL (Off-Policy)

Best For: Discrete action spaces, sample efficiency

Strengths: Proven, simple, works well for small$medium problems

npx agentdb@latest create-plugin -t q-learning -n q-agent

Use Cases:

• Grid worlds, board games

• Navigation tasks

• Resource allocation

• Discrete decision-making

Configuration:

{

  "algorithm": "q-learning",

  "learning_rate": 0.001,

  "gamma": 0.99,

  "epsilon": 0.1,

  "epsilon_decay": 0.995

}

3. SARSA

Type: Value-Based RL (On-Policy)

Best For: Safe exploration, risk-sensitive tasks

Strengths: More conservative than Q-Learning, better for safety

npx agentdb@latest create-plugin -t sarsa -n sarsa-agent

Use Cases:

• Safety-critical applications

• Risk-sensitive decision-making

• Online learning with exploration

Configuration:

{

  "algorithm": "sarsa",

  "learning_rate": 0.001,

  "gamma": 0.99,

  "epsilon": 0.1

}

4. Actor-Critic

Type: Policy Gradient with Value Baseline

Best For: Continuous actions, variance reduction

Strengths: Stable, works for continuous$discrete actions

npx agentdb@latest create-plugin -t actor-critic -n ac-agent

Use Cases:

• Continuous control (robotics, simulations)

• Complex action spaces

• Multi-agent coordination

Configuration:

{

  "algorithm": "actor-critic",

  "actor_lr": 0.001,

  "critic_lr": 0.002,

  "gamma": 0.99,

  "entropy_coef": 0.01

}

5. Active Learning

Type: Query-Based Learning

Best For: Label-efficient learning, human-in-the-loop

Strengths: Minimizes labeling cost, focuses on uncertain samples

Use Cases:

• Human feedback incorporation

• Label-efficient training

• Uncertainty sampling

• Annotation cost reduction

6. Adversarial Training

Type: Robustness Enhancement

Best For: Safety, robustness to perturbations

Strengths: Improves model robustness, adversarial defense

Use Cases:

• Security applications

• Robust decision-making

• Adversarial defense

• Safety testing

7. Curriculum Learning

Type: Progressive Difficulty Training

Best For: Complex tasks, faster convergence

Strengths: Stable learning, faster convergence on hard tasks

Use Cases:

• Complex multi-stage tasks

• Hard exploration problems

• Skill composition

• Transfer learning

8. Federated Learning

Type: Distributed Learning

Best For: Privacy, distributed data

Strengths: Privacy-preserving, scalable

Use Cases:

• Multi-agent systems

• Privacy-sensitive data

• Distributed training

• Collaborative learning

9. Multi-Task Learning

Type: Transfer Learning

Best For: Related tasks, knowledge sharing

Strengths: Faster learning on new tasks, better generalization

Use Cases:

• Task families

• Transfer learning

• Domain adaptation

• Meta-learning

Training Workflow

1. Collect Experiences

// Store experiences during agent execution

for (let i = 0; i < numEpisodes; i++) {

  const episode = runEpisode();

  for (const step of episode.steps) {

    await adapter.insertPattern({

      id: '',

      type: 'experience',

      domain: 'task-domain',

      pattern_data: JSON.stringify({

        embedding: await computeEmbedding(JSON.stringify(step)),

        pattern: {

          state: step.state,

          action: step.action,

          reward: step.reward,

          next_state: step.next_state,

          done: step.done

        }

      }),

      confidence: step.reward > 0 ? 0.9 : 0.5,

      usage_count: 1,

      success_count: step.reward > 0 ? 1 : 0,

      created_at: Date.now(),

      last_used: Date.now(),

    });

  }

}

2. Train Model

// Train on collected experiences

const trainingMetrics = await adapter.train({

  epochs: 100,

  batchSize: 64,

  learningRate: 0.001,

  validationSplit: 0.2,

});

console.log('Training Metrics:', trainingMetrics);

// {

//   loss: 0.023,

//   valLoss: 0.028,

//   duration: 1523,

//   epochs: 100

// }

3. Evaluate Performance

// Retrieve similar successful experiences

const testQuery = await computeEmbedding(JSON.stringify(testState));

const result = await adapter.retrieveWithReasoning(testQuery, {

  domain: 'task-domain',

  k: 10,

  synthesizeContext: true,

});

// Evaluate action quality

const suggestedAction = result.memories[0].pattern.action;

const confidence = result.memories[0].similarity;

console.log('Suggested Action:', suggestedAction);

console.log('Confidence:', confidence);

Advanced Training Techniques

Experience Replay

// Store experiences in buffer

const replayBuffer = [];

// Sample random batch for training

const batch = sampleRandomBatch(replayBuffer, batchSize: 32);

// Train on batch

await adapter.train({

  data: batch,

  epochs: 1,

  batchSize: 32,

});

Prioritized Experience Replay

// Store experiences with priority (TD error)

await adapter.insertPattern({

  // ... standard fields

  confidence: tdError,  // Use TD error as confidence$priority

  // ...

});

// Retrieve high-priority experiences

const highPriority = await adapter.retrieveWithReasoning(queryEmbedding, {

  domain: 'task-domain',

  k: 32,

  minConfidence: 0.7,  // Only high TD-error experiences

});

Multi-Agent Training

// Collect experiences from multiple agents

for (const agent of agents) {

  const experience = await agent.step();

  await adapter.insertPattern({

    // ... store experience with agent ID

    domain: `multi-agent/${agent.id}`,

  });

}

// Train shared model

await adapter.train({

  epochs: 50,

  batchSize: 64,

});

Performance Optimization

Batch Training

// Collect batch of experiences

const experiences = collectBatch(size: 1000);

// Batch insert (500x faster)

for (const exp of experiences) {

  await adapter.insertPattern({ /* ... */ });

}

// Train on batch

await adapter.train({

  epochs: 10,

  batchSize: 128,  // Larger batch for efficiency

});

Incremental Learning

// Train incrementally as new data arrives

setInterval(async () => {

  const newExperiences = getNewExperiences();

  if (newExperiences.length > 100) {

    await adapter.train({

      epochs: 5,

      batchSize: 32,

    });

  }

}, 60000);  // Every minute

Integration with Reasoning Agents

Combine learning with reasoning for better performance:

// Train learning model

await adapter.train({ epochs: 50, batchSize: 32 });

// Use reasoning agents for inference

const result = await adapter.retrieveWithReasoning(queryEmbedding, {

  domain: 'decision-making',

  k: 10,

  useMMR: true,              // Diverse experiences

  synthesizeContext: true,    // Rich context

  optimizeMemory: true,       // Consolidate patterns

});

// Make decision based on learned experiences + reasoning

const decision = result.context.suggestedAction;

const confidence = result.memories[0].similarity;

CLI Operations

# Create plugin

npx agentdb@latest create-plugin -t decision-transformer -n my-plugin

# List plugins

npx agentdb@latest list-plugins

# Get plugin info

npx agentdb@latest plugin-info my-plugin

# List templates

npx agentdb@latest list-templates

Troubleshooting

Issue: Training not converging

// Reduce learning rate

await adapter.train({

  epochs: 100,

  batchSize: 32,

  learningRate: 0.0001,  // Lower learning rate

});

Issue: Overfitting

// Use validation split

await adapter.train({

  epochs: 50,

  batchSize: 64,

  validationSplit: 0.2,  // 20% validation

});

// Enable memory optimization

await adapter.retrieveWithReasoning(queryEmbedding, {

  optimizeMemory: true,  // Consolidate, reduce overfitting

});

Issue: Slow training

# Enable quantization for faster inference

# Use binary quantization (32x faster)

Learn More

• Algorithm Papers: See docs$algorithms/ for detailed papers

• GitHub: https:/$github.com$ruvnet$agentic-flow$tree$main$packages$agentdb

• MCP Integration: npx agentdb@latest mcp

• Website: https:/$agentdb.ruv.io

Category: Machine Learning / Reinforcement Learning

Difficulty: Intermediate to Advanced

Estimated Time: 30-60 minutes