-
Notifications
You must be signed in to change notification settings - Fork 2k
Description
🚀 EPIC: Integrate Agentic-Flow Multi-Provider Agent Execution Engine
📋 Epic Overview
Epic Goal: Integrate agentic-flow's multi-provider agent execution engine into claude-flow, enabling 99% cost savings, 352x faster local edits, privacy-first local inference, and access to 66+ specialized agents.
Strategic Value:
- Cost Optimization: Choose from 4+ providers (Anthropic, OpenRouter, ONNX, Gemini) per task
- Speed Enhancement: Agent Booster provides 352x faster local WASM-based code editing
- Privacy Mode: ONNX local inference for sensitive code (100% offline)
- Agent Library: 66+ pre-built specialized agents vs current ~20
- Developer Choice: Smart defaults with granular control
Current State: Claude-flow provides excellent coordination (swarms, topology, memory, hooks) but is locked to Anthropic provider only.
Desired State: Claude-flow becomes the unified platform: coordination layer + flexible multi-provider execution layer.
🎯 Success Metrics
| Metric | Current | Target | Impact |
|---|---|---|---|
| Cost per task | $0.15 (Anthropic only) | $0.001-0.15 | 99% potential savings |
| Provider options | 1 | 4+ | 4x flexibility |
| Local edit speed | 2.5s (API call) | 0.007s (WASM) | 352x faster |
| Available agents | ~20 | 66+ | 3.3x more |
| Privacy options | Cloud only | Local + Cloud | Offline mode enabled |
| Backwards compatibility | N/A | 100% | Zero breaking changes |
🏗️ Architecture Overview
┌──────────────────────────────────────────────────┐
│ Claude Code (Task tool) │
│ Primary invocation interface │
└──────────────────────────────────────────────────┘
↓
┌──────────────────────────────────────────────────┐
│ Claude-Flow Package │
│ │
│ ╔════════════════════════════════════════════╗ │
│ ║ Coordination Layer (Existing) ║ │
│ ║ • Swarm topology (mesh, hierarchical) ║ │
│ ║ • Task orchestration across agents ║ │
│ ║ • Cross-agent memory management ║ │
│ ║ • Hook system for automation ║ │
│ ║ • Neural pattern training ║ │
│ ╚════════════════════════════════════════════╝ │
│ ↓ │
│ ╔════════════════════════════════════════════╗ │
│ ║ Execution Layer (NEW: Agentic-Flow) ║ │
│ ║ • 66+ specialized agent definitions ║ │
│ ║ • Multi-provider engine: ║ │
│ ║ - Anthropic (quality) ║ │
│ ║ - OpenRouter (cost) ║ │
│ ║ - ONNX (privacy/free) ║ │
│ ║ - Gemini (free tier) ║ │
│ ║ • Agent Booster (352x faster WASM) ║ │
│ ║ • Model optimization engine ║ │
│ ╚════════════════════════════════════════════╝ │
└──────────────────────────────────────────────────┘
Key Principle: Claude-flow coordinates strategy, agentic-flow executes with choice.
📦 Integration Strategy
Recommended Approach: Dependency Integration
Add agentic-flow as npm dependency:
{
"name": "claude-flow",
"version": "2.6.0",
"dependencies": {
"agentic-flow": "^1.0.0"
}
}Benefits:
- ✅ Clean separation of concerns (coordination vs execution)
- ✅ Leverage existing battle-tested agentic-flow code
- ✅ Easy updates when agentic-flow improves
- ✅ Smaller maintenance burden
- ✅ Community can contribute to both packages independently
Alternatives Considered:
- ❌ Fork & Merge: Heavy refactoring, harder to maintain
- ❌ Separate + Bridge: More complex for users, fragmented experience
🗂️ File Structure Changes
claude-flow/
├── src/
│ ├── coordination/ # EXISTING: Swarm logic
│ │ ├── swarm-manager.js
│ │ ├── topology.js
│ │ └── task-orchestrator.js
│ │
│ ├── execution/ # NEW: Agentic-flow integration
│ │ ├── agent-executor.js # Wrapper around agentic-flow
│ │ ├── provider-manager.js # Multi-provider orchestration
│ │ ├── model-optimizer.js # Auto-select best model
│ │ └── booster-adapter.js # WASM booster integration
│ │
│ ├── cli/
│ │ ├── sparc.js # EXISTING
│ │ ├── hooks.js # EXISTING
│ │ ├── agent.js # NEW: Agent execution commands
│ │ ├── booster.js # NEW: Fast WASM edit commands
│ │ └── config.js # NEW: Provider configuration
│ │
│ ├── mcp/
│ │ ├── coordination-tools.js # EXISTING
│ │ └── execution-tools.js # NEW: Agent execution MCP tools
│ │
│ └── hooks/ # EXISTING (enhanced with execution events)
│
├── agents/ # NEW: Custom agent storage
│ └── custom/ # User-defined agents
│
└── package.json # UPDATED: Add agentic-flow dependency
🔧 CLI Enhancement Plan
New Commands
# Agent Execution
npx claude-flow agent run <agent> "<task>" [options]
npx claude-flow agent list [--source all|package|local]
npx claude-flow agent info <agent-name>
npx claude-flow agent create <name> [--category custom]
npx claude-flow agent optimize "<task>" [--priority quality|cost|speed|privacy]
# Ultra-Fast Booster (WASM local edits)
npx claude-flow booster edit <file> [--instructions "..."]
npx claude-flow booster batch <files...>
npx claude-flow booster parse <markdown-file>
# Provider Management
npx claude-flow config set-provider <provider> [--model <model>]
npx claude-flow config list-providers
npx claude-flow config test-provider <provider>
# Enhanced SPARC with Provider Control
npx claude-flow sparc tdd "<task>" \
--spec-provider anthropic \
--code-provider openrouter \
--test-provider onnx
# Auto-Optimization
npx claude-flow sparc tdd "<task>" --optimize-for cost|speed|qualityBackwards Compatibility
# Existing commands continue to work (default to Anthropic)
npx claude-flow sparc tdd "feature" # ✅ Still works
npx claude-flow hooks pre-task # ✅ Still works
npx claude-flow mcp start # ✅ Still works
# New capabilities are opt-in enhancements
npx claude-flow sparc tdd "feature" --provider openrouter # NEW⚙️ Configuration Integration
Extended .claude/settings.json
{
"claude-flow": {
"coordination": {
"topology": "mesh",
"maxAgents": 8,
"strategy": "balanced"
},
"execution": {
"defaultProvider": "anthropic",
"providers": {
"anthropic": {
"model": "claude-sonnet-4-5-20250929",
"apiKey": "${ANTHROPIC_API_KEY}"
},
"openrouter": {
"model": "meta-llama/llama-3.1-8b-instruct",
"apiKey": "${OPENROUTER_API_KEY}",
"costPriority": "high"
},
"onnx": {
"model": "Xenova/gpt2",
"enabled": true,
"privacyMode": true
},
"gemini": {
"apiKey": "${GOOGLE_GEMINI_API_KEY}",
"freeTier": true
}
},
"optimization": {
"strategy": "balanced",
"maxCostPerTask": 0.50,
"preferLocal": false
},
"booster": {
"enabled": true,
"autoFormat": true,
"useMarkers": true
}
},
"sparc": {
"modes": {
"specification": { "provider": "anthropic", "priority": "quality" },
"pseudocode": { "provider": "anthropic", "priority": "quality" },
"architecture": { "provider": "anthropic", "priority": "quality" },
"implementation": { "provider": "openrouter", "priority": "cost" },
"testing": { "provider": "onnx", "priority": "speed" },
"refinement": { "provider": "anthropic", "priority": "quality" }
},
"autoOptimize": true
}
}
}🔌 MCP Tool Integration
Unified Tool Namespace
Existing Coordination Tools (Unchanged):
mcp__claude-flow__swarm_init
mcp__claude-flow__swarm_status
mcp__claude-flow__agent_spawn
mcp__claude-flow__task_orchestrate
mcp__claude-flow__memory_usage
mcp__claude-flow__neural_statusNew Execution Tools:
mcp__claude-flow__agent_execute // Execute agent with provider choice
mcp__claude-flow__agent_list // List available agents
mcp__claude-flow__agent_create // Create custom agent
mcp__claude-flow__agent_optimize // Auto-select best model
mcp__claude-flow__booster_edit // Ultra-fast WASM edit
mcp__claude-flow__booster_batch // Batch edit multiple files
mcp__claude-flow__provider_configure // Configure providersTool Delegation Pattern
// src/mcp/execution-tools.js
const AgenticFlow = require('agentic-flow');
async function agent_execute(params) {
const { agent, task, provider, model, ...options } = params;
// Get provider from config or use default
const selectedProvider = provider || getDefaultProvider();
// Trigger pre-execution hook
await hooks.trigger('pre-agent-execute', { agent, task, provider: selectedProvider });
// Delegate to agentic-flow execution engine
const result = await AgenticFlow.execute({
agent,
task,
provider: selectedProvider,
model,
...options
});
// Trigger post-execution hook (train patterns, update memory)
await hooks.trigger('post-agent-execute', { agent, task, result });
return result;
}🚀 Implementation Phases
Phase 1: Foundation (Week 1-2)
Goal: Basic agentic-flow integration with core functionality
Tasks:
- Add agentic-flow as npm dependency
- Create
src/execution/directory structure - Implement
agent-executor.jswrapper - Implement
provider-manager.jsfor multi-provider support - Add basic CLI commands:
agent run,agent list - Write unit tests for execution layer
- Update package.json and dependencies
Deliverables:
- Working agent execution via CLI
- Provider selection functional
- Basic test coverage (>80%)
Success Criteria:
- Can execute agents with Anthropic provider
- Can execute agents with OpenRouter provider
- All tests passing
- Documentation updated
Phase 2: CLI Enhancement (Week 2-3)
Goal: Complete CLI experience with all agent capabilities
Tasks:
- Implement
src/cli/agent.jswith full command set - Implement
src/cli/booster.jsfor WASM edits - Implement
src/cli/config.jsfor provider management - Add Agent Booster integration (
src/execution/booster-adapter.js) - Implement model optimization engine (
src/execution/model-optimizer.js) - Add custom agent creation CLI
- Add provider testing/validation
- Create configuration wizard for first-time setup
Deliverables:
- Full CLI command suite
- Agent Booster working (352x faster edits)
- Custom agent creation
- Provider configuration UI
Success Criteria:
- All CLI commands functional
- Booster edits working with <10ms latency
- Can create and use custom agents
- Provider config persists correctly
Phase 3: SPARC Integration (Week 3-4)
Goal: Enhance SPARC modes with multi-provider execution
Tasks:
- Update
src/cli/sparc.jsto use execution layer - Add per-mode provider configuration
- Implement auto-optimization for SPARC workflows
- Add
--providerand--optimize-forflags - Update SPARC mode logic to delegate to agentic-flow
- Add performance benchmarking
- Create cost tracking for SPARC runs
- Add detailed logging for provider selection
Deliverables:
- SPARC modes use execution layer
- Per-mode provider control
- Auto-optimization working
- Cost/performance tracking
Success Criteria:
- SPARC TDD workflow works with mixed providers
- Auto-optimization reduces cost by 50%+
- Performance improves by 2x+
- Backwards compatibility maintained (existing commands work)
Phase 4: MCP Tools & Hooks (Week 4-5)
Goal: Expose execution capabilities via MCP and integrate with hooks
Tasks:
- Create
src/mcp/execution-tools.js - Implement all execution MCP tools (agent_execute, agent_list, etc.)
- Update coordination tools to delegate to execution layer
- Add hook integration for execution events:
-
pre-agent-executehook -
post-agent-executehook -
provider-selectedhook -
model-optimizedhook
-
- Implement memory integration (store provider decisions)
- Implement neural pattern training from execution results
- Add token tracking across providers
- Update MCP server documentation
Deliverables:
- MCP tools for agent execution
- Hook integration complete
- Memory/neural integration
- Token tracking dashboard
Success Criteria:
- MCP tools functional via
claude mcp add - Hooks fire correctly for execution events
- Memory persists provider decisions
- Neural training improves provider selection over time
Phase 5: Testing & Documentation (Week 5-6)
Goal: Comprehensive testing, documentation, and release readiness
Tasks:
- Write comprehensive test suite:
- Unit tests for all new modules (>90% coverage)
- Integration tests for execution layer
- E2E tests for CLI commands
- Provider-specific tests
- Booster performance tests
- Update documentation:
- Update CLAUDE.md with execution layer instructions
- Create provider setup guide
- Add optimization best practices guide
- Create migration guide for users
- Add troubleshooting guide
- Create example workflows:
- Cost-optimized development workflow
- Privacy-first local workflow
- Speed-optimized workflow
- Hybrid workflow (mixed providers)
- Performance benchmarking:
- Benchmark each provider
- Compare with v2.5 baseline
- Document cost savings
- Create release notes and changelog
Deliverables:
- Test coverage >90%
- Comprehensive documentation
- Example workflows
- Benchmark results
- Release-ready package
Success Criteria:
- All tests passing
- Documentation complete and reviewed
- Example workflows tested
- Benchmarks show expected improvements
- Ready for v2.6.0 release
🎯 Key Design Decisions
1. Agent Type Mapping
Challenge: Claude-flow has coordination types, agentic-flow has 66+ specialized agents.
Solution: Two-tier mapping system
// Coordination layer defines high-level roles
const coordinationAgents = ['researcher', 'coder', 'tester', 'reviewer'];
// Execution layer maps to specialized agents (configurable)
const executionMapping = {
'coder': 'full-stack-developer', // Default
'researcher': 'technical-researcher',
'tester': 'qa-engineer',
'reviewer': 'senior-code-reviewer'
};
// Users can override mappings in .claude/settings.json
{
"agentMapping": {
"coder": "react-specialist", // Custom override
"backend": "backend-api-developer"
}
}2. Provider Selection Strategy
Four strategies available:
Strategy 1: Explicit (User Choice)
npx claude-flow agent run coder "task" --provider openrouter --model llama-3.1-8bStrategy 2: Automatic (Optimization Engine)
npx claude-flow agent run coder "task" --optimize-for cost
# System automatically selects: OpenRouter + llama-3.1-8bStrategy 3: Configuration-Based
// .claude/settings.json defines defaults per mode
{
"sparc": {
"modes": {
"implementation": { "provider": "openrouter" }
}
}
}Strategy 4: Hybrid (Recommended)
# Uses config defaults, allows CLI override
# Config says "openrouter", but user can override with --provider anthropic3. Hook Integration Pattern
Agentic-flow agents trigger claude-flow hooks:
// src/execution/agent-executor.js
async function executeAgent(agent, task, options) {
// Pre-execution hook
await hooks.trigger('pre-agent-execute', {
agent,
task,
provider: options.provider,
timestamp: Date.now()
});
// Provider selection (auto-optimize if requested)
const provider = options.optimize
? await modelOptimizer.selectProvider(agent, task, options)
: options.provider;
// Provider selected hook
await hooks.trigger('provider-selected', { agent, provider });
// Execute via agentic-flow
const result = await AgenticFlow.execute({
agent,
task,
provider,
...options
});
// Post-execution hook (train neural patterns, update memory)
await hooks.trigger('post-agent-execute', {
agent,
task,
result,
provider,
cost: result.cost,
tokens: result.tokens
});
// Store in memory for future optimization
await memory.store(`execution/${agent}/${Date.now()}`, {
agent,
provider,
success: result.success,
cost: result.cost,
duration: result.duration
});
return result;
}4. Backwards Compatibility Requirements
CRITICAL: Existing workflows must continue working without changes.
Compatibility Matrix:
| Command | v2.5.0 Behavior | v2.6.0 Behavior | Compatible? |
|---|---|---|---|
npx claude-flow sparc tdd "task" |
Uses Anthropic | Uses Anthropic (default) | ✅ Yes |
npx claude-flow hooks pre-task |
Works | Works (enhanced with execution events) | ✅ Yes |
npx claude-flow mcp start |
Works | Works (new tools available) | ✅ Yes |
| Task tool from Claude Code | Works | Works (enhanced with providers) | ✅ Yes |
Migration Path:
- v2.5.0 users upgrade to v2.6.0 → Zero breaking changes
- New features are opt-in via flags or config
- Existing configs continue to work
- New configs add execution layer settings
💡 Example Workflows
Workflow 1: Cost-Optimized Development
# Configure OpenRouter as cost-saver
npx claude-flow config set-provider openrouter \
--model meta-llama/llama-3.1-8b-instruct \
--cost-priority high
# Run SPARC TDD with mixed providers
npx claude-flow sparc tdd "user authentication system" \
--spec-provider anthropic \ # High quality specs
--code-provider openrouter \ # 99% cost savings
--test-provider onnx # Free local testing
# Result: $0.15 → $0.02 (87% cost reduction)Workflow 2: Privacy-First Local Development
# Configure ONNX for local inference
npx claude-flow config set-provider onnx \
--model Xenova/gpt2 \
--privacy-mode true
# All processing happens locally
npx claude-flow agent run security-auditor \
"Audit proprietary code for vulnerabilities" \
--provider onnx
# Use Agent Booster for ultra-fast local edits
npx claude-flow booster edit src/auth.js \
--instructions "Add rate limiting to login endpoint"
# Result: 100% offline, $0 cost, 352x faster editsWorkflow 3: Speed-Optimized Rapid Development
# Use Agent Booster for all code changes
npx claude-flow booster batch \
src/components/*.jsx \
--instructions "Add PropTypes to all components"
# Result: 352x faster than API calls (0.007s vs 2.5s per file)Workflow 4: Balanced Hybrid Approach
# Let the optimizer choose for each phase
npx claude-flow sparc tdd "payment processing" --optimize-for balanced
# System automatically:
# - Uses Anthropic for complex architecture decisions
# - Uses OpenRouter for implementation (cost savings)
# - Uses ONNX for repetitive testing (speed + cost)
# - Uses Agent Booster for quick refactors (352x faster)
# Result: Optimal quality/cost/speed balance🚨 Potential Challenges & Mitigations
Challenge 1: Token Tracking Across Providers
Problem: Each provider has different token counting methods.
Solution:
- Create abstract
TokenCounteradapter - Provider-specific implementations
- Unified tracking API
- Cache token counts in memory
// src/execution/token-counter.js
class TokenCounter {
async count(text, provider) {
switch(provider) {
case 'anthropic':
return await anthropicCounter.count(text);
case 'openrouter':
return await openrouterCounter.count(text);
case 'onnx':
return await onnxCounter.count(text);
default:
return estimateTokens(text);
}
}
}Challenge 2: Agent Definition Conflicts
Problem: Both packages define "coder", "tester", etc.
Solution:
- Execution layer uses mapping configuration
- Allow user overrides in settings
- Clear precedence: user config > package defaults
- Document mapping in settings
Challenge 3: Performance Overhead
Problem: Wrapper layer could add latency.
Solution:
- Minimal abstraction (thin wrapper)
- Direct delegation to agentic-flow
- Async operations to prevent blocking
- Performance benchmarks to catch regressions
Challenge 4: Configuration Complexity
Problem: More options = more complexity for users.
Solution:
- Smart defaults (Anthropic, existing behavior)
- Progressive disclosure (advanced options optional)
- Configuration wizard for setup
- Preset profiles (cost, speed, quality, privacy)
📊 Expected Benefits Summary
| Category | Improvement | Details |
|---|---|---|
| Cost | Up to 99% savings | OpenRouter: $0.001 vs Anthropic $0.15 |
| Speed | 352x faster edits | Agent Booster: 0.007s vs 2.5s |
| Privacy | 100% offline mode | ONNX local inference |
| Flexibility | 4x provider choice | Anthropic, OpenRouter, ONNX, Gemini |
| Agents | 3.3x more agents | 66+ specialized agents vs ~20 |
| Quality | Maintained | Anthropic still available for critical tasks |
🎪 Why This Integration is Powerful
- Unified Experience: One package, all capabilities (coordination + execution)
- Cost Control: Choose provider per task (save 99% on non-critical tasks)
- Speed Options: WASM booster for instant edits (352x faster)
- Privacy Mode: Local ONNX for sensitive code (100% offline)
- Backwards Compatible: Existing workflows unchanged (zero breaking changes)
- Progressive Enhancement: Opt into new features when ready
- Best of Both Worlds: Coordination intelligence + execution flexibility
This integration would make claude-flow the definitive multi-agent development platform with unmatched flexibility, cost control, and performance.
📝 Related Documentation
- Agentic-Flow GitHub
- Claude-Flow Documentation
- Provider Comparison Matrix (TBD)
- Migration Guide (TBD)
- Optimization Best Practices (TBD)
🏷️ Labels
epic, enhancement, integration, high-priority, multi-provider, cost-optimization, performance
👥 Stakeholders
- Engineering Team: Implementation and testing
- Documentation Team: User guides and migration docs
- Product Team: Feature prioritization and UX
- Community: Feedback and testing
✅ Definition of Done
- All phases completed (Phases 1-5)
- Test coverage >90% for new code
- Documentation complete and reviewed
- Backwards compatibility verified (existing workflows work)
- Performance benchmarks meet targets (2x+ speed, 50%+ cost savings)
- Zero breaking changes for existing users
- Release notes and changelog published
- v2.6.0 released and deployed
- Community announcement and migration guide available
Estimated Timeline: 5-6 weeks
Effort: ~200-250 hours
Priority: High
Target Version: v2.6.0