Production specification
Project Virgin: Fleet MCP Control Plane
A safe control plane for operating a fleet of Devin bots through MCP: inventory, desired configuration, work assignment, execution lifecycle, audit history, and health.
01Overview
A control plane for safely operating a fleet of Devin bots through MCP. It centralizes bot inventory, desired configuration, work assignment, execution lifecycle, audit history, and health reporting. The proposed plane exposes an authenticated MCP and a versioned HTTP API from a Cloudflare Worker, uses Cloudflare D1 for durable state, and relies on queue-backed asynchronous dispatch with idempotency, approval gates, and an append-only audit.
This is a proposed baseline: no prior outline or concrete Devin API contract was available in accessible context, so provider endpoints, credentials, and account capabilities must be verified, not assumed.
02Goals and non-goals
Goals
- Discover and manage bots across environments, keeping a single view of deployed Devin bots and their desired configuration.
- Submit, inspect, cancel, and retry work with a full audit trail of every action.
- Provide a stable MCP interface so client tools can drive the fleet consistently.
- Enforce tenant isolation, least privilege, idempotency, rate limits, and approval where required.
- Reconcile desired versus observed state, and keep operating when the provider is slow or flaky.
- Offer resilience through retries, known ambiguity handling, and dead-letter tracking.
Non-goals
- Reimplement the Devin runtime or planning engine.
- Arbitrary remote shell access, direct credential retrieval, or unrestricted proxying.
- Guarantee provider-side exactly-once execution; we track at-least-once semantics and reconcile ambiguity ourselves.
- Multi-provider orchestration in the initial release.
03Architecture
The control plane is composed of these moving parts:
- MCP / API Worker - authentication, authorization, validation, the MCP and HTTP surface, and correlation of requests.
- Fleet domain service - policy, state transitions, idempotency, and transactions live here.
- Cloudflare D1 - durable storage for the registry, jobs, audit events, idempotency keys, and provider mappings.
- Queue - dispatch, reconciliation, retries, and dead-letter handling.
- Dispatcher - drives the Devin adapter and holds server secrets.
- Provider adapter - the only component that talks to Devin.
- Webhook ingress or polling - absorbs provider state changes.
- Logs, metrics, traces, alerts - observability across the whole path.
Request flow through the system:
01client
02authenticated Worker (auth + policy + validation)
03D1 (durable state)
04queue (async dispatch)
05dispatcher + adapter
06provider (Devin)
Callbacks or polling normalize provider state and append to the audit stream. Provider secrets and raw credentials are never exposed past the dispatcher. Dev, staging, and prod each run isolated databases, queues, secrets, and workspaces. CI runs migrations with staged rollout and rollback-compatible schema changes.
04Security
- Short-lived OAuth / JWT or service identity; validate issuer, audience, expiry, and signature.
- Scope rows by tenant derived from the verified identity, never from client input.
- Roles:
viewer,operator,administrator,approver. - Provider tokens live only in encrypted runtime secrets; D1 stores references, not the material.
- Verify webhook signatures, freshness, and replay protection.
- TLS everywhere, restrictive CORS, payload limits.
- Parameterized SQL, strict validation, and rate limits on every entry point.
- Audit mutations, approvals, dispatches, cancellations, retries, and auth failures; redact secrets and full prompts from logs and audit.
- Work that requires approval cannot dispatch until an approver confirms the exact action hash.
- Define retention and deletion policies for all stored data.
05State model
Bot lifecycle
provisioning - active - draining - disabled
draining blocks new assignments but lets in-flight work finish cleanly.
Job states
queued - dispatching - running - succeeded - failed - canceled
Plus approval_required, retry_wait, and unknown. Terminal states are immutable except for annotations; retries create new attempts. Cancellation is best effort, and provider confirmation of cancellation is tracked separately.
Invariants
- Tenant plus operation idempotency uniqueness.
- A job's bot is always tenant-owned.
- Every transition is audited and carries an optimistic version.
- Provider events are deduplicated.
- Policy and approval are checked before dispatch.
06HTTP API
Base path /v1, JSON bodies, RFC 3339 UTC timestamps, and an Idempotency-Key header on all mutations. Errors use a uniform shape:
{ "error": { "code", "message", "request_id", "details" } }| Method | Path | Notes |
|---|---|---|
| GET | /healthz | liveness |
| GET | /readyz | readiness |
| GET | /bots?status=&cursor=&limit= | list bots |
| POST | /bots | register a bot |
| GET | /bots/{bot_id} | get a bot |
| PATCH | /bots/{bot_id} | update a bot |
| POST | /bots/{bot_id}/drain | start draining |
| POST | /bots/{bot_id}/enable | re-enable a bot |
| POST | /jobs | submit work, returns 202 |
| GET | /jobs | list jobs with filters |
| GET | /jobs/{job_id} | get a job |
| POST | /jobs/{job_id}/cancel | cancel a job |
| POST | /jobs/{job_id}/retry | retry a job |
| POST | /approvals/{approval_id}/decision | approve or reject |
| POST | /webhooks/devin | provider webhook, signature required |
Example job payload
{
"bot_id": "bot_123",
"prompt": "...",
"metadata": { "ticket": "ABC-123" },
"timeout_seconds": 1800,
"approval_policy": "default"
}Example response
{
"job_id": "job_...",
"status": "queued",
"created_at": "...",
"request_id": "..."
}Pagination and codes
- Cursor is opaque; default page size 50, maximum 200.
- Inaccessible resources return
404. 409conflict / version,422invalid or policy,429throttling,503transient.
07MCP interface
The MCP interface exposes the same operations as the HTTP API in /v1, with JSON-RPC-style method signatures. Tenant and principal are always derived from authentication, never from client input.
Tools
list_bots({ status?: "active" | "draining" | "disabled", limit?: number, cursor?: string })
-> { bots: BotSummary[], next_cursor?: string }
get_bot({ bot_id: string })
-> { bot: BotDetail }
register_bot({ name: string, provider: "devin", provider_bot_ref: string,
capabilities?: string[], max_concurrency?: number })
-> { bot: BotDetail }
set_bot_lifecycle({ bot_id: string, lifecycle: "active" | "draining" | "disabled",
reason?: string })
-> { bot: BotDetail, approval_required?: boolean }
submit_job({ bot_id: string, prompt: string, metadata?: Record<string, string>,
timeout_seconds?: number, idempotency_key: string,
approval_policy?: string })
-> { job_id: string, status: string, approval_id?: string }
get_job({ job_id: string })
-> { job: JobDetail }
list_jobs({ bot_id?: string, status?: string, limit?: number, cursor?: string })
-> { jobs: JobSummary[], next_cursor?: string }
cancel_job({ job_id: string, reason?: string, idempotency_key: string })
-> { job_id: string, status: string, cancellation_requested: boolean }
retry_job({ job_id: string, reason?: string, idempotency_key: string })
-> { job_id: string, attempt: number, status: string }
get_fleet_health({})
-> { overall: "healthy" | "degraded" | "unhealthy", counts: object, checked_at: string }Tool errors
UNAUTHENTICATEDFORBIDDENNOT_FOUNDVALIDATION_ERRORCONFLICTRATE_LIMITEDPROVIDER_UNAVAILABLEAPPROVAL_REQUIREDINTERNAL
Mutating tools describe their side effects, and run least-privilege plus policy-gated. Approval-required mutations return APPROVAL_REQUIRED and refuse to act until approved.
08D1 schema
SQLite-compatible, UUIDv7 / ULID identifiers, UTC ISO times, foreign keys enabled, and migrations numbered. Provider tokens and webhook secrets are never persisted; retained prompts and results are encrypted, key-versioned, and deletable. Consider D1 scale and archival limits. Domain and audit updates happen in a transaction where possible. Idempotency keys are hashed; a replayed key with a differing request hash is rejected.
CREATE TABLE tenants (
id TEXT PRIMARY KEY, name TEXT NOT NULL, created_at TEXT NOT NULL
);
CREATE TABLE bots (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL REFERENCES tenants(id),
name TEXT NOT NULL,
provider TEXT NOT NULL,
provider_bot_ref TEXT NOT NULL,
lifecycle TEXT NOT NULL CHECK(lifecycle IN ('provisioning','active','draining','disabled')),
health TEXT NOT NULL DEFAULT 'unknown',
capabilities_json TEXT NOT NULL DEFAULT '[]',
max_concurrency INTEGER NOT NULL DEFAULT 1 CHECK(max_concurrency > 0),
secret_ref TEXT,
version INTEGER NOT NULL DEFAULT 1,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
UNIQUE(tenant_id,provider,provider_bot_ref),
UNIQUE(tenant_id,name)
);
CREATE INDEX bots_tenant_lifecycle ON bots(tenant_id,lifecycle);
CREATE TABLE jobs (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL REFERENCES tenants(id),
bot_id TEXT NOT NULL REFERENCES bots(id),
status TEXT NOT NULL CHECK(status IN ('approval_required','queued','dispatching','running','retry_wait','succeeded','failed','canceled','unknown')),
prompt_ciphertext TEXT,
prompt_redacted TEXT,
metadata_json TEXT NOT NULL DEFAULT '{}',
timeout_seconds INTEGER,
provider_task_ref TEXT,
current_attempt INTEGER NOT NULL DEFAULT 0,
approval_policy TEXT NOT NULL DEFAULT 'default',
version INTEGER NOT NULL DEFAULT 1,
created_by TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL,
started_at TEXT,
finished_at TEXT,
error_code TEXT,
error_summary TEXT
);
CREATE INDEX jobs_tenant_status_created ON jobs(tenant_id,status,created_at);
CREATE INDEX jobs_bot_status ON jobs(bot_id,status);
CREATE TABLE job_attempts (
id TEXT PRIMARY KEY,
job_id TEXT NOT NULL REFERENCES jobs(id),
attempt_no INTEGER NOT NULL,
status TEXT NOT NULL,
provider_task_ref TEXT,
started_at TEXT,
finished_at TEXT,
provider_error_code TEXT,
safe_error_summary TEXT,
created_at TEXT NOT NULL,
UNIQUE(job_id,attempt_no)
);
CREATE TABLE idempotency_keys (
tenant_id TEXT NOT NULL,
operation TEXT NOT NULL,
key_hash TEXT NOT NULL,
request_hash TEXT NOT NULL,
response_json TEXT,
resource_id TEXT,
created_at TEXT NOT NULL,
expires_at TEXT NOT NULL,
PRIMARY KEY(tenant_id,operation,key_hash)
);
CREATE TABLE approvals (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL REFERENCES tenants(id),
job_id TEXT REFERENCES jobs(id),
action_type TEXT NOT NULL,
action_hash TEXT NOT NULL,
status TEXT NOT NULL CHECK(status IN ('pending','approved','rejected','expired')),
requested_by TEXT NOT NULL,
decided_by TEXT,
decision_reason TEXT,
created_at TEXT NOT NULL,
expires_at TEXT NOT NULL,
decided_at TEXT
);
CREATE TABLE audit_events (
id TEXT PRIMARY KEY,
tenant_id TEXT NOT NULL REFERENCES tenants(id),
actor_id TEXT NOT NULL,
resource_type TEXT NOT NULL,
resource_id TEXT NOT NULL,
action TEXT NOT NULL,
old_state_json TEXT,
new_state_json TEXT,
request_id TEXT NOT NULL,
created_at TEXT NOT NULL
);
CREATE INDEX audit_resource_time ON audit_events(tenant_id,resource_type,resource_id,created_at);
CREATE TABLE provider_events (
provider TEXT NOT NULL,
provider_event_id TEXT NOT NULL,
received_at TEXT NOT NULL,
processed_at TEXT,
payload_hash TEXT NOT NULL,
outcome TEXT,
PRIMARY KEY(provider,provider_event_id)
);
09Provider adapter
createTask(botRef, prompt, metadata, timeout) -> providerTaskRef
getTask(ref) -> normalized status / result / error
cancelTask(ref) -> accepted / status
verifyWebhook(headers, rawBody) -> boolean
normalizeWebhook(event) -> normalized stateThe adapter enforces a bounded timeout, safe transient retries, rate limits, secret rotation, redaction, and a circuit breaker. It returns an explicit unknown when a provider outcome cannot be determined. A timed-out create may have succeeded, so the control plane reconciles before retrying.
10Dispatch
- A durable outbox is used when D1 and the queue cannot transact atomically; leases, attempts, exponential jitter, and bounded retries protect every dispatch.
- Dead-letter alerts on exhausted retries.
- Validation, auth, and definite provider rejection are never retried.
- Ambiguous create timeout maps to
unknownand triggers reconciliation. - Polling is bounded and adaptive; signed webhooks are preferred where supported.
- Provider events are deduplicated and transitions validated; stale states are reconciled.
11Observability and SLO
Initial SLO targets to validate:
- 99.9% API availability monthly.
- Read p95 under 500 ms.
- Submit acknowledgement p95 under 1 sec, excluding provider execution.
- Dispatch acceptance within 30 sec normally.
- Zero cross-tenant access.
Structured logs carry request_id, opaque tenant_id, actor, bot / job / attempt / provider, and operation, latency, and outcome. No credentials or full prompts are logged. Monitor queue age and depth, provider latency, state counts, stale leases, webhook failures, retries, pending approvals, and D1 errors. Alert on backlog, unknown states, auth failures, provider outage, and migration failure.
12Tests and gates
Tests
- Validators, auth, state transitions, and idempotency.
- D1 migrations, foreign keys, indexes, and rollback compatibility.
- MCP contract tests.
- A fake provider covering timeout ambiguity, retries, duplicate webhooks, cancel races, and rate limits.
- Tenant isolation, security, replay, injection, redaction, and payload tests.
- Load and queue recovery scenarios plus a staging soak.
Gates
- Compatible migrations.
- Alerts and runbooks in place.
- Rollback tested.
- Auth and idempotency suites pass.
- No secret leakage in logs.
- Operator and approver flows validated.
13Milestones
| Milestone | Duration | Focus |
|---|---|---|
| 0. Discovery and contract | 1-2 days | Confirm Devin API, auth, lifecycle, webhooks, polling, rates, cancel, workspaces, test env; adapter contract and threat model. |
| 1. Foundation | 2-3 days | Worker, D1, env, auth, request IDs, CI; health and staging migrations. |
| 2. Fleet registry | 2-3 days | CRUD, lifecycle, tenant, audit, read MCP; role and isolation tests. |
| 3. Job control | 3-5 days | Submit, outbox, attempts, idempotency, status, adapter; fake and staging ambiguity tests. |
| 4. Lifecycle resilience | 2-4 days | Webhooks, poll, cancel, retry, dead letter, reconcile, approval; race and replay tests, dashboards. |
| 5. Hardening and pilot | 3-5 days | Security, load, soak, runbooks, alerts on a limited fleet; SLO evidence and rollback drill. |
| 6. GA | - | Gradual onboarding, capacity, incident rotation, quarterly access and retention review. |
Estimates are ranges and depend on provider access.
14Runbooks
Provider or dispatch outage
- Pause dispatch, preserve the queue, and avoid a retry storm.
- Reconcile
unknownstates before resuming.
Stuck job
- Inspect audit and provider state.
- Never assume success without provider confirmation.
Credential rotation
- Rotate via canary, switch, verify, revoke old, and audit.
Cross-tenant incident
- Contain, preserve evidence, assess logs, notify, then rotate.
15Open decisions
- Actual Devin API and bot / workspace / session model.
- Webhook signature, idempotency, and event lookup.
- Identity and tenant mapping.
- Prompt and result retention, encryption, residency, and deletion.
- Approval and content policies.
- Fleet scale, concurrency, quotas, and SLO.
- Combined versus separate MCP and REST interfaces.
- D1 capacity and archival strategy.
16Principles
The plan sticks to explicit transitions, optimistic concurrency, provider adapters, at-least-once external calls with reconciliation, attributable mutations, and deny-by-default when ambiguous.