A source-linked comparison across the complete Podium ADE rubric. 8 criteria have sourced findings for both products; 8 were also cross-checked by a second agent for both. 37 rows have at least one sourced finding, including 37 with an agent-cross-checked finding. No score or winner is inferred.
Each finding carries its own verification status, scope and dates. Humans approve the rubric, templates and prototype; they do not review every finding. An agent source cross-check is not a hands-on product test. A missing sourced finding is not evidence that a capability is absent.
Rubric
0.3 · 51 criteria
Last source check
2026-09-23
Next source check due
2026-10-07
Agent cross-checked for both
8 criteria
Conductor source checks
2026-09-23
Devin source checks
2026-09-23
Access and agent choice
ACC-01
Client access
Judgment rule. List desktop OS and minimum versions, iOS, Android, browser, mobile browser, CLI and editor extensions separately. For each client, record start, steer, approve, review and resume actions, notifications and plan restrictions. Test moving the same task from desktop to phone; distinguish an installable mobile app from a mobile website and from status-only monitoring.
Conductor
Agent cross-checkedpartial
macOS is available; iOS is announced.
Scope
Surfaces: Conductor macOS local workspaces, Conductor iOS announcement · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Browser [unknown] Browser: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Mobile browser [unknown] Mobile browser: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Cli [unknown] Cli: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Editor extensions [unknown] Editor extensions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Task actions [unknown] Task actions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Notifications [unknown] Notifications: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Cross device continuity [unknown] Cross device continuity: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Mobile browser [unknown] Mobile browser: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Task actions [unknown] Task actions: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Notifications [unknown] Notifications: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Cross device continuity [unknown] Cross device continuity: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. List supported harnesses and model providers separately, including custom adapters. Record whether the product embeds a harness, controls its sessions through an API, or only launches its terminal command. Verify at least two different harnesses before describing mixed-harness operation as observed.
Conductor
Agent cross-checkedpartial
Four first-party agents documented.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Other atomic questions remain unverified; no shared scenario was executed.
5 atomic answers
Harnesses [supported] Claude Code, Codex, Cursor, OpenCode. Sources: conductor:home.
Model providers [unknown] Model providers: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Custom adapters [unknown] Custom adapters: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Session integration [unknown] Session integration: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Mixed harness operation [unknown] Mixed harness operation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
ACC-03
Subscriptions, API keys and vendor credits
Judgment rule. Record bring-your-own agent subscription, bring-your-own API account, vendor credits and local models as separate routes. List supported plans, models/modes, multiple-account switching and what works without a vendor account or token purchase. Existing subscription support cannot be inferred from API-key support. Record required extra payments and documented provider restrictions.
Conductor
Agent cross-checkedpartial
Cloud accepts existing subscriptions and API keys.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Provider-specific terms, local models and account switching need verification.
7 atomic answers
Agent subscriptions [supported] Accepted in Cloud. Sources: conductor:cloud-faq.
Vendor credits [unknown] Vendor credits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Local models [unknown] Local models: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Account switching [unknown] Account switching: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Vendor account requirement [unknown] Vendor account requirement: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Provider restrictions [unknown] Provider restrictions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
ACC-04
Onboarding and project prerequisites
Judgment rule. Measure fresh install to first successful task, with manual steps and failures. Test a local directory without Git, a local Git repo without a remote, and a GitHub-backed repo separately. Record whether account, project, host, credential and permission prerequisites are explained before accepting an unusable prompt. Evaluate managed-cloud setup and first-class BYO VPS installer/onboarding independently of hidden settings or a DIY shell recipe.
Conductor
Agent cross-checkedpartial
Git-backed setup with credential checks.
Scope
Surfaces: Conductor macOS local workspaces · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
No-Git and remote-less success were not tested.
7 atomic answers
First task setup [supported] Import or create repository, configure scripts, start chat. Sources: conductor:install, conductor:first.
No git [unknown] No git: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Local git without remote [unknown] Local git without remote: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Managed cloud setup [unknown] Managed cloud setup: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Byo vps onboarding [unknown] Byo vps onboarding: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
ACC-05
Programmable access
Judgment rule. Inventory actual CLI, API, SDK, MCP and webhook operations for create, inspect, steer, stop and review. Record authentication and API stability. Having an MCP client that uses external tools is different from exposing the product to other agents over MCP.
Conductor
Agent cross-checkedpartial
Cloud has a beta API and hosted MCP.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
beta
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
No public SDK or webhook contract verified.
8 atomic answers
Cli [unknown] Cli: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Sdk [unknown] Sdk: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Webhooks [unknown] Webhooks: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Operations [unknown] Operations: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Authentication [supported] Bearer API key or organization-scoped MCP OAuth. Sources: conductor:api, conductor:mcp.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
ACC-06
External tools and extensions
Judgment rule. Record MCP client support, plugins, custom tools, skill/instruction import, integration authentication and permission scope. Name which component provides each connection and whether setup is product-managed or inherited from the underlying harness. Verify one external-tool call and its access boundary where available.
Conductor
Agent cross-checkedpartial
MCP tool use and approvals are documented.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Other atomic questions remain unverified; no shared scenario was executed.
7 atomic answers
Mcp client [supported] Configured MCP tools can be called. Sources: conductor:security.
Plugins [unknown] Plugins: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Custom tools [unknown] Custom tools: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Instructions import [unknown] Instructions import: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Authentication [unknown] Authentication: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Permission scope [partial] Some tool calls request approval. Sources: conductor:security.
Integration owner [unknown] Integration owner: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
ACC-07
Client implementation
Judgment rule. For each desktop/mobile client, record native platform UI, Electron, Tauri, React Native, web/PWA or unknown, plus hybrids. Require official technical docs or pinned source/build evidence. Tauri is not synonymous with fully native UI. Keep implementation choice separate from measured responsiveness and usability.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Planning and context
PLAN-01
Intake and external tracker relationship
Judgment rule. Record local creation, prompt intake, issue import and chat/Git triggers. Identify the authoritative tracker, stable issue IDs, one-way or two-way sync, supported edits and operations requiring a trip back to Linear/GitHub/Jira. Importing or attaching a ticket is not native issue management. Answer PLAN-06 and PLAN-07 separately.
Conductor
Agent cross-checkedpartial
GitHub and Linear issues can start workspaces.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Bidirectional field sync and authoritative issue tracking not established.
9 atomic answers
Local creation [unknown] Local creation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Prompt intake [unknown] Prompt intake: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
External import [supported] Create workspace from GitHub or Linear issue. Sources: conductor:workflow.
Chat git triggers [unknown] Chat git triggers: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Authoritative tracker [unknown] Authoritative tracker: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Stable ids [unknown] Stable ids: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Sync [unknown] Sync: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Supported edits [unknown] Supported edits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
External roundtrips [unknown] External roundtrips: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Devin documents delegation from external issue and chat tools.
Scope
Surfaces: Devin cloud · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
This is a source pass; unfilled atomic questions remain unknown and no shared scenario was run.
9 atomic answers
Local creation [unknown] Local creation: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Prompt intake [unknown] Prompt intake: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Authoritative tracker [unknown] Authoritative tracker: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Stable ids [unknown] Stable ids: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Sync [unknown] Sync: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Supported edits [unknown] Supported edits: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
External roundtrips [unknown] External roundtrips: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record task plans and acceptance criteria, editors, revision history and links to implementation. Test amending a task while work is running, restarting and revisiting it later. A task-specific starting brief, reusable memory and a persistent product specification are different objects; product specs are judged in PLAN-08.
Conductor
Agent cross-checkedpartial
Workspace todos organize longer work.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
A todo list does not establish revisioned acceptance plans or restart guarantees.
7 atomic answers
Plans [partial] Agent-visible todo list. Sources: conductor:todos.
Acceptance criteria [unknown] Acceptance criteria: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Editors [unknown] Editors: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Revisions [unknown] Revisions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Implementation links [unknown] Implementation links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Amendment [unknown] Amendment: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Restart persistence [unknown] Restart persistence: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-03
Hierarchy, dependencies and evolving scope
Judgment rule. Record epics, arbitrary parent/child depth, blockers, readiness, dependency enforcement and state propagation. Start with a simple interactive task, grow it into an epic and retain its identity, discussion, history and outputs. Verify blockers actually prevent dispatch. Record whether a rigid preselected workflow is required, and whether changing scope requires recreating work.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-04
Shared context and memory
Judgment rule. List repository instructions, skills, project docs and cross-session/cross-harness memory. Record scope, provenance, citation, correction/deletion and stale-context controls. Test what survives a new session or colleague takeover. A notes page or shared file is not automatically a maintained spec or managed memory.
Conductor
Agent cross-checkedpartial
Repository instructions and task context are documented.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Managed memory provenance, correction and freshness remain unknown.
10 atomic answers
Instructions [supported] Repository Settings and checked-in files. Sources: conductor:workflow.
Skills [unknown] Skills: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Project docs [unknown] Project docs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Memory [unknown] Memory: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Scope [supported] Project guidance versus workspace chat/.context. Sources: conductor:workflow.
Provenance [unknown] Provenance: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Correction [unknown] Correction: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Deletion [unknown] Deletion: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Freshness [unknown] Freshness: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Takeover continuity [unknown] Takeover continuity: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-05
Architecture and API awareness
Judgment rule. Record code search, repository indexing, external knowledge and multimodal inputs, with source freshness and access scope. Ask an agent to use an existing API under a documented architecture rule, then propose a conflicting change. Check retrieval, citation, conflict detection and review-time enforcement separately. A large context window or an instruction file alone does not prove architectural consistency.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-06
Native issue management and idea backlog
Judgment rule. Enumerate create, edit, discuss, prioritize, label, assign, search, filter, archive, reopen and project/epic views within the product. Save an idea without creating a running agent, refine it with another person, and explicitly mark it ready or dispatch it later. Record separate task/session entities, metadata depth, multiple sessions per task, durable state transitions and whether an external tracker remains necessary.
Conductor
Agent cross-checkedpartial
Workspace todos exist; an independent issue backlog is unverified.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Cannot conclude native issues are absent. Parking before a workspace, hierarchy and rich metadata need direct research.
Edit [unknown] Edit: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Discuss [unknown] Discuss: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Prioritize [unknown] Prioritize: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Label [unknown] Label: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Assign [unknown] Assign: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Search [unknown] Search: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Filter [unknown] Filter: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Archive [unknown] Archive: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Reopen [unknown] Reopen: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Project epic views [unknown] Project epic views: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Idea parking [unknown] Idea parking: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Explicit dispatch [unknown] Explicit dispatch: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Metadata [unknown] Metadata: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Durable states [unknown] Durable states: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
External tracker requirement [unknown] External tracker requirement: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-07
Agent-native issue operations
Judgment rule. Verify an agent can read/create/update the same durable issue IDs and fields humans use, claim/assign work, change state, create dependencies/children and report blockers/results through documented CLI/API/MCP. Check that human views reflect actual writes with attribution, permissions and conflict handling. Distinguish shared authoritative records, synchronized external records, agent-private metadata and prompt-only conventions.
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
The MCP creates and manages workspaces/sessions; those operations do not establish native issue CRUD, shared issue IDs, dependencies or issue claims. No shared human/agent issue scenario was run.
14 atomic answers
Shared ids [unknown] Shared ids: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Read [partial] Mentioning @todos supplies the current workspace checklist to an agent. Sources: conductor:todos.
Create [unknown] MCP workspace/session creation does not establish issue creation; a shared issue API remains unverified.
Update [unknown] Update: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Claim [unknown] Claim: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Assign [unknown] Assign: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
State changes [unknown] State changes: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Dependencies [unknown] Dependencies: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Children [unknown] Children: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Results [unknown] Results: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Human visibility [unknown] Human visibility: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Attribution [unknown] Attribution: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Permissions [unknown] Permissions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Conflicts [unknown] Conflicts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
PLAN-08
Living product specifications
Judgment rule. Record persistent requirements across tasks/releases, editable by humans and agents with attributable revisions and approval rules. Verify requirement-to-task/implementation links, retrieval during later tasks, drift detection and consideration during review/quality control. Amend a requirement after one task ships, then implement another. A one-time spec-to-plan workflow, per-task brief, memory store or unreferenced document does not establish a living product spec.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Coordination and automation
COORD-01
Parallel work
Judgment rule. Record independently running sessions and tasks, documented quotas and observed concurrency. Describe how a user sees running, waiting, failed and finished work. Several chat tabs are not proof that runs execute concurrently.
Conductor
Agent cross-checkedpartial
Multiple workspaces or agents in one workspace run in parallel.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Quotas [unknown] Quotas: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Observed concurrency [unknown] Observed concurrency: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Status visibility [unknown] Status visibility: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Surfaces: Devin cloud · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
This is a source pass; unfilled atomic questions remain unknown and no shared scenario was run.
5 atomic answers
Concurrent sessions [unknown] Concurrent sessions: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Quotas [unknown] Quotas: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Observed concurrency [unknown] Observed concurrency: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Status visibility [unknown] Status visibility: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record assignment to humans or agents, claims, queues, routing and duplicate-work prevention. Test two workers attempting the same task. Distinguish a display label, an advisory lock and enforced exclusive ownership.
Conductor
Agent cross-checkedpartial
Cloud workspaces can be reassigned to teammates.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Agent routing and exclusive claims remain unknown.
7 atomic answers
Human assignment [supported] Reassign to changes workspace assignee. Sources: conductor:collab.
Agent assignment [unknown] Agent assignment: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Claims [unknown] Claims: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Queues [unknown] Queues: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Routing [unknown] Routing: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Duplicate prevention [unknown] Duplicate prevention: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Ownership enforcement [unknown] Ownership enforcement: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
COORD-03
Inspectable delegation and communication
Judgment rule. Record parent/child roles, model/harness/effort choice, messages, result return and cross-harness teams. Distinguish product orchestration, provider-native subagents, installed orchestration skills and scripts. Record activation, auth and experimental flags. Verify actual child identity, live progress, retained results and ability to reopen or follow up after completion. A final sentence claiming two reviewers ran is not proof. Test the documented invocation as well as the shared natural-language request.
Conductor
Agent cross-checkedpartial
API/MCP can delegate and retrieve sessions.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
No parent-child runtime test or retained fleet graph established.
12 atomic answers
Roles [unknown] Roles: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Model harness effort [unknown] Model harness effort: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Messages [unknown] Messages: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Results return [unknown] Results return: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Cross harness [unknown] Cross harness: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Activation [unknown] Activation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Authentication [unknown] Authentication: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Live progress [unknown] Live progress: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retained results [supported] Transcript messages can be retrieved. Sources: conductor:mcp.
Followup [supported] Send or queue subsequent prompts. Sources: conductor:mcp.
Invocation [unknown] Invocation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
COORD-04
Unattended triggers
Judgment rule. Record schedules, event triggers, recurring tasks and completion conditions. Name where the scheduler runs, overlap handling, retries and whether closing the client stops it. A desktop cron recipe is a manual integration unless the product owns it.
Conductor
Agent cross-checkedpartial
Cloud routines support schedules and GitHub actions.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: 0.85.0 · Plan: Pro/Teams · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Recurring tasks [unknown] Recurring tasks: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Completion conditions [unknown] Completion conditions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Overlaps [unknown] Overlaps: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retries [unknown] Retries: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Client independence [unknown] Client independence: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
COORD-05
Intervention and attention
Judgment rule. Record pause, stop, resume, steering, notifications, pending decisions and escalation. Test an in-run amendment and a harmless approval prompt in both chat and terminal modes. Record silent waits, hidden CLI-only questions and whether stop terminates execution while preserving history. Require discoverable errors rather than inferring success from silence.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
COORD-06
Fleet and epic management UI
Judgment rule. Use a disclosed fixture of 12 task/agent states across several epics, including blocked, failed, awaiting input, reviewing and completed work. Locate parent/child relationships, dependencies, attention needs, history and workload without reading every transcript. Record navigation steps and errors. Boards, graphs, waterfalls, timelines or another interface can qualify; no required visual form. State whether the fixture was simulated or used live agents.
Conductor
Agent cross-checkedpartial
Cloud Home filters organize workspaces; an experimental workspace status matrix is documented. Fleet usability was not tested.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Cloud collaboration docs and v0.85.0 changelog (2026-09-09), reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Home filters cover teammate, project and archive state. The matrix is experimental. Parent/child navigation, dependencies and the common 12-task fleet fixture remain unverified.
8 atomic answers
Fleet fixture [unknown] Fleet fixture: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Parent child [unknown] Parent child: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Dependencies [unknown] Dependencies: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Attention [partial] Home/sidebar workspace status matrix is opt-in under Settings → Experimental; effectiveness at fleet scale was not tested. Sources: conductor:release.
History [unknown] History: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Workload [unknown] Workload: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Navigation errors [unknown] Navigation errors: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Desktop advertises a session board and agent command center.
Scope
Surfaces: Devin Desktop, formerly Windsurf · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
This is a source pass; unfilled atomic questions remain unknown and no shared scenario was run.
8 atomic answers
Fleet fixture [unknown] Fleet fixture: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Parent child [unknown] Parent child: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Dependencies [unknown] Dependencies: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Attention [supported] Running and review state views shown in product page. Sources: devin:desktop.
History [unknown] History: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Workload [supported] Spaces and Kanban session management. Sources: devin:desktop.
Navigation errors [unknown] Navigation errors: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Fixture mode [unknown] Fixture mode: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Compare a quick interactive change with a growing epic. Record mandatory workflow/preset selection, natural-language configuration, per-agent model/effort control, changes during execution and whether the UI exposes what actually ran. Test a coordinator with two reviewer children; capture roles, relationships and results during work and after completion. Document discoverability and auth friction without judging an unsupported magic prompt as proof of missing capability.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Execution and reliability
RUN-01
Laptop-independent execution
Judgment rule. List local, user-owned VPS/remote host, self-hosted service, vendor cloud and hybrid modes. Record provisioning, payer, compute resources, region, idle/max lifetime and disconnection rules. Close/disconnect the originating client and continue from another device. A hosted UI or keep-awake setting is not evidence of remote execution. Record managed-cloud convenience separately from BYO setup effort in ACC-04.
Conductor
Agent cross-checkedpartial
Managed Cloud runs after the Mac closes, subject to sandbox sleep and expiry.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Hard sandbox lifetime applies; BYO offering is future work.
12 atomic answers
Local [unknown] Local: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Self hosted [unknown] Self hosted: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Hybrid [unknown] Hybrid: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Provisioning [unknown] Provisioning: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Payer [unknown] Payer: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Expiry [unknown] Expiry: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Other device continuation [unknown] Other device continuation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Hybrid [unknown] Hybrid: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Provisioning [unknown] Provisioning: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Payer [unknown] Payer: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Resources [unknown] Resources: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Region [unknown] Region: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Expiry [unknown] Expiry: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Client disconnect [unknown] Client disconnect: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Other device continuation [unknown] Other device continuation: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record shared checkout, branch, worktree, container, VM and OS sandbox separately. Name filesystem, process, network and secret boundaries. Validate code separation with two changes; do not claim hostile-code containment without separate security evidence.
Conductor
Agent cross-checkedpartial
Local worktrees isolate code, Cloud uses Linux sandboxes.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Cross-workspace security and network restrictions not tested.
12 atomic answers
Checkout [unknown] Checkout: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Branch [unknown] Branch: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Container [unknown] Container: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Vm [unknown] Vm: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Os sandbox [unknown] Os sandbox: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Filesystem [unknown] Filesystem: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Process [unknown] Process: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Network [unknown] Network: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Secrets [unknown] Secrets: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Code separation [unknown] Code separation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Security boundary [partial] Local agents retain user permissions; Cloud sandbox scope differs. Sources: conductor:first, conductor:security.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-03
Recovery, expiry and continuity
Judgment rule. Test client disconnect, worker crash, service restart and documented sandbox expiry safely. Separate transcript reload, process reconnection, task resumption and state reconstruction. Record lost context, duplicated actions, manual repair and ability to continue a multi-day epic after compute replacement. Short sandbox lifetimes are reported with their continuity behavior, not automatically treated as failure.
Conductor
Agent cross-checkedpartial
Cloud preserves files/history but not running processes.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
No multi-day recovery scenario executed.
12 atomic answers
Client disconnect [unknown] Client disconnect: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Worker crash [unknown] Worker crash: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Service restart [unknown] Service restart: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Expiry [supported] 23h50m hard stop; four-hour inactivity sleep. Sources: conductor:lifetime.
Transcript reload [supported] History survives sleep. Sources: conductor:lifetime.
Process reconnect [unsupported] Running processes do not survive sleep. Sources: conductor:lifetime.
Task resume [unknown] Task resume: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
State reconstruction [unknown] State reconstruction: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Context loss [unknown] Context loss: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Duplicate actions [unknown] Duplicate actions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Multi day continuity [unknown] Multi day continuity: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-04
Development environment reliability
Judgment rule. Record dependency setup, node_modules/cache handling, snapshots, services, port allocation and secret injection. Start two workspaces with the same service requirements; note installation duplication, port collisions and repair steps. Repeat on a clean remote host. Record which problems the product handles automatically and which the user must diagnose.
Conductor
Agent cross-checkedpartial
Shared Cloud builds seed new environments; ports require configuration.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Dependency repair, resource contention and cache behavior untested.
10 atomic answers
Dependencies [unknown] Dependencies: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Caches [unknown] Caches: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Services [unknown] Services: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Secrets [unknown] Secrets: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Duplicate installs [unknown] Duplicate installs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Collisions [unknown] Collisions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Remote setup [unknown] Remote setup: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Automatic repair [unknown] Automatic repair: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-05
Limits and failure handling
Judgment rule. Record time, turn, token and spend limits; timeout, retry and provider-failure behavior. Test one controlled failure and a small safe budget. Record whether limits are warnings, enforced stops, or external billing controls.
Conductor
Agent cross-checkedpartial
Cloud enforces sandbox sleep/lifetime limits.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Retries, provider failure handling and spend enforcement unknown.
8 atomic answers
Time limits [supported] Hard maximum 23h50m; inactivity four hours. Sources: conductor:lifetime.
Turn limits [unknown] Turn limits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Token limits [unknown] Token limits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Spend limits [unknown] Spend limits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Timeouts [unknown] Timeouts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retries [unknown] Retries: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Provider failures [unknown] Provider failures: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Enforcement [unknown] Enforcement: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-06
Truthful status and operational statistics
Judgment rule. Record logs, traces, failure reasons, usage exports and statistics by task, agent, project and time. Induce a worker failure and an approval wait: check agent, task and board state agree and remain correct after reconnect. Distinguish working, blocked, failed, idle, ready for review and delivered. For speed/success claims disclose fixture, model, hardware, attempts and failures. UI responsiveness needs observed interaction timings; never infer a framework from lag.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-07
Per-task API and compute cost
Judgment rule. Record visible model/API spend per task, session, child and epic, including aggregation across harness changes. Distinguish billed charges, estimates, tokens, subscription allowance and unavailable amounts; never render unknown subscription marginal cost as zero. Reconcile totals with provider evidence where available. Record compute fees separately, refresh delay, budget alerts, export and drill-down from summary statistics.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
RUN-08
Compute-aware scheduling and verification
Judgment rule. Record concurrency limits, queues, CPU/RAM limits, backpressure, per-host resource awareness and reuse/deduplication of checks. Queue multiple browser end-to-end tests under a declared safe capacity; verify excess work queues or throttles rather than overloading the host. Inspect cancellation, fairness, recovery and whether overlapping epics share limits. Do not run an uncontrolled load test.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Human collaboration and control
TEAM-01
Live human collaboration
Judgment rule. Record shared visibility, issue discussion, comments, presence and simultaneous co-prompting as independent capabilities. Use two named accounts to steer the same agent and record input ordering/conflict handling, attribution and permission controls. Read-only links and shared issues do not establish co-steering. Record team/free-tier restrictions and whether the original owner must stay online.
Conductor
Agent cross-checkedpartial
Shared Cloud workspaces support live chats and presence; private workspaces also exist.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Pro collaboration with up to 5 Pro users; Teams and Enterprise organization collaboration. Teams invite-only. · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Concurrent-input ordering/conflicts and ownership restrictions not tested.
11 atomic answers
Visibility [supported] Organization members can access shared Cloud workspaces; private workspaces are an explicit exception. Sources: conductor:collab, conductor:release.
Discussion [unknown] Discussion: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Comments [unknown] Comments: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Presence [supported] Avatars and typing indicators. Sources: conductor:collab.
Coprompting [supported] Vendor documentation describes prompting together in a shared workspace; input ordering was not tested. Sources: conductor:collab, conductor:cloud-faq.
Ordering [unknown] Ordering: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Conflicts [unknown] Conflicts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Attribution [unknown] Attribution: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Permissions [unknown] Permissions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Plan restrictions [unknown] Plan restrictions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Owner online [unknown] Owner online: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
TEAM-02
Roles and access boundaries
Judgment rule. Record organization/project/task roles, membership lifecycle, SSO and provisioning where offered. Test unauthorized access and revocation with consenting test accounts. Describe documented versus observed enforcement, including which plan provides it.
Conductor
Agent cross-checkedpartial
Private Cloud workspaces and enterprise identity controls are listed.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Revocation and exact role enforcement untested.
9 atomic answers
Organization roles [unknown] Organization roles: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Project roles [unknown] Project roles: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Membership [unknown] Membership: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Unauthorized access [unknown] Unauthorized access: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Revocation [unknown] Revocation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Plan scope [unknown] Plan scope: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
TEAM-03
Approvals and temporary access
Judgment rule. Record command, tool, network, secret and delivery approvals, approver identity and scope lifetime. Demonstrate allow, deny and revocation, including temporary tunnels or shared access after its thread/session ends. Separate default full access, available safer modes and product-enforced policy from instructions the model is expected to follow.
Conductor
Agent cross-checkedpartial
Local tool approvals are documented.
Scope
Surfaces: Conductor macOS local workspaces · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Approver identity, temporary grants and default modes not established.
14 atomic answers
Command [partial] Shell command approval is supported. Sources: conductor:security.
Tool [partial] Some tool calls require approval. Sources: conductor:security.
Network [unknown] Network: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Secret [unknown] Secret: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Delivery [unknown] Delivery: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Approver identity [unknown] Approver identity: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Lifetime [unknown] Lifetime: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Allow [unknown] Allow: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Deny [unknown] Deny: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Revoke [unknown] Revoke: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Tunnels [unknown] Tunnels: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Defaults [unknown] Defaults: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Safer modes [unknown] Safer modes: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Enforcement [unknown] Enforcement: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
TEAM-04
Decision and activity history
Judgment rule. Record attributed actions, approvals, edits, discussion and decision links. Check whether history persists across users and sessions, can be exported, and can be altered. Use the word audit only with a defined retention and integrity model.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
TEAM-05
Review experience and guidance
Judgment rule. Evaluate locating pending reviews, understanding a change, navigating evidence, commenting, requesting changes and finding the next shipping action. Measure steps/time and missed evidence on a shared fixture. Record keyboard/narrow-screen usability, context switching, clarity of agent roles and preservation of review context. Generic diff/PR access and a guided review workflow are different capabilities. Link high-volume review findings to SHIP-06.
Conductor
Agent cross-checkedpartial
Diff comments and suggested PR actions guide review.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Review effort, narrow-screen usability and missed-evidence rates untested.
13 atomic answers
Pending reviews [unknown] Pending reviews: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Change understanding [supported] Unified and commit-filtered diffs. Sources: conductor:diff.
Evidence navigation [unknown] Evidence navigation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Comments [supported] Line comments and GitHub review threads. Sources: conductor:diff.
Request changes [unknown] Request changes: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Effort [unknown] Effort: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Missed evidence [unknown] Missed evidence: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Keyboard [unknown] Keyboard: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Narrow screen [unknown] Narrow screen: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Context switching [unknown] Context switching: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Roles [unknown] Roles: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retained context [unknown] Retained context: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
TEAM-06
Colleague takeover and continuity
Judgment rule. Have a second authorized person take over ongoing work with the same task, session history, decisions, artifacts, permissions and runtime context. Test after the first person disconnects. Record ownership transfer, attribution and lost context. Separately record transferring to another model/harness; a forked transcript or shared link alone is not full human takeover.
Conductor
Agent cross-checkedpartial
Teammates can continue and reassign a Cloud workspace.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Shared Cloud workspace with authorized teammate; paid collaboration plan. · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Artifact/runtime completeness and owner-disconnect test remain open.
Private workspace access is not implied by the shared-workspace documentation.
Same task [partial] Same workspace retained. Sources: conductor:collab.
Session history [unknown] Session history: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Decisions [unknown] Decisions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Artifacts [unknown] Artifacts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Permissions [unknown] Permissions: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Runtime context [unknown] Runtime context: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Owner disconnect [unknown] Owner disconnect: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Ownership [supported] Reassign to teammate. Sources: conductor:collab.
Attribution [unknown] Attribution: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Context loss [unknown] Context loss: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Harness transfer [unknown] Harness transfer: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Human can take control of cloud development tools.
Scope
Surfaces: Devin cloud · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Human control of an agent environment does not establish colleague-to-colleague takeover or shared prompting.
12 atomic answers
Ongoing takeover [unknown] Ongoing takeover: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Same task [unknown] Same task: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Session history [unknown] Session history: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Decisions [unknown] Decisions: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Artifacts [unknown] Artifacts: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Permissions [unknown] Permissions: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Runtime context [supported] Embedded IDE and interactive browser control. Sources: devin:intro.
Owner disconnect [unknown] Owner disconnect: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Ownership [unknown] Ownership: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Attribution [unknown] Attribution: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Context loss [unknown] Context loss: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Harness transfer [unknown] Harness transfer: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record diffs, documents, screenshots, recordings, previews and test/reviewer outputs. Check for a searchable/browsable artifact gallery or equivalent retrieval, linked to task and exact revision, usable after sessions end. Reopen a completed child reviewer and retrieve its full findings. Check whether internal prompts or tool metadata unintentionally appear in the produced application or other user-facing output. In-chat attachments, persistent review evidence and a cross-task gallery get distinct answers.
Conductor
Agent cross-checkedpartial
Cloud .context attachments now sync to the Mac.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: 0.85.0 changelog, released 2026-09-09 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Gallery, revision binding and child outputs not established.
13 atomic answers
Diffs [unknown] Diffs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Documents [supported] Plans in .context sync. Sources: conductor:release.
Screenshots [supported] Screenshots and attachments sync. Sources: conductor:release.
Recordings [unknown] Recordings: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Previews [unknown] Previews: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Test outputs [unknown] Test outputs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Reviewer outputs [unknown] Reviewer outputs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Gallery [unknown] Gallery: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Task links [unknown] Task links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Revision links [unknown] Revision links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Persistence [unknown] Persistence: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Child findings [unknown] Child findings: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Prompt leakage [unknown] Prompt leakage: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Surfaces: Devin cloud · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
This is a source pass; unfilled atomic questions remain unknown and no shared scenario was run.
13 atomic answers
Diffs [unknown] Diffs: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Documents [unknown] Documents: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Screenshots [unknown] Screenshots: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Recordings [unknown] Recordings: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Test outputs [supported] Shell output/logs. Sources: devin:intro.
Reviewer outputs [unknown] Reviewer outputs: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Gallery [unknown] Gallery: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Task links [unknown] Task links: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Revision links [unknown] Revision links: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Persistence [unknown] Persistence: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Child findings [unknown] Child findings: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Prompt leakage [unknown] Prompt leakage: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record tests, lint, types, CI, independent review and persistent-spec conformance checks as required versus optional. Cause a failed test and a conflict with an existing product requirement. Verify review catches the conflict and delivery blocks where claimed. An agent saying checks passed is not a passing gate; record configured and enforced policy separately.
Conductor
Agent cross-checkedpartial
Checks aggregate CI; todos explicitly block merge.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
No living-spec gate or exact-revision bypass test.
11 atomic answers
Tests [unknown] Tests: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Lint [unknown] Lint: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Types [unknown] Types: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Independent review [unknown] Independent review: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Spec checks [unknown] Spec checks: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Requirement conflicts [unknown] Requirement conflicts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Configured policy [unknown] Configured policy: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Enforced policy [unknown] Enforced policy: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
SHIP-03
Git and pull request workflow
Judgment rule. Record branches/worktrees, commits, PR creation, review comments, CI response, rebase and conflict handling. Complete one fixture change through a draft PR. Distinguish an integrated workflow from the agent running Git commands without product state.
Conductor
Agent cross-checkedpartial
Workspace branch-to-PR workflow is documented.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Conflict repair and state consistency not tested.
9 atomic answers
Branches [unknown] Branches: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Worktrees [unknown] Worktrees: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Commits [unknown] Commits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Pull requests [supported] Guided Create PR. Sources: conductor:diff.
Comments [supported] Review comments return to agent. Sources: conductor:diff.
Ci response [supported] Suggested failing-check actions. Sources: conductor:diff.
Rebase [unknown] Rebase: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Conflicts [unknown] Conflicts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Product state [unknown] Product state: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Surfaces: Devin cloud · Version: Public documentation checked 2026-09-23 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
native
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
This is a source pass; unfilled atomic questions remain unknown and no shared scenario was run.
9 atomic answers
Branches [unknown] Branches: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Worktrees [unknown] Worktrees: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Commits [unknown] Commits: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Comments [unknown] Comments: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Ci response [unknown] Ci response: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Rebase [unknown] Rebase: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Conflicts [unknown] Conflicts: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Product state [unknown] Product state: Initial primary-source pass did not establish this question. No installed-product or shared-scenario test was run.
Judgment rule. Record who can approve, merge, release and deploy, plus checks against the exact revision. Verify the documented authority boundary without publishing real changes. PR creation, merge and production deployment are separate capabilities.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
SHIP-05
Completion evidence and rollback
Judgment rule. Record task-to-commit/PR/build/deployment links, delivery status, retained evidence and rollback or revert support. Change the candidate revision after review and examine whether approval/evidence is invalidated. Agent exit alone does not prove delivery.
Conductor
Agent cross-checkedpartial
Local checkpoints support destructive turn reversion.
Scope
Surfaces: Conductor macOS local workspaces · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Not a deployment rollback or immutable evidence guarantee.
9 atomic answers
Commit links [unknown] Commit links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Pr links [unknown] Pr links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Build links [unknown] Build links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Deployment links [unknown] Deployment links: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Delivery status [unknown] Delivery status: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retained evidence [partial] Snapshots are separate private Git refs. Sources: conductor:checkpoint.
Rollback [unknown] Rollback: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Revert [supported] Reverts code and deletes subsequent conversation. Sources: conductor:checkpoint.
Stale approval invalidation [unknown] Stale approval invalidation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
SHIP-06
Batch review and high-volume shipping
Judgment rule. Return to a queue of at least six completed fixture tasks and review a subset. Record summaries, grouping, risk/size filters, retained reviewer findings, artifact navigation, request-changes flow, approval of exact revisions, dependency-aware merge/release queues and clear next actions. Check stale approvals invalidate after edits and duplicate checks respect RUN-08. Distinguish basic diff/PR actions from a coordinated review-and-delivery queue; do not require Podium terminology.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Ownership, privacy and economics
OWN-01
Source and license
Judgment rule. Record license and revision per component, including client, server and hosted extensions. Distinguish open source, source-available and closed components. Repository visibility is not a license classification or proof of hosted parity.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-02
Hosting and exit
Judgment rule. Record self-hosting scope, required vendor services, data export/import, standard formats and account cancellation behavior. Verify what remains usable without the vendor account. Exported code alone does not mean tasks, context and history are portable.
Conductor
Agent cross-checkedpartial
BYO cloud hosting is not currently offered.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Local desktop use is separate; portable history/export unverified.
Vendor dependencies [unknown] Vendor dependencies: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Export [unknown] Export: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Import [unknown] Import: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Formats [unknown] Formats: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Cancellation [unknown] Cancellation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Account independence [unknown] Account independence: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
History portability [unknown] History portability: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-03
Data handling and telemetry
Judgment rule. Record where code, prompts, credentials, logs and artifacts go, retention/deletion, training use, residency and subprocessors. Separately record telemetry disclosure, collection defaults, opt-in/opt-out controls and observed first-run prompts. No consent dialog is an observation, not proof of no telemetry, secret telemetry or unlawful processing. Separate orchestration-vendor and model-provider policies.
Conductor
Agent cross-checkedpartial
Storage differs between local and managed Cloud.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Not specified · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-23
Retention, training, telemetry defaults and deletion need policy review.
15 atomic answers
Code data [partial] Cloud repository/session data stored by Conductor. Sources: conductor:security.
Prompts [supported] Local requests go directly to provider; Cloud synchronizes session I/O. Sources: conductor:security.
Credentials [unknown] Credentials: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Logs [unknown] Logs: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Artifacts [unknown] Artifacts: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Retention [unknown] Retention: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Deletion [unknown] Deletion: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Training [unknown] Training: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Residency [unknown] Residency: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Subprocessors [unknown] Subprocessors: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Telemetry disclosure [unknown] Telemetry disclosure: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Telemetry defaults [unknown] Telemetry defaults: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Telemetry controls [unknown] Telemetry controls: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Consent prompt [unknown] Consent prompt: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Provider separation [unknown] Provider separation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-04
Price and included usage
Judgment rule. Record currency, billing interval, seats/seat minimums, workspace charges, included usage, credits, trial conditions, overages, taxes treatment and separate model/compute costs. Map billing routes to ACC-03. Cite public prices or dated quotes. Per-seat versus pooled billing is a fact, not a universal quality judgment.
Conductor
Agent cross-checkedpartial
Free $0; Pro $50/month; Teams $60/user/month.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Teams is invite-only. Included Cloud hours are not quantified; future usage pricing and provider-account charges must be checked separately.
12 atomic answers
Currency [partial] Public page displays dollar amounts; currency code was not independently established. Sources: conductor:pricing.
Seats [supported] Teams priced per user. Sources: conductor:pricing.
Seat minimum [unknown] Seat minimum: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Workspace fees [unknown] Workspace fees: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Included usage [partial] Cloud included now; quantity unspecified. Sources: conductor:pricing.
Credits [unknown] Credits: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Trial [unknown] Trial: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Overages [unknown] Overages: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Taxes [unknown] Taxes: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Model cost [unknown] Model cost: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Compute cost [unknown] Compute cost: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-05
Scenario cost and controls
Judgment rule. Price stated workloads for one, two and five developers, with fixed tasks, concurrency, tokens and compute-hours. The two-person scenario models Till and Michael collaborating. Include subscription, platform, model and compute charges; separate fixed, variable, included and unknown costs. No invented average task cost. Link cost visibility and reconciliation to RUN-07.
Conductor
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
Devin
No sourced public findingnot-published
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-06
Availability and support
Judgment rule. Record GA/beta/access restrictions, support channels, documented response commitments, release activity, status history and deprecation/export options. Funding, stars and popularity can be context but are not reliability scores. An archived repository does not prove the hosted service is discontinued.
Conductor
Agent cross-checkedpartial
Teams access is invite-only; desktop has a current changelog.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: 0.87.3 changelog; 0.85.0 routines release · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Installed version and SLA terms not inspected.
8 atomic answers
Release stage [unknown] Release stage: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Support [unknown] Support: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Response commitments [unknown] Response commitments: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Status history [unknown] Status history: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Deprecation [unknown] Deprecation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Export options [unknown] Export options: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-07
Enterprise assurance
Judgment rule. Record each claimed assurance separately: ISO standard/certificate scope, issuer and validity; SOC 2 report type, covered service and audit period; report/certificate access and verification date. Distinguish public documentation, vendor assertion, independently accessible evidence and unavailable evidence. A logo, enterprise pricing tier, SSO or a provider certification does not prove this product is certified or audited. Report controls in TEAM-02 separately; no legal-compliance verdict.
Conductor
Agent cross-checkedpartial
Conductor reports SOC 2 Type II; the audit report and covered service scope were not inspected.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Pricing FAQ reopened 2026-09-16; company-level assertion, audited service boundary unknown · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
This confirms the public vendor assertion, not independent audit assurance. Auditor, covered service boundary, audit period and ISO certification remain unknown.
9 atomic answers
Iso scope [unknown] Iso scope: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Iso issuer [unknown] Iso issuer: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Iso validity [unknown] Iso validity: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Soc2 type [supported] Vendor-reported Type II. Sources: conductor:pricing.
Soc2 scope [unknown] Soc2 scope: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Soc2 period [unknown] Soc2 period: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Verification date [unknown] Verification date: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Evidence status [partial] Vendor assertion only. Sources: conductor:pricing.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-08
Useful free tier and evaluation access
Judgment rule. Distinguish an ongoing free tier, time-limited trial, credits-only trial, open-source self-hosting costs, paid-only access and gated early access. Map which shared scenarios work for free, especially two-person collaboration, BYO subscriptions, remote compute, mobile, issue management and review. Record card requirement, usage caps and whether meaningful evaluation needs payment or sales approval.
Conductor
Agent cross-checkedpartial
Local Mac workspaces are advertised as ongoing free use; Cloud and multiplayer start on paid Pro.
Scope
Surfaces: Conductor macOS local workspaces, Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Card/trial conditions and scenario usability untested.
10 atomic answers
Ongoing free [supported] Local Mac workspaces. Sources: conductor:pricing.
Trial [unknown] Trial: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Credits trial [unknown] Credits trial: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Self host cost [unknown] Self host cost: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Paid only [unknown] Paid only: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Early access [unknown] Early access: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Free scenarios [partial] Local parallel agents with own credentials. Sources: conductor:pricing.
Card required [unknown] Card required: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Usage caps [unknown] Usage caps: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Paid evaluation [supported] Cloud/multiplayer start on Pro. Sources: conductor:pricing.
No publishable sourced finding is available for this criterion. This does not mean the product lacks the capability.
OWN-09
Pricing transparency
Judgment rule. Record publicly stated units, included limits, overage rates, compute/model pass-through or markup, minimum commitment, cancellation and sales-only unknowns. Use transparent, partially specified, quote-only or unknown, with named missing fields. A quote-only enterprise price is a disclosed procurement constraint, not proof of dishonesty.
Conductor
Agent cross-checkedpartial
Plan rates public; Cloud allowance is not quantified.
Scope
Surfaces: Conductor Cloud through macOS app and API · Version: Official sources reopened 2026-09-16 · Plan: Not specified · OS: Not specified · Execution modes: Not specified · Region: Not specified · Harness: Not specified
Availability
unknown
Delivery
unknown
Checked
2026-09-23
Agent cross-check
2026-09-23
Check again by
2026-10-07
Cancellation and taxes not checked.
9 atomic answers
Units [unknown] Units: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Included limits [partial] Cloud hours not quantified. Sources: conductor:pricing.
Overages [unknown] Overages: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Model markup [unknown] Model markup: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Minimum commitment [unknown] Minimum commitment: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Cancellation [unknown] Cancellation: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Sales only unknowns [unknown] Sales only unknowns: Initial official website, documentation, pricing and changelog pass did not establish this question; no installed-app or authenticated Cloud test was run.
Free local use; Pro $50 monthly; Teams $60/user monthly and invite-only; Cloud and multiplayer paid. Included cloud quantity unspecified. Vendor reports SOC 2 Type II; report requires request.
