Tasks, Schedules, and Actions

Add or Change Task Behavior

• Keep task input on the task specification rather than duplicating prompt payload in run requests.
• Allocate immutable organization-local identifiers transactionally from the owning department.
• Add dependencies only through the cycle-checking service.
• Keep dependency conditions explicit: success, completion, or artifact.
• Start only due, allowed, dependency-ready work with no conflicting active execution.
• Scheduling, Context preparation and transactional admission share the task-state readiness rule. A repaired dependency can release a task whose status still says blocked; canonical edges must all be satisfied, and human, approval, integration and other blockers remain enforced. Validate recovery through the full worker-start service, not just its admission repository.
• Replace a failed required cross-root edge with one bounded recovery review. Keep same-owner recovery in that department and route cross-owner recovery to Operator.
• Treat an integration task input as required unless it explicitly sets required: false.

A missing required integration does not run the model and does not consume aggregate run budget. The worker creates one idempotent Human Inbox question and finalizes the attempt as waiting_human; answering the blocker resumes through a fresh request. Optional integrations may be absent without blocking execution.

Completion Contracts

Use backend-decidable criteria wherever the postcondition is mechanical:

• output_file matches a safe outputs/ path and may check size, SHA-256, detected media type, bounded UTF-8 text, or RFC 6901 JSON Pointer values;
• command_exit matches the exact command hash and expected exit code; and
• integration_call matches connector key, tool name, and canonical arguments when supplied.

The trusted worker derives sanitized receipts from actual harness events and synchronized files. The backend rechecks that a file receipt points inside the active run workspace, reads the canonical object, and evaluates the contract before the task transition. Text and JSON inspection is limited to 1 MiB. Media type comes from file bytes, never the extension or an agent claim.

The worker does not stage a second execution result before completion. The completion transaction creates the immutable execution outcome from the exact post-sync summary and artifact list, then evaluates criteria and advances the task. Agent runs may stage only branch-specific semantic outcomes. An already-staged trusted execution result remains subject to exact equality with the worker-retained result, so stale or conflicting output cannot be accepted.

An empty criteria list closes from a successful execution result and must not create a verify run. Use it when the operation is self-verifying and no separate postcondition adds value. Legacy text, metric, and artifact criteria remain compatible as display-only expectations. They neither gate completion nor create another run. If semantic judgment must gate downstream work, create an explicit review task with a bounded evidence contract.

Verification remains available for profile learning and retained pre-cutover task states. It uses the ordinary unrestricted harness, but the prompt prohibits workspace mutation and external effects; the manifest has no output directory, integration grants, MCP servers, or network; workspace sync rejects every change; and backend authorization accepts reads plus the one mode-legal verification outcome only.

The first failed completion report uses one automatic execution repair when budget remains. A repeated failure creates one idempotent Human Inbox repair request; answering queues a fresh versioned execution. Because this question follows completed execution, it does not refund that run’s consumed credit. Budget exhaustion produces a terminal failed result with the completion report instead of another loop.

Run Requests and Attempts

run_requests hold task-only mutable queue/retry intent. runs are immutable attempts. Workers create attempts and increment attempt counts. Scheduler and runbeat never create attempts.

Scheduler-created request identity includes the task, mode, task version, graph revision, and total immutable attempt count. Repeated scans of one generation are harmless, but a terminal request followed by a same-phase completion repair receives a fresh identity. Attempt retries stay on the existing request. PostgreSQL dedupe and active task/mode constraints remain the final guard when multiple scheduler processes race; those expected conflicts are handled as no-ops.

Every task attempt receives a backend-authored prompt that names its selected run mode and exact Task Runtime exit. If an agent-controlled semantic phase returns without its required typed outcome, the backend rejects completion and the worker records a retryable protocol failure. Runbeat retries that same phase only within its existing attempt and task-run budgets.

Non-verification attempts also receive the work/ versus outputs/ boundary. Before an approval or human wait can discard source work, Task Runtime writes outputs/handoff/source.patch and outputs/handoff/continuation.md. The waiting outcome keeps its existing refund behavior, and the resumed attempt applies the retained patch to a fresh checkout rather than relying on a local branch or harness session.

tasks.agentRunCount is total immutable attempt history. tasks.consumedAgentRunCount is the aggregate budget counter used at claim time. Claim increments both in the same transaction; successful human_interrupt and action_interrupt finalization refunds only the consumed counter.

An action handoff validates receipts owned by the same run, task, and task version. It must include every unresolved action; a named action remains valid after success, failure, rejection, or cancellation. Completed inline actions need not be named. Finalization blocks only on unresolved actions and leaves an already-resolved handoff pending for scheduler continuation. Run completion locks run, request, then task; action resolution locks task before updating its action. This closes the callback-before-handoff race. With multiple action blockers, only the final resolution advances the task version and queues continuation, preserving sibling authorization and execution class. Unknown outcomes remain blocked until reconciled; continuation never replays a successful write. The worker activity log records “Result ready for finalization” before the backend call. Only the backend lifecycle state establishes run success; preparing output cannot prove finalization. Failures and backend-opened completion repair waits remain charged. Do not infer either count from Redis delivery.

A deterministic required-integration blocker is also finalized through the worker boundary. The backend derives its human_interrupt outcome from the exact waiting output and staged question in the same transaction; no model run or competing outcome write is involved.

Do not accept maxCostMinorUnits until the runtime records actual metered cost. A migration removes legacy values instead of enforcing a synthetic zero-cost counter.

Late writes must include and satisfy the relevant lease, state, and version checks. The backend issues lease expiry and heartbeat cadence; workers never submit their own expiry. Preserve failed and timed-out attempts.

Public run reads use a dedicated joined projection containing task/profile identity, attempt, mode, status, outcome, error, usage, and timestamps. Never return the execution prompt, hydrated context, working state, worker/lease fields, or workspace/checkpoint/trajectory/log locations through the owner API. Run-history cursors bind the sort key, direction, value, and run ID so changing sort order cannot reuse a stale cursor.

Run activity is a separate owner-safe child collection, not a JSONB array on runs. Append entries row-wise under (organizationId, runId, sequence), fence worker writes by the active lease, and serve at most 100 ascending entries per page with before/after sequence cursors. The run’s logState distinguishes pre-feature, active, complete, and partial retention. Activity delivery is best-effort observability and must never change the run outcome.

Admission serializes the shared worker pool and starts the oldest eligible request in FCFS order. It locks organization, profile, and task state before inserting an attempt. A full pool or earlier eligible request defers admission without charging work. Organization and profile caps are retired. See Worker Capacity for ordering and deployment configuration. Runbeat selects active attempts by canonical leaseExpiresAt, closes an expired run, cancels its staged questions/actions, removes its active task reference, and moves the request to backoff in one PostgreSQL transaction. Exhaustion fails the request and task in that transaction. The same transactional recovery also closes legacy running tasks whose current request already exhausted before task finalization. Failed-run repair only transitions claimed requests once; the scheduler owns later due publication. Background pagination must use the same key as its cursor.

Work Dispatch Control

Organization workDispatchStoppedAt suppresses new Redis worker wakeups without changing durable task, schedule, request, or run state. Scheduler and resume paths may continue creating queued requests while stopped, but must not publish them. Admission also rejects already-published wakeups while stopped; active runs continue. Clearing the timestamp republishes every due queued or retry_wait request from PostgreSQL.

Do not model this as organization pause, cancellation, a Redis-only flag, or an onboarding emergency phrase. Add every new wakeup-producing path through the shared dispatch guard.

Manual Graph Retry

Terminal failure stores failureOriginTaskId on the failed origin and every ancestor or sibling cancelled by that cascade. A manual retry locks the root graph, resolves that causal marker, and reopens only tagged failed/cancelled tasks in depth order. Clear runtime blockers and terminal results, advance task and graph versions, and reset dependencies only when both endpoints reopen.

Preserve immutable runs, counters, completion history, completed tasks, unrelated manual cancellations, and aggregate budgets. Refuse retry when the failure cannot be resolved or any reopened executable task has exhausted maxAgentRuns. Legacy exact failure reasons may be read for pre-marker graphs; new writers always persist the typed marker.

Add a Schedule

1. Define owner, cadence, run mode, and expected task outcome.
2. Store nextRunAt as the schedule cursor.
3. Let the scheduler create normal tasks and run requests.
4. Publish a wakeup only after durable state changes.
5. Make duplicate scans and wakeups harmless.
6. Keep healthy polls quiet.
7. Mark built-in schedules as system; persist the creating profile and exact run/chat provenance for agent schedules.
8. Coalesce missed intervals into one catch-up firing, then advance to the next future occurrence.
9. Back off firing failures exponentially. Disable an agent-created schedule after five consecutive failures. Keep a built-in system schedule enabled, mark it as requiring attention, and continue bounded retries.
10. Test pause, resume, due calculation, duplicates, backoff, provenance, and restart recovery.

System schedules remain organization-owned and read-only to agent execution capabilities. Department agents may schedule only their own department, Operator may delegate any department, and Chat may schedule work only from an explicit active user request.

Backend startup reconciles required built-in schedules for each active organization. It creates missing schedules, repairs stale templates or disabled cursors, and disables orphaned profile learning schedules. During each scheduler tick, due run-request retries publish before schedule firing, and one failing phase cannot starve later task repair.

Schedule lists accept status, origin, profile, and department filters. Their opaque cursor binds the selected ordering and explicitly preserves PostgreSQL null ordering for optional next/last run times; never combine an ID-only cursor with a different sort column.

Add a Human Question

Stage a typed blocker during an active run. Open it only when run finalization moves the task to waiting-human. The response removes that exact blocker, advances the task version, and queues fresh execution only when no blockers remain.

Cancel Task Work

Cancellation is transactional across the requested task and every active descendant. Cancel their queued requests, active attempts, staged/open questions, and actions that have not started. Publish one best-effort abort signal per active run after commit; durable run state remains authoritative. Preserve terminal descendants and action outcomes that are already executing or uncertain.

Resolve outgoing dependency edges by their declared condition: cancellation satisfies completion, fails success and artifact, and blocks a dependent only when an unresolved edge is required. A later action reconciliation must never reopen a terminal task.

A failed required dependency between separate roots creates one version-bound review task. The failed edge remains as non-required history and a required completion edge points to the recovery review. Completing, failing, or cancelling that bounded review releases the dependent for a fresh decision; it does not recursively create another recovery. Dependencies inside one failing root do not create orphan recovery work because the root closes together.

Automatic replans stop at the task budget, which is at most three. An amendment at that boundary opens one idempotent human question and refunds that waiting run credit. If the human authorizes a final amendment, the next review may apply it without increasing the bounded replan counter and marks the override used. Any later amendment fails the task deterministically instead of opening another question or leaving run finalization in conflict.

Add a Consequential Action

1. Define typed arguments and result.
2. Establish stable idempotency.
3. Snapshot authorization epoch, task version, risk, and policy.
4. Wait for approval or acquire a fenced execution lease.
5. Attempt the provider call once.
6. Store a redacted receipt for confirmed outcomes.
7. Mark uncertain outcomes unknown.
8. Reconcile through a human without replaying the effect.

Serialize action admission for each task and validate its current version, active run and owner. Autonomous consequential actions require an enabled department profile executing a task. Research, review, planning and operator task runs remain read-only. Chat actions use a separate recorded owner request, with no fabricated task or run. There are no per-task or daily action quotas; integration grants, selected resources, approvals, provider rate limits and bounded run/retry/replan budgets remain enforced.

Recovery Tests

Cover duplicate wakeups, missed wakeups, stale heartbeats, exhausted attempts, late worker writes, refunded waits, output-sync failure, a 100-file output set, a rejected 101-file output set with zero uploads, missing required versus optional integrations, cross-organization capabilities, cross-department/root mutations, multiple blockers, exhausted replans, the single replan override, stale approvals, and abandoned action leases. Also cover cursor reuse under a different sort, nullable schedule-date pagination, and public run projection redaction. Completion coverage must include empty criteria, mixed formal and display-only criteria, absent or spoofed receipts, byte-detected media, the 1 MiB structured-file bound, exact static matching, one automatic repair, repeated-failure human repair, budget exhaustion, retained-state compatibility, and verification capability fencing.

List pagination

Task and Human Inbox lists return opaque nextCursor values. Pass them unchanged with the same sort key and direction. Restart from the first page after changing filters or ordering; old UUID-only task/inbox cursors must also restart. Stable ID tie-breakers, explicit null ordering, and exact database timestamps prevent skipped or repeated rows in an unchanged result set. These live lists are not a frozen snapshot: concurrent edits can move an item between pages.

Bulk operations

Tasks, chats, notifications and pending approvals offer Bulk actions. Select individual items across pages, the current page, or every item matching the current filters (up to 5,000). Review the frozen selection and confirm within 15 minutes. New matching items are excluded; changed items are skipped. Task cancellation and retry also fence the task graph because descendants can be affected.

Progress survives navigation and restarts. Reopen Recent operation on the same list. Retry applies only to failed items, with at most three attempts; skipped items need a new preview. Approval permits execution through the existing action policy and authorization checks. Unknown outcomes are never automatically retried.

The owner API is /organizations/{organizationId}/bulk-operations: create /preview, inspect a saved operation, then /submit or /retry. Preview items paginate independently from aggregate progress. These owner controls do not add an agent CLI capability. PostgreSQL durability rows hold the frozen selection and item results. The scheduler commits each database transition and its result together; existing workers dispatch any resulting work after commit.

Agent schedule edits

oblivectl schedule show <key-or-id> reads the current definition. schedule update uses the same shared request schema as the internal GET/PATCH schedule API. PATCH requires expectedUpdatedAt, locks the current schedule row and rejects stale edits with 409 schedule_changed. Nested task changes merge with the current definition; provenance, owner, emitted work and paused status are preserved. Timing changes must produce a future next run. Department agents remain limited to their own schedules; Chat and Operator can manage ordinary schedules across departments. System schedules are read-only through the agent boundary.

Owner Operations In Chat

actions.run_id is nullable only for Chat-originated actions. source_chat_id, source_message_id and source_user_message_id identify the conversation, assistant turn and human request. A database check permits one origin shape. Chat approval uses requestedByMessageId; autonomous approval continues to use requestedByRunId. No synthetic task or run is created.

All supported MCP and companion CLI writes use the same action executor, provider validation, idempotency, leases, bounded retries and unknown-outcome recovery. Chat CLI exit 77 carries an action ID for pending or uncertain results. Agents read it with oblivectl action show; they must not claim completion or repeat the effect with a new key. Native Git pushes remain owned Engineering tasks.

The internal /actions and /inbox aliases reuse owner controllers and validators behind a signed, active Chat-turn guard. They support list/show, approval/rejection/confirmed-result recording and Inbox answer/acknowledge. Mutations revalidate the human source inside the database transaction. Autonomous task identities and service tokens alone cannot invoke these owner aliases.

Verification covers schema and CLI boundaries, real PostgreSQL admission/execution/recovery, and a complete HTTP journey with signed Chat and task capabilities. No model run or live provider write is needed for these checks. Writing quality and deciding whether a natural-language request authorizes an effect remain instruction-level behavior, not a deterministic schema guarantee.