AI Hiring Workflow Outage: Keep Recruiting Running

An AI hiring workflow outage plan keeps applications, decisions, interviews, and human communication in your ATS while model-dependent enrichment fails separately. Save the hiring transaction first, queue AI work with source versions, cap retries, expose a manual path, and reconcile stale work before recovery.

AI is no longer confined to a writing assistant in another tab. A model may parse a résumé, summarize an interview, suggest a reply, search records, or call tools that change a recruiting pipeline. If the model becomes part of the path that must succeed before the hiring record is saved, a provider incident becomes your incident too.

You cannot promise uninterrupted automation. You can keep the hiring process understandable and usable when automation stops.

What did the 93-minute AI outage overlap actually prove?

On September 3, 2026, confirmed incidents at Anthropic, xAI, and OpenAI overlapped for 93 minutes, from 14:43 to 16:16 UTC. They did not start together, and the public evidence does not establish a common cause.

Anthropic opened its main incident at 13:26 UTC for elevated errors across multiple Claude models. Its incident record shows Claude.ai, the Claude API, Claude Code, and Claude Cowork among the affected services. Anthropic said it had identified a cause but described it publicly only as an infrastructure issue. The impact ended at 16:16 UTC, two hours and 50 minutes after the first notice. A separate Sonnet 5 incident had ended at 12:56 UTC, so it should not be folded into one continuous outage.

xAI’s disruption began at 13:30 UTC. Its US East API status record shows recovery at 17:07 UTC, while web, mobile, X integration, and other API surfaces reported similar windows. SpaceXAI later referred to an outage at its Memphis compute center. It did not publish whether power, networking, hardware, or software failed.

OpenAI reported impact beginning at 14:43 UTC. Its incident page names ChatGPT and Codex components, not the OpenAI API. A spokesperson attributed the disruption for some users to a routing error. A solution was implemented at 15:17 UTC, but monitoring continued until the incident was marked resolved at 16:55 UTC. Calling this a 34-minute outage would confuse time to mitigation with final resolution.

The overlap punctures a casual assumption: choosing a famous AI provider does not remove operational risk. But timing alone cannot tell you that the providers shared a cloud, network, data center, attack, or traffic surge. Contemporaneous reporting found no provider statement establishing such a link. Treat a common-cause story as speculation unless a postmortem proves it.

For a hiring team, the useful question is not why three status pages were red. It is what your candidates and recruiters could still do while they were.

Which hiring workflows must survive an AI outage?

Your authoritative hiring actions must work without a model. A provider failure may delay enrichment, but it should not erase an application, conceal a decision, or leave a candidate unable to reach a person.

Start by separating transactions from assistance. Transactions change the durable state of the hiring process. Assistance derives information or drafts a possible next action.

The boundary is about authority, not whether a feature feels important. A recruiter may rely heavily on a generated summary, but the original interview notes still need to exist. A drafting assistant may save hours, but the recruiter must be able to write a message directly. A search model may surface likely matches, but an authorized person still needs another way to open a candidate record.

NIST’s AI Risk Management Framework playbook recommends viable non-AI alternatives, human roles, override mechanisms, and contingency processes for third-party failures. Applied to recruiting, that means degraded mode cannot be a blank spinner. The interface should say what failed, confirm what was saved, and offer the next safe action.

Candidates feel this failure too. If intake succeeds but parsing is delayed, tell the candidate the application was received. Do not make them resubmit because an enrichment call timed out. The same durable-path thinking behind an accessible candidate portal applies here: the essential route must keep working when an optional layer does not.

Write down your boundary before an incident. For every model-backed feature, ask: What record exists if this call never returns? What can a person do next? If neither answer is clear, the model probably owns too much of the transaction.

How do you separate the hiring transaction from AI enrichment?

Commit the core record locally, then request AI enrichment as a separate, observable unit of work. The candidate-facing action should not wait for a model response unless the feature is explicitly optional and offers a clear way to continue without it.

A practical path looks like this:

candidate or recruiter action → ATS transaction → durable AI task → bounded worker → provider → versioned result → human review

Suppose a candidate submits an application. Save the candidate, application, consent, attachment reference, and initial stage in one database transaction. Only after that transaction commits should the system enqueue résumé extraction or summarization. If the provider is unavailable, the application remains real. The recruiter sees “summary pending” rather than an empty record, and the candidate receives confirmation based on the saved application, not on the enrichment result.

A durable queue absorbs work while a dependency is unavailable, but it is not a complete resilience strategy. Microsoft’s queue-based load-leveling guidance calls out the need for queue-depth monitoring, bounded processing, idempotent consumers, dead-letter handling, and ordering decisions. Without those controls, recovery can turn a provider outage into a backlog storm.

Each task should carry an immutable source version. That can be a résumé revision, interview-note version, job-criteria version, or message-template version. Before a worker writes a result, it should compare that version with current state. If the candidate replaced the résumé or the application moved forward, the old task is no longer merely late. It is stale.

The safe outcome is often to supersede stale work, not replay it. Keep the old task for audit purposes, mark why it was skipped, and enqueue a current version only if the result is still useful. This protects recruiters from reading a polished summary of data that no longer governs the application.

Your interface needs the same precision. Use distinct states such as pending, running, retrying, failed, superseded, cancelled, and complete. “AI unavailable” is more useful than an endless loading state. “Application saved; résumé summary delayed” is better still because it identifies both the successful transaction and the failed enrichment.

How do you make retries safe before adding provider failover?

Retries are safe only when repeating a request cannot repeat a business effect. Solve that before adding another model provider, because failover creates another route through which duplicate or stale actions can escape.

AI tasks fall into two risk classes. Regenerating an internal summary twice wastes money and may create conflicting artifacts. Repeating a candidate-facing or state-changing action can do direct harm: two rejection emails, two interview bookings, two stage moves, or an offer sent after withdrawal.

Give each operation a business-level idempotency key. A useful shape is:

account + application + capability + source_version + workflow_step

Persist that key with the task and any resulting artifact or side effect. Before sending, booking, rejecting, or moving a stage, check both the key and current local state. AWS’s guidance on making retries safe with idempotent APIs explains why a caller-provided identifier is more reliable than trying to infer whether two similar requests express the same intent.

Then choose one retry layer. An SDK may already retry. Your background worker may retry too. A workflow engine may add another loop. If each layer makes three attempts, one failure can produce far more calls than your policy appears to allow. Use typed transient errors, exponential backoff, jitter, a cap, and a deadline. Authentication failure, invalid input, and a rejected safety request are not reasons to keep knocking.

Cross-provider failover is not the first control to add. Different model names can still share identity, networking, regional capacity, or another control-plane dependency. OpenAI’s July 2026 postmortem described a regional identity-capacity problem where automatic failover redirected too little traffic. Real independence has to match the failure path.

If you do add a fallback, define what it is allowed to do. A second provider may be acceptable for a low-risk internal summary. It may not be acceptable for a sensitive résumé if contracts, residency, retention, or model behavior differ. Outputs can also vary enough to break downstream assumptions. Keep provider and model details in the audit record, validate the fallback result, and preserve the no-AI path.

How do you test a five-step degraded-mode runbook?

A useful outage runbook is short enough to follow under pressure and explicit enough to prevent improvisation. Test these five steps with the model connection disabled, not with a slide deck.

1. Declare degraded mode and protect the core path

Confirm the provider symptom, then disable or bypass noncritical synchronous calls. Keep application intake, record access, manual stage changes, interview administration, and human-written communication available. Put a plain status message where recruiters will see it. Record the incident start and the person coordinating recovery.

2. Contain retries and preserve intent

Open the circuit for the affected capability or pause its consumers. Do not delete queued work. Stop stacked retries before they amplify load or exhaust rate limits. Preserve each task’s idempotency key, source version, deadline, provider request ID, and last typed error.

Google’s SRE guidance on cascading failures recommends degraded results, deadlines, load shedding, and careful retry behavior because retry traffic can prolong an overload. Your aim is to reduce pressure while keeping hiring intent intact.

3. Route people to a manual path

Give recruiters capability-specific instructions. Open the stored résumé instead of waiting for a summary. Use keyword or database search instead of embeddings. Write the candidate message directly. Record the stage decision in the ATS rather than asking an agent to perform it. Assign owners to time-sensitive applications and interviews so “AI pending” does not become “candidate forgotten.”

If your team uses an assistant to operate recruiting tools, revisit the permission and approval boundaries in deploying AI recruiting agents with MCP. The tool contract may remain available while the model driving it does not.

4. Probe recovery with a limited budget

Do not release the entire backlog when a status page turns green. Send a small set of current, low-risk tasks through a half-open circuit. Watch latency, error rate, rate-limit responses, queue age, and output validation. Restore consumers gradually and reserve capacity for new candidate work.

5. Reconcile before replay

Recovery is a database comparison, not a queue drain. For every pending or failed task, ask whether its source version is current, its deadline still matters, an equivalent artifact already exists, or a human completed the action manually. Reconcile external side effects such as email and calendar bookings before any retry.

Prioritize current candidate communication and active requisitions. Mark stale work superseded. Cancel work whose business moment has passed. Replay only current tasks with valid idempotency keys, then compare expected and actual records. Preserve counts of completed, skipped, cancelled, and manually resolved tasks in the incident timeline.

Run this exercise quarterly and after material workflow changes. Measure whether an application can still be submitted, whether a recruiter can identify failed enrichment, and whether recovery produces zero duplicate candidate-facing actions. Those outcomes matter more than a nominal uptime percentage.

What does Kit keep working, and where does it still depend on AI?

Kit keeps core hiring records and many manual workflows separate from AI enrichment, but it does not promise an offline hiring stack or health-based cross-provider failover. The honest continuity boundary is that the hiring record can persist while model-backed assistance fails visibly.

Public application intake saves the candidate, application, consent, submission, and initial stage in a database transaction before post-commit résumé extraction runs. If extraction fails, the application does not disappear with it. Manual stage moves persist the canonical pipeline state whether they originate in the web interface or through an MCP tool. The model choosing to call that tool is a separate dependency.

Recruiters can create interview invitations, candidates can confirm native bookings, and teams can preserve human-written messages and Liquid templates without a model call. Those paths still depend on systems such as calendar availability, Google Meet or Calendly provisioning, background jobs, and SMTP delivery. Calling them fully offline would be false.

Kit also has bounded fallbacks in specific places. AI reply drafting can return a deterministic composer prefill when the provider or transport fails. Some semantic searches fall back from Gemini embeddings to PostgreSQL text search. Application and candidate summary tools serialize stored records rather than generating prose inside the tool. A connected model may interpret those records, but it does not own them. Our guide to MCP for hiring explains that separation between the model and the recruiting tools it can request.

The gaps are just as important. Kit does not monitor provider health and switch automatically between Gemini, Anthropic, and OpenRouter. Model-tier degradation remains within the configured provider. Generic chat failures surface as errors rather than entering a universal replay system. Failed résumé and metafield extraction can require manual recovery, and embeddings remain tied to Gemini even if chat uses another provider. Kit does not replay every failed enrichment, nor does it remove calendar, SMTP, or model dependencies.

That is the standard to apply to any AI-native ATS: when the model status page turns red, the hiring record should not. Keep applications and decisions authoritative, keep AI work derived and recoverable, and make the human route obvious before you need it.