How to Screen Engineers for Real Skill in the AI Era

To screen engineers for real skill when AI can fake the answers, allow AI in the assessment and score how the candidate directs, verifies, and corrects it. Give them a debug-and-extend task on code they did not write, then run a structured rubric that weights reasoning, output validation, and tradeoff thinking over raw output correctness. The job’s bottleneck has moved from writing code to verifying it, so your interview should measure verification.

You can no longer tell the two candidates apart from their submissions. One directs an AI assistant with judgment, catches its mistakes, and ships something sound. The other prompts the same model, gets working-looking code, and has no idea it is wrong. Both produce a passing take-home. The difference only shows up later, in production, in the review queue, and in the trust your team loses. This is an assessment-design problem, and it is solvable without banning AI or playing detective. For the broader market shift behind this, see the bigger picture on the credential collapse.

The supply problem hiring teams just inherited

The candidate pool is increasingly full of engineers who can ship with an assistant but cannot reason without one. The clearest evidence comes from the institutions training them, where failing grades have spiked and faculty are publicly naming AI overuse as the cause.

In Spring 2026, the failure rate in UC Berkeley’s CS 10 course hit 35.3%, up from under 10% in both Spring 2024 and Spring 2025, according to grade data reported by the Daily Californian. CS 61A logged 10.6% F’s and EECS 127 logged 16.8%. The semester average grade fell to about a C+ (2.3 GPA), well below the department’s 2.8 to 3.3 guideline.

Teaching professor Dan Garcia attributed the collapse to a “vast increase in academic dishonesty” from large language model use, citing nearly 30 students caught cheating on take-home exams in CS 10 alone. Professor Gireeja Ranade noted students struggling with linear-algebra prerequisites; one disclosed that their linear-algebra class ran an “open-internet, open-AI policy” for homework and exams. More than 1,300 UC faculty have since signed a petition to reinstate the SAT and ACT for STEM admissions.

That “open-AI policy” linear-algebra class is the whole problem in miniature. AI did not just help students cheat on a CS exam. It masked the missing fundamentals one layer upstream, so a student can clear a course whose prerequisites they never actually absorbed. By the time that person reaches your pipeline, the gap is invisible on paper and only surfaces when something breaks and the assistant is wrong.

Why banning AI in the interview mis-measures the job

Banning AI in a technical interview tests a workflow no working engineer uses anymore. It optimizes your assessment for a skill the job no longer rewards in isolation, and it is unenforceable on top of that.

AI is now woven into normal development. Sonar’s State of Code Developer Survey 2026, based on more than 1,100 developers, found that 42% of committed code is already AI-generated or AI-assisted, projected to reach 65% by 2027. At Google, CEO Sundar Pichai reported in an April 2026 update that roughly 75% of new code is AI-generated and engineer-reviewed, up from 50% the previous fall (reported via Exponent and Tekedia coverage of the internal program). When three quarters of new code at a flagship engineering org starts with an assistant, an interview that forbids the assistant is measuring a fiction.

Engineering leaders already feel the measurement break. Karat’s 2025–2026 AI Workforce Transformation Report, drawn from 400 engineering leaders across the US, India, and China, found that 71% say AI makes technical skills harder to assess. The same report notes strong engineers are now valued at three times or more their total compensation, which raises the cost of getting the screen wrong. The industry response is to allow AI, not forbid it: around 38% of organizations now permit AI in technical interviews per IEEE-USA’s April 2026 analysis, with New York employer adoption near 25% and projected to climb toward 50%. Companies including Canva, Rippling, Red Hat, Meta, and Shopify already allow it.

Why “detect the cheater” is a losing arms race

Trying to catch AI use is a fight you will keep losing. The detection tooling is behind the cheating tooling, and the gap is widening, so any strategy built on “ban it and catch them” decays the moment you ship it.

The numbers are stark. According to Fabric, which analyzed more than 50,000 candidates, AI cheating on take-home assignments more than doubled from 15% in June 2025 to 35% by December 2025. Modern tools solve standard take-homes in under five minutes and render answers through invisible GPU overlays that never appear in a screen share. Fabric reports 59% of hiring managers already suspect candidates use AI in assessments, and Karat’s leaders estimate more than half of candidates use AI even when told not to. Looking further out, Gartner projects that by 2028, one in four candidate profiles will be fake, built from synthetic text, voice, or deepfakes (cited via Fabric).

You cannot win a rendering arms race against software that hides itself from the screen. So stop trying. The detection failure is not a reason to despair; it is the argument for changing what you measure. If you cannot reliably tell whether AI was used, then design an assessment where it does not matter, because using AI well is exactly the thing you want to observe.

The real signal moved from generation to verification

The scarce skill is no longer producing code. It is judging whether the code an assistant produced is actually correct, and fixing it when it is not. Sonar names this directly: AI’s net effect is a “verification bottleneck,” not a raw productivity boost.

The data underneath that phrase is sobering. In Sonar’s survey, 96% of developers do not fully trust AI-generated code, yet only 48% always verify it before committing, and 38% say reviewing AI code takes more effort than reviewing a human’s. The verification cost does not disappear at the QA gate either. Lightrun’s 2026 State of AI-Powered Engineering report found that 43% of AI-generated code changes still need manual debugging in production after passing QA and staging (reported via VentureBeat).

Read those two facts together. A large and growing share of code is AI-generated, and a large share of it is subtly wrong in ways that survive automated checks and reach production. The engineer who creates value in that world is the one who reads output critically, builds a minimal reproduction, checks the logs, and proves a fix rather than trusting the model. That is the capability your interview needs to surface. If the job’s bottleneck is verification, the assessment should measure verification.

What to actually measure: directs, verifies, corrects

Stop scoring whether the candidate produced a correct function. Start scoring how they direct the assistant, verify its output, and recover when it is wrong. This is the model the leading programs have already adopted, and it generalizes cleanly to a startup-sized loop.

Google’s reported AI-assisted pilot allows an approved assistant in the coding round for junior and mid-level US roles and scores “AI fluency, including prompt engineering, output validation, and debugging skills.” Crucially, candidates who “rely heavily on AI without demonstrating their own understanding” receive negative feedback. That single design choice is the whole thesis: fluency earns points, dependency loses them. DoorDash went further, replacing its traditional coding round with a 60-minute AI-assisted working session on a realistic project, scored on “tool use, approach to debugging, judgment, and communication under real constraints.” DoorDash’s engineering team is explicit that “the true differentiators are decision-making, system-level reasoning, and ownership.”

Interview guides from programs like Formation and Sierra converge on the same validation language: strong candidates “create a minimal repro, read the logs, and write targeted checks to prove a fix works, rather than blindly trusting AI output.” Two task formats expose this best.

The debug-and-extend task on code they did not write

The single most diagnostic exercise is reading and fixing code the candidate did not write. “Can you read someone else’s code, find what’s wrong, and fix it?” is the question AI dependency cannot fake, because the work is verification, not generation. Hand the candidate a small, partially working codebase with a planted logical flaw, the kind of bug that looks correct and passes a naive test. Then ask them to find it, fix it, and extend the system with one realistic feature.

AI is explicitly allowed. What you watch for is how they get oriented in unfamiliar code, whether they reproduce the bug before “fixing” it, whether they trust the assistant’s first suggestion or check it, and how they react when the model confidently proposes something wrong. A dependent candidate pastes the error into a prompt and ships whatever comes back. A fluent one uses the assistant to move faster while owning every decision.

The system-design conversation that forces tradeoffs

Pair the debugging task with a system-design discussion that forces explicit tradeoff reasoning. Not “design Twitter,” but a scoped, concrete decision: how would you cache this, where does it break under load, what do you give up by choosing the simpler option? AI can sketch an architecture diagram. It cannot, in a live back-and-forth, defend a choice you push on and adjust when you add a constraint. The conversation reveals whether the reasoning is the candidate’s or the model’s.

The rubric: five criteria worth scoring

Keep the rubric short, behavioral, and weighted toward judgment over output. Score each candidate on the same five criteria so the signal is comparable across your pipeline.

1. Directs AI well. Clear, purposeful prompts; knows what to ask for and why, rather than typing the whole problem in and hoping.
2. Verifies output. Reproduces, reads logs, writes targeted checks; does not commit on faith.
3. Corrects AI errors. Catches the plausible-but-wrong suggestion and explains why it is wrong.
4. Reasons about tradeoffs. Defends decisions, weighs alternatives, adjusts under new constraints.
5. Communicates and owns. Narrates the thinking, manages scope, takes responsibility for the result.

Why work samples beat credentials and LeetCode

A demonstrated work sample is among the highest-validity hiring signals ever measured, and it has held up for 25 years. The classic Schmidt and Hunter (1998) meta-analysis in personnel psychology puts work-sample validity around 0.54 and structured interview validity around 0.51, with a general-mental-ability-plus-work-sample composite near 0.63. A later revision by Roth, Bobko, and McFarland (2005) settles work samples closer to 0.33, still far above the weak proxies.

The weak proxies are exactly the ones AI now breaks. Résumés, years of experience, and unverified credentials sit near the bottom of the validity tables, and the degree signal is degrading in real time as the Berkeley grades show. Memorized-algorithm puzzles fare no better: a live LeetCode round is just as AI-fakeable and just as off-signal as a take-home, which is why LeetCode-style screens are becoming obsolete. The lesson is consistent across the research and the market. Watch the person do realistic work and reason about it. Do not trust a credential to do that work for you.

There is a fairness dividend, too. DoorDash reports that AI-assisted formats let “engineers with non-traditional backgrounds truly shine,” because the assistant closes the execution gap and the round measures judgment, which transfers from math, physics, or self-teaching. Allowing AI is not a concession to cheaters. It widens the funnel to people who reason well but did not follow the standard CS path.

How to build this in your own pipeline

You do not need an outsourced interview service or a proctoring vendor to run this. You need two stages, configured once: an AI-allowed code assignment built around verification, and a structured live debrief with a real rubric. This is the gap Kit fills for startup-sized teams. For entry-level hiring, the Junior Engineer Pipeline template ships both stages pre-configured.

Kit is an AI-native ATS with a configurable, stage-based pipeline, and two existing stage types map directly onto this design. The code assignment stage is GitHub-template-backed, so instead of a greenfield “build X” prompt you provision a debug-this-codebase / extend-this-partial-system template, the task shape that rewards verification over generation. Kit handles the repo setup, instructions, deadline auto-submission, and archiving. Your instructions can say plainly: “AI is allowed; we will ask you to walk us through how you directed and verified it.” Then the live interview stage carries the rubric, with reviewer assignments and structured team review so the five criteria are scored the same way for every candidate and stay comparable across the funnel.

One honest guardrail. No tool, Kit included, reliably detects AI use; the invisible-overlay data makes that clear. That limitation is precisely why the answer is task and rubric design rather than detection. Kit’s role is to make the right design easy to ship and standardize, not to play whack-a-mole with cheating tools. For the companion piece on the assignment itself, see how to structure code assignments candidates don’t hate.

The shift is straightforward to state and harder to resist than it should be. Stop banning AI, because the job uses it. Stop detecting AI, because you cannot win that race. Instead, design an assessment where using AI well is the visible skill: a debug-and-extend task on unfamiliar code, a tradeoff conversation that AI cannot ghost-write, and a rubric that scores how the candidate directs, verifies, and corrects the assistant. That is how you tell the engineer who commands AI from the one who is merely dependent on it, and it is the difference between a hire who ships and one who fills your review queue.

Frequently asked questions

Should I let candidates use AI in a coding interview?

Yes, for the rounds where you want to measure how they work. The job uses AI heavily, with 42% of committed code already AI-assisted per Sonar, so a no-AI round tests a workflow no one uses. Keep AI use round-specific: DoorDash, for example, allows it in its AI-assisted working session but not in every round. Score how the candidate directs and verifies the assistant, not whether they produced a correct function alone.

How do you detect if a candidate is too dependent on AI?

You do not detect it through proctoring, which the data shows is a losing arms race. You surface it through task design. Give them a debug-and-extend task on unfamiliar code with a planted logical flaw, then watch whether they verify the assistant’s output or trust it blindly. Dependency shows up the moment the model is confidently wrong and the candidate cannot tell.

What is the best technical interview format in 2026?

A debug-and-extend work sample on code the candidate did not write, paired with a scoped system-design conversation, both with AI allowed and a rubric weighting reasoning over output. Work samples carry the highest predictive validity in the personnel-selection literature (around 0.33 to 0.54 per Schmidt and Hunter and later revisions), far above résumés, years of experience, or memorized-algorithm puzzles.

Does allowing AI disadvantage non-traditional candidates?

The opposite. DoorDash reports that AI-assisted formats let engineers from non-traditional backgrounds shine, because the assistant closes the execution gap and the round measures judgment, which transfers from fields like math and physics. Allowing AI widens the funnel to strong reasoners who did not follow the standard CS path.