How to Evaluate Candidates Whose Projects Never Shipped

To evaluate candidates whose projects never shipped, examine their individual decisions, actions, and observations against the role’s criteria. Record which outcomes were never tested. Ask only for information they are authorized to share, and offer a fictional task when confidential work cannot be discussed.

A cancelled project can leave a candidate with years of relevant experience and no public release to show. Your interview has to establish what that experience demonstrates. A studio name, a convincing story, and a missing launch each tell you less about the person’s work than a careful account of one decision.

What can cancelled work tell you about a candidate?

Cancelled work can provide evidence about reasoning, execution, and collaboration before release. It cannot supply an observed production outcome that never happened.

On October 21, EA announced that it would stop early Mass Effect development and close BioWare and its Edmonton workspace. EA also said it would support impacted employees in exploring opportunities within the company. The announcement prompted a large Hacker News discussion.

That announcement establishes EA’s decisions. It does not establish anyone’s individual performance, explain every reason for the cancellation, or tell you which employees want an external role. If someone later chooses to apply to your company, assess their evidence without importing a verdict from the news.

The same problem appears outside games. An internal platform gets shelved after an acquisition. A research prototype loses funding. A product team changes direction before customers use its work. In each case, the interviewer needs to distinguish the candidate’s contribution from the fate of the wider project.

The US Office of Personnel Management describes accomplishment records as accounts of relevant experience assessed against job competencies. Those accounts examine the situation, the person’s actions, and the results. They provide a useful starting point for a contribution discussion, although a candidate’s account still needs examination.

The method below is our practical recommendation. No cited source validates this complete cancelled-project interview or proves that it improves hiring outcomes. Its purpose is to make your reasoning clear: what did you learn about this person, what remains unknown, and what evidence would change the decision?

Set the confidentiality boundary before asking for evidence

Give candidates a clear way to participate without sharing restricted information. Establish that boundary before requesting a portfolio, screenshots, source code, or a detailed verbal account.

Cancellation does not give a candidate permission to disclose former-employer material. Neither does a private link, a password, a recruiter’s request, or your offer to sign a new NDA. WIPO’s trade-secret guidance explains that confidentiality duties can continue after employment, with the details depending on the circumstances and applicable law.

Use an invitation along these lines:

Choose an example you are authorized to discuss. Please exclude confidential code, assets, customer information, internal metrics, and other restricted details. You may describe your own contribution at a permitted level. If that is difficult, tell us and we will use a fictional scenario supplied by us.

Apply the boundary to spoken explanations too. A candidate can reveal sensitive architecture or commercial plans without uploading a file. Ask them to pause if a question would require restricted details, and move to your alternative material. You do not need to interpret their agreement during an interview to respect that limit.

WIPO’s trade-secret management guide addresses the risk of receiving a former employer’s secrets and recommends clear instructions and controls when hiring. If restricted material arrives, stop circulating it and follow your organization’s legal and security handling process. Do not promise that redaction, deletion, or a closed interview resolves the issue.

Your existing portfolio process may already allow private-work descriptions and an alternative exercise. Our guide to GitHub contributions as a hiring signal covers that broader evidence problem. For a cancelled project, the additional task is to mark precisely where the observed result ends.

Reconstruct one contribution and mark the unknown outcome

Choose one role-relevant decision and follow it from the candidate’s remit to the evidence available before cancellation. Keep the limits of that evidence visible throughout the discussion.

Start with the competency you need to assess. For a backend role, that could be diagnosing a performance regression. For a technical lead, it might be choosing between a staged migration and a replacement. “Tell us about your biggest project” leaves too much room for the project brand to carry the answer.

OPM’s structured-interview guidance recommends predetermined questions and consistent rating scales and standards. Apply that principle to the contribution you are examining. Prepare the core prompts before meeting candidates, then use follow-ups to clarify their evidence.

Ask about remit, choice, and observation

These prompts are a suggested format, not a validated questionnaire:

1. What were you responsible for, and which parts belonged to other people?
2. What constraint or uncertainty did you personally need to resolve?
3. What decision did you make, and which alternative did you reject?
4. What did you test or observe before the project stopped?
5. What did that observation establish, and what did it leave unknown?
6. What would you need to examine before making the same decision here?

Keep collective achievements and personal actions distinct. “We reduced latency” may describe a real team result. Ask which diagnostic step the candidate performed, what they proposed, who made the final choice, and how people handed over the work. A credible answer can include a limited remit and a colleague’s decisive contribution.

Listen for changes in reasoning. What did the candidate initially believe? Which observation challenged it? Did they alter the implementation, ask for help, or stop an unpromising approach? You are examining decisions under the conditions they faced, without pretending to replay the entire project.

Keep a test result within its actual scope

Consider this fictional example, unrelated to BioWare. An engineer investigated a networking regression in an unreleased game. They can discuss the diagnostic method without disclosing restricted implementation details.

The engineer explains how they isolated a queueing problem, compared alternatives, and tested a change in a controlled environment. That account may give you useful evidence about diagnosis and tradeoffs. The project then ended before release.

Write the boundary beside the result: candidate-reported observation in the described test environment; no production exposure or live-player data available. Do not turn the result into a claim about launch reliability. Do not treat the absent launch as evidence that the change failed either.

Probe the candidate’s understanding of that boundary. Which conditions did the test leave out? What might behave differently with real traffic? What would they monitor after release? Their answers can reveal reasoning about uncertainty, while remaining distinct from evidence of operating a live system.

Use a fictional task when the real example is off limits

When a candidate cannot provide a permitted account, use material you are entitled to supply. Define the competency, expected effort, and assessment criteria before asking them to begin.

OPM describes work samples and simulations as tasks resembling the work of the job. It also notes their development and administration costs and their suitability for competencies expected on entry. A task needs careful design; calling it realistic does not establish its quality.

For the fictional networking example, you might provide a small trace, a simple system description, and an unfamiliar symptom. Ask the candidate to choose an investigation sequence and explain the observations that would change it. Supply enough context that success does not depend on knowing your internal architecture or owning the right portfolio artifact.

You can use our senior engineer design-review interview when architectural reasoning is the target. It already provides an approach based on fictional material. Our guide to structuring code assignments covers the scope and burden of a coding task.

An alternative route should assess the same role competency with a comparable burden. That requires review; two tasks are not equivalent merely because you give them the same score range. Check the required knowledge, available context, time demand, and opportunities to demonstrate the criterion. Publish the route before candidates need to request it.

A gap may remain. A short diagnostic exercise cannot establish sustained production ownership. A hypothetical discussion cannot demonstrate every aspect of hands-on execution. Decide whether the missing competency is essential now, whether another bounded assessment can examine it, or whether you lack enough evidence to make an offer.

Keep any follow-up narrow. Respecting confidentiality should not require an extra weekend project. Avoid asking candidates to produce useful work for your business as their substitute, and explain the purpose of the task you do choose.

Give the candidate the choice before the meeting. A brief scheduling message can describe the contribution discussion, identify the fictional route, and explain how to select it. That lets the candidate prepare an appropriate example instead of discovering the boundary while an interviewer waits. Ask whether the materials are accessible and whether the expected effort is workable. When you change the route, tell the panel what it can assess and keep the same role criteria in view.

Write the decision around evidence you actually have

A hiring decision should connect each important criterion to observed or reported evidence, its limits, and any necessary follow-up. Keep an unassessed competency visible rather than converting uncertainty into a confident score.

Use a short contribution record in the review. This suggested format captures the missing-release problem:

Role competency:
Candidate's remit and collaborators:
Decision examined and alternative considered:
Permitted evidence discussed:
Observation available before cancellation:
Outcome never observed:
Remaining question:
One follow-up, or reason no follow-up is needed:

Record the source of an assertion. “Candidate described a controlled test” and “interviewer observed diagnosis on supplied material” are different kinds of evidence. A detailed narrative does not establish authorship by itself. A public credit also needs interpretation before it can establish the scope of someone’s work.

The next action should follow the gap you found:

These are proposed interviewer actions, not findings about a particular candidate. Declining to disclose restricted material is not evidence of low technical ability. Equally, a strong explanation of uncertainty does not fill every technical gap. Apply your stated criteria to the evidence you obtained.

Have reviewers write their assessments before a group discussion where practical. Ask each person to identify the evidence supporting their recommendation and the most consequential unknown. The debrief can then address a concrete disagreement, such as whether the role needs production experience that the interview has not established.

Keep the evidence and follow-up together in Kit

Your hiring workflow should preserve the permitted evidence, the role criterion, and the unresolved question where the team makes its decision. The assessment method remains your responsibility.

In Kit, you can configure an interview or work-sample stage, provide candidate instructions and a fictional brief, and record scores, comments, and a recommendation in the human review. Reviews retain a rubric snapshot with the criteria used for that review. The workflow gives your team a place to document the contribution and decide whether a follow-up is needed.

Kit does not clear NDAs, automatically redact employer material, verify authorship, or confirm shipped-project credits. An upload or external link does not establish permission to share it. Keep those judgments outside the promise of the tool, and collect only the material your process actually needs.

For your next role, identify any public-release requirement you cannot justify. Replace it with a defined competency, a permitted evidence route, and an explicit way to record what remains unknown. You can start from a role template, or try Kit to keep the brief and team review in the same hiring process.