AI improves the work. Does it improve the people doing it?
Bespoke research that measures, before and after AI adoption, whether your employees keep, build, or lose their own professional judgment.
RETAIN (Retained Expertise Tracking in AI-enabled work)
-
Overview
Most organizations measure what AI helps people produce: speed, volume, and the quality of the output. Few measure what their people can still do without it. That gap matters wherever a person must catch a plausible wrong answer, handle an exception the tool has not seen, or take over when the tool is unavailable. These two studies come from very different settings and show a similar pattern: performance with AI tends to rise, and performance without it can fall. Neither study tells you what is happening in your organization, on your tasks, with your people. RETAIN does.
-
28.4% to 22.4%
Adenoma detection rate in colonoscopies performed without AI, before versus after endoscopists were exposed to AI. Observational study across four endoscopy centers in Poland. (Budzyń et al., 2025)
-
+48%, then −17%
High school math students using a general-purpose AI tutor scored higher on assisted practice, then 17% lower than the no-AI group on a later exam without AI. A tutor with built-in safeguards largely avoided the drop. (Bastani et al., 2025)
-
Output is Not Ability
Working with AI can leave a person's own skill in one of three places. Which one is true for your team is an empirical question. RETAIN does not assume the worst, or the best.
-
People Learn
Working alongside AI exposes people to better approaches, and their unaided skill improves.
-
Skill Holds While Work Speeds Up
Independent ability stays intact. The same people simply get the work done faster.
-
Output Improves While Ability Weakens
The work looks better, and the person's own capability quietly declines.
-
Deskilling is more than one problem. Erosion is experienced staff losing a skill they once had. Non-acquisition is newer staff never building it. Offloading is skill that is intact but unused, and recoverable. Each calls for a different response, so a study has to say which one it is measuring.
-
What RETAIN Is
RETAIN is a research study designed and run for one organization at a time. It measures whether the people doing a consequential task keep and build their own capability while using AI, and shows you where that capability is fragile.
Each study is built around a task your organization depends on, such as reviewing reconciliations, assessing contracts, or triaging cases. A qualified domain expert defines what good judgment looks like. Behavieural designs the measurement, collects the data, and reports what it does and does not show.
-
Built Around Your Task
Tests use realistic cases from your domain, written with a qualified expert, not generic quizzes.
-
Measures the Person, Not the Tool
The outcome is what people can do unaided, compared with what they can do with AI.
-
Designed to Support a Trend Claim
A baseline, a follow-up, and a comparison group, where the design allows, so results can speak to change over time.
-
How a RETAIN Study Works
Choose the task. We agree on one consequential task. A qualified domain expert defines what good judgment looks like and which mistakes matter.
Build the instrument. We create parallel sets of realistic cases at matched difficulty, including cases with errors a competent professional should catch and exceptions the tool may mishandle.
Take a baseline. Ideally before AI is introduced. Participants predict their own performance, then complete cases under realistic conditions.
Follow up. After a period of AI use, participants complete parallel cases with and without AI. Where the design allows, we compare against teams with later or lower exposure.
Report and recommend. You receive team-level findings, a plain statement of what the evidence can and cannot support, and specific workflow recommendations.
Test the fix (optional). If you change practice, review steps, or escalation rules, we re-test to see whether the change worked.
The time from baseline to follow-up depends on how often the task is performed, and usually runs to months.
-
What We Measure
Each study combines four measures, tailored to your task.
-
Unaided Performance
How well people do the task without AI, on realistic cases graded against a standard set by a domain expert.
-
Calibration
How closely confidence matches ability. Participants predict their own unaided score before testing. The gap between prediction and result shows whether people know where their limits are.
-
Transfer
Performance on exceptions and unfamiliar cases, where routine competence and genuine judgment come apart.
-
How People Use AI
Whether they delegate to the tool or check it. Usage patterns are a likely explanation for any change in capability, and they point to what to fix.
-
Start Before AI Arrives, or Start Where You Are
If you are about to roll out AI, that is the best moment to start. Staggering the rollout across teams gives the study its comparison group. If AI is already embedded, a study is still worthwhile. It will state plainly what the design can and cannot conclude.
-
Before AI Rollout
Design. Baseline before adoption, staggered rollout across teams, follow-up measurement
What it can show. Whether capability changed after adoption, compared with teams that have not yet adopted
What it cannot show. Effects beyond the task and period measured
-
Already Using AI
Design. Unaided performance compared with a lower-exposure group, and with staff who trained before versus after AI
What it can show. Current independent capability, and whether newer staff have developed the skill
What it cannot show. That AI caused a loss, because change over time needs a baseline
What Your Organization Receives
-
Team-Level Findings
Where independent capability is strong, where it is fragile, and on which task types.
-
A Plain Statement of the Evidence
What the results support, what they do not, and why.
-
Workflow Recommendations
Specific changes to practice requirements, review and escalation rules, and how AI output is presented and checked.
-
A Full Research Report
Methods, results with effect sizes and confidence intervals, and limitations.
-
An Optional Re-Test
Evidence on whether the changes you made worked.
-
Research Standards & Participant Protections
RETAIN is run as research, so the protections are built into the design.
-
Pre-registered
The design and analysis plan are registered before data collection, so results cannot be shaped after the fact.
-
Reported Either Way
Results are reported whether they show gains, losses, or no change.
-
Team-Level Reporting
Individual data is collected under coded IDs and is never shared with your organization. Findings are reported in aggregate, with a minimum group size.
-
Agreed Use, in Advance
Before data collection, we agree in writing how results will and will not be used, so a learning exercise does not become covert performance management.
-
Ethics Review
Each study is designed for review by a research ethics board.
-
Consent-Based Publication
Anonymized findings contribute to published research only with your written consent.
-
Who RETAIN Is For
RETAIN is built for organizations where individual judgment carries real consequences, and where leaders can name a capability they worry about losing.
-
Accounting, Audit, and Finance
Teams reviewing reconciliations, controls, or AI-assisted analysis.
-
Healthcare and Clinical Teams
Settings where detection and triage skills must hold when the tool is unavailable.
-
Legal, Compliance, and Risk
Functions that review contracts, policies, and AI-generated assessments.
-
Organizations About to Adopt AI
The best moment for a baseline is before rollout.
-
No. Individual data is collected under coded IDs for analysis only, and results are reported at team level. We do not produce rankings of individuals for managers.
-
Only when the design supports it. Starting before rollout, with a baseline, a follow-up, and a comparison group, can support a claim about change. Otherwise we report current independent capability and state what cannot be concluded.
-
No. Without a baseline, a study cannot show change over time. It can show what your people can do unaided today, and whether newer staff have built the skill.
-
Behavieural does not judge domain competence on its own. A qualified expert, from your organization or brought in for the study, defines the standard and the cases.
-
It depends on the task and the design. We assess feasibility with you before committing, and very small teams are reported as descriptive case studies, not statistical inference.
-
The time between baseline and follow-up depends on how often your people perform the task. It usually runs to months.
-
With a conversation. We will tell you whether a study is feasible and which design your situation allows.
-
About the Researcher
Eugene Y. Chan (PhD) designs and leads every RETAIN study.
As Full Professor and Van Norman Chair of Business at Tyndale University (Toronto) and Founder and Principal Strategist of Behavieural, Eugene is a behavioral scientist with more than 90 peer-reviewed articles, including in Financial Times Top 50 outlets, a SSHRC Insight Grant, and an Associate Editor role at the Australasian Marketing Journal. His work applies behavioral science to AI trust, adoption, and governance.
Behavieural does not judge domain competence on its own. Every study is built with a qualified expert in the field being tested.