AI creates value when it reduces manual work or helps consultants identify candidates for open roles. Measure time saved alongside shortlist quality and placement outcomes. Check how the team uses that extra capacity for candidate conversations, follow-up and client development.

The difficulty is that AI changes several things at once. A consultant who no longer types CVs into profiles may also make more candidate calls, and results improve for reasons that have little to do with the tool itself. A simple, consistent approach separates those effects well enough to support sound decisions.

Set a baseline

Choose a task such as CV entry, vacancy capture or shortlist preparation. Record the time required, corrections, relevant submissions and placement progress before introducing the tool. Use the same measures afterwards for a meaningful comparison.

Pick a task with enough volume to show a difference. For a nursing desk that might be vacancy capture from client emails. For a locum desk it might be registering new doctors, whose CVs often run to several pages of rotations and include a registration number that must be recorded accurately. For payroll, it could be timesheets that arrive as scans or photographs.

Over two to four weeks, record:

  • The number of items handled, such as CVs, vacancy emails or timesheets.
  • The average time from receipt to a record someone can act on.
  • Corrections made after entry, and who made them.
  • The outcome further along: submissions, bookings or approved timesheets.

Keep the method light. A timed sample from two or three consultants is more useful than an elaborate study that nobody finishes.

Compare similar work

Use comparable role types and periods. A change in client demand or candidate supply can affect placement numbers independently of AI. Include those factors when interpreting the results.

Comparing a quiet holiday period with a winter peak on a nursing desk will show more placements whatever tools are in use. Where possible, compare the same consultants on similar requirement types, or introduce the tool on one desk while a comparable desk carries on as before for a few weeks. Note anything else that changed during the period, such as a new client, a new starter or revised targets. Small samples swing easily, too. A handful of difficult CVs in one week can make a tool look worse than it is, so judge it over several weeks rather than a few days.

Measure quality alongside speed

Track gaps in captured information, relevant submissions from matched candidates and client feedback. Include the time spent resolving exceptions and checking suitability. Faster preparation should create better opportunities for consultants to progress, without adding avoidable work later.

What good and poor output look like

Good output arrives in the right fields and needs little checking. A captured vacancy carries the grade, speciality, dates, location and the client's job reference. A parsed CV places work history and registration numbers where compliance expects to find them. Flagged gaps turn out to be genuine.

Poor output moves the work rather than removing it. Consultants re-check every field, duplicates slip through, or errors surface later when a client queries a submission or a timesheet line. If a parsed CV saves a few minutes at entry but compliance later spends longer correcting a misread registration number, the task has shifted rather than shrunk. Count that rework against the time saved.

Check how consultants use it

Review how consultants use the ranked candidates and the time saved by automation. Investigate weak matches or missing information, then improve the criteria, source records or workflow.

Low adoption usually has a practical cause: outputs that need too much checking, a step that sits outside the consultant's normal screen, or training that never happened. Ask the team two questions each month. What did you do with the time you saved? What did you have to fix? The first shows whether capacity reached candidate calls, follow-up and client development. The second tells you where the criteria, records or workflow need attention.

Apply the same scrutiny to ranked shortlists. If consultants regularly skip the top of the list, find out whether the vacancy criteria, the availability records or the weighting is the cause before concluding that the tool does not work.

For tools that propose changes rather than make them, such as suggested record updates from call transcripts, track how many suggestions are accepted, edited or rejected. A high rejection rate is a quality signal. A high acceptance rate followed by later corrections suggests suggestions are being approved without being read.

Review the commercial result

Compare usage costs with the time saved and the effect on relevant submissions, conversion and placement value. Keep a short monthly review covering effort, quality, adoption and outcomes, using consistent definitions. Use the findings to decide where AI can make the strongest contribution to desk performance.

Work out a cost per useful output: per vacancy captured, per CV parsed or per timesheet extracted. Compare it with the consultant or administrator time it replaces, including correction time. In RecSphere, for example, AI vacancy capture, CV parsing, call review and timesheet capture use credits, and your team reviews every output, so cost and correction effort can be tracked task by task. Candidate matching is not AI and uses no credits, so assess it separately as a shortlisting tool.

A one-page monthly review

  • Effort: time per task compared with the baseline.
  • Quality: corrections at entry and errors found later.
  • Adoption: the share of eligible items processed with the tool.
  • Outcomes: relevant submissions, conversion and placement value.
  • Cost: credits or charges used, and the cost per useful output.

Hold the definitions steady from month to month. If "relevant submission" means a candidate the client agreed to interview in one review and any CV sent in the next, the trend is meaningless.

Common measurement mistakes

  • Measuring speed at entry and ignoring correction work later in the process.
  • Introducing a new tool alongside a team restructure, so neither effect can be isolated.
  • Relying on impressions instead of a timed sample.
  • Counting time saved without checking where that time went.
  • Extending AI to new tasks before the first one is working well.

Decide what happens next

The monthly review should end in a decision rather than a report. Three outcomes cover most cases:

  1. If effort fell and quality held, extend the tool to the next task with the same measures in place.
  2. If effort fell but corrections rose, fix the inputs or the review step before extending anything.
  3. If nothing changed, check adoption first. A tool that is rarely used cannot show results.