Use a practical demo to assess the manual work AI saves and the quality of its outputs. Bring a sample CV, vacancy request, call transcript or timesheet. Follow the results into the relevant records and check whether they help consultants progress the next step.

A polished demo on the vendor's own data shows that a feature exists. It does not show how the tool copes with a locum doctor's CV covering years of rotations, a forwarded client email requesting four shifts across two sites, or a scanned timesheet with handwritten corrections. Those are the inputs your team handles every week, so they are the ones worth testing.

Prepare your test material

Decide what you will bring before the demo is booked, and agree with the vendor how sample files will be handled. Remove personal details you do not need, or use realistic made-up examples.

Three or four CVs in different formats: a short care worker CV in Word, a long locum CV in PDF and one with a registration number in the header.
Two client vacancy emails, including one that covers several shifts or quotes the client's job reference.
A call transcript in which a candidate mentions changed availability or a new preferred location. Only use a recorded call where the candidate agreed to recording and your own data policies allow it to be shared.
A timesheet as a PDF and as a photograph, ideally one with a correction on it.

Write down what a correct result looks like for each item beforehand. You can then score the output against your own answer rather than reacting to how impressive it looks on screen.

Look at capture and parsing

Assess whether the tool extracts the information you need and puts it in the correct fields. Include time spent correcting the result when comparing it with manual entry.

Check specific fields rather than forming an overall impression. On a vacancy, look for the grade, speciality, dates, location and the client's job reference. On a CV, look at work history, qualifications and registration numbers. On a timesheet, check dates, start and finish times, breaks and any shift lines that were crossed out.

Then watch what happens with a difficult input. Does the tool flag a possible duplicate vacancy or a missing detail, or create a new record without comment? A flag that prompts a quick check is more useful than a confident record that is quietly wrong. If time allows, run the same input twice. Consistent results matter as much as a single good one, because your team will process the same kinds of document week after week.

Ask what matching considers

Review the factors used, their weighting and the score breakdown. Check whether consultants can understand why a candidate appears before deciding to contact or submit them.

Start by asking whether matching uses AI at all. Some platforms rank candidates with a scoring model built on fields the agency controls, which is easier to explain and adjust. RecSphere's matching works this way: it is not AI and uses no credits. Whichever approach a vendor takes, a consultant should be able to see the reasoning.

Which candidates are excluded before scoring, for example because they do not hold the required grade or speciality?
Which factors are scored, such as rate, shift availability, location, preferences, margin, qualifications, client pre-approval and experience?
Who can change the weights, and does a change apply across every desk?
Can consultants still run a manual search alongside the ranked list?

Then run a live vacancy and ask an experienced consultant whether the top of the list contains the people they would call first. If it does not, the score breakdown should show why.

Check the review process

Understand which records can change automatically and which require approval. The team should know how to reject an output and correct the source information.

In healthcare staffing the consequences are practical. A misread registration number, or an availability change taken from a misheard call, can put the wrong person forward for a shift. Ask the vendor to show where each output goes before it is saved: into a review step, a pending state or straight onto the record.

For call review in particular, confirm that a recruiter approves each proposed update and that an administrator decides whether the feature is switched on. Ask, too, how a rejected output is handled, so the same mistake is not repeated on the next record. Finally, look at the trail a change leaves behind: who approved it, when, and what the record showed before.

Confirm what is available today

Separate live functionality from planned development. Review plan access, supported inputs, integrations and usage charges using the proposed commercial setup.

Ask to see each function working in the plan and setup you are being quoted for. Treat slides, roadmaps and beta labels as separate from what your team can use on day one. If a capability is central to your decision, ask for written confirmation of which plan includes it. Check how AI usage is charged, whether any credits are included and how more are arranged when they run low.

Measure a complete task

Compare preparation, checking and correction time. Ask consultants whether the output helps them progress candidates and client work, then review its effect on relevant submissions, placement conversion and the time available for account development.

Time the whole task, from receiving the input to a record a consultant can act on. For a locum CV, that includes opening the output, correcting fields and confirming the registration number, not just the seconds the file took to process. A simple scoring sheet keeps vendors comparable:

Fields correct first time.
Minutes spent correcting.
Flags raised, and how many were useful.
Whether the output reached the right record.
Estimated cost per item at your expected volume.

Score every vendor with the same sheet and the same people, ideally within the same week, so differences reflect the tools rather than the testers.

Common demo mistakes

Accepting the vendor's sample files in place of your own.
Judging the output by how quickly it appears rather than how accurate it is.
Leaving out the people who will check the output every day, such as compliance and payroll staff.
Assuming a feature shown in one plan is included in the plan you are being quoted.