Use AI where a person can still check the result

AI earns its place when it helps somebody examine evidence faster or notice something they might otherwise miss. Triage, summarisation, classification and pattern finding can all be useful. None of them should hide an important decision or make accountability disappear.

This is the management test:

AI may accelerate the examination of evidence. It should not make responsibility disappear.

In field operations, the work rarely begins with a perfect dataset. It begins with photographs, measurements, site notes, locations, asset records and incomplete observations gathered under real constraints. An AI system can help organise that material. It cannot make weak evidence strong, settle who has authority or become responsible for the operational consequence.

The safest place to begin is one bounded evidence task rather than a broad promise to “use AI”: a defined set of inputs, a limited output, a named reviewer and a result that can be checked against the original record.

Keep the suggestion separate from the decision

The pattern I trust is simple. The model makes a suggestion and records enough context for somebody to review it. A named person accepts, changes or rejects it. The system keeps the original evidence alongside the final action.

That distinction matters in field work. A model might flag an unclear photograph, group similar site notes or suggest which jobs need attention first. It should not make a permit, safety or commercial decision appear to have been made by the evidence itself.

Use different words for different things:

• Evidence is what was observed or recorded.
• Inference is what the model suggests the evidence may mean.
• Decision is the action an authorised person or controlled rule permits.
• Outcome is what happened after that action.

When these are collapsed into one polished answer, a reviewer can no longer see where fact ended and judgement began.

Define a bounded evidence task

“Summarise field data” leaves the task undefined. It says nothing about which records may be used, what the output is for or who must verify it.

A bounded task answers six questions:

1. Evidence boundary: Which records, versions, dates and locations may the model examine?
2. Output boundary: Is it classifying, extracting, ranking, summarising or proposing an action?
3. Traceability: Can every important statement be followed back to the source material?
4. Abstention: Can the system return “insufficient evidence” instead of inventing a complete answer?
5. Verification: Who compares the output with the original evidence, and what must they check?
6. Accountability: Who remains responsible for the operational decision and its consequence?

For example:

Review the photographs and notes attached to one completed site visit. Identify whether each required evidence type appears to be present, point to the relevant source item and flag anything unclear. Do not decide whether the job is compliant. Route incomplete or ambiguous cases to the named supervisor.

The task is narrow enough to test. Its evidence and output are visible. Most importantly, it does not borrow authority from the person who must decide.

The NIST AI Risk Management Framework describes AI risk management as a continuing cycle of governing, mapping, measuring and managing risk. A bounded task makes that practical: the organisation can identify the context, test the behaviour and assign control before exposing more work to the system.

Preserve a provenance bundle

An AI output without its history is difficult to challenge and harder to improve.

Keep a compact provenance bundle with every consequential suggestion:

• the identifiers and versions of the evidence examined;
• the model, prompt or configuration version used;
• the output produced;
• any evidence-quality or uncertainty flags;
• the reviewer and time of review;
• whether the suggestion was accepted, changed or rejected;
• the operational action that followed.

This does not require exposing internal model mechanics to every user. It requires preserving enough of the chain to reconstruct what the system saw, what it proposed and who authorised the result.

The W3C provenance model provides a useful underlying structure: entities, activities and agents. In operational terms, that means retaining the evidence, the processing step and the people or systems responsible for it.

Where a source can change, store the exact version, snapshot or integrity reference used for the review. A link to a live record is not enough if the record may later be corrected. The original evidence and the corrected record may both matter.

Make insufficient evidence a valid result

A system designed to answer every case will eventually sound confident about a case it cannot support.

Give it controlled result states such as:

These descriptions are more useful than displaying a raw confidence percentage. A score of 82% can look precise without telling a manager whether the photograph was blurred, the source record was stale or the model has been validated for this kind of case.

There are at least three different uncertainties to control:

• whether the evidence itself is complete and reliable;
• how well the model performs on comparable cases;
• whether the proposed operational action is safe and authorised.

Keep those three uncertainties visible instead of compressing them into one number.

Design human verification around the evidence

“Human in the loop” is not a control unless the person has the evidence, authority and time to intervene.

The reviewer should see:

• the original photograph, note, measurement or record;
• the specific question the model was asked;
• the model’s suggestion and supporting references;
• missing, contradictory or low-quality evidence;
• the actions the reviewer is authorised to take;
• the consequence of accepting the result.

Show the evidence before the polished summary. If the AI wording is easier to read than the source, reviewers will naturally confirm the wording rather than test it.

Record corrections as useful operational data. A disagreement may expose poor evidence, an unclear instruction, a model weakness or a genuine edge case in the work. “The human changed it” is not a sufficient explanation.

Human review also needs capacity. If every case is sent to one supervisor, the system has moved the bottleneck rather than removed it. Decide which cases require full review, which need sampling and which must stop automatically.

Set autonomy from consequence and reversibility

The right level of automation depends less on how impressive the model is than on what happens when it is wrong.

Autonomy should decrease as consequence and irreversibility increase. The fact that a model can produce an answer does not mean the workflow should act on it.

NASA’s first AI-planned drives for the Perseverance rover provide a disciplined example. According to NASA’s Jet Propulsion Laboratory, the system generated route waypoints from specified terrain data. Engineers then processed the commands through a digital twin and verified more than 500,000 telemetry variables before transmission. The AI proposed a bounded plan; simulation, verification and human authority remained around execution.

Field operations do not need a Mars rover control room. They do need the same separation of proposal, verification, authorisation and action.

Use a management scorecard

A catalogue of AI metrics can wait. Begin with the few measures that show whether the bounded task remains controlled.

A rising acceptance rate is not automatically good. It may mean the task is improving, or that reviewers have stopped challenging the output. Review a sample of accepted cases against the original evidence.

Start with one evidence task

Choose a task where the input, reviewer and acceptable result are known. Test incomplete, ambiguous and contradictory examples as well as the clean ones. Record when the model is uncertain and make the route to a person obvious.

A sensible first release has:

1. one clearly named evidence task;
2. a fixed source set and version history;
3. an explicit set of allowed result states;
4. a named reviewer with authority to correct it;
5. a fail-closed route for missing evidence or authority;
6. a retained record of suggestion, review and action;
7. a small test set containing awkward as well as normal cases;
8. a stop condition if error, backlog or evidence quality exceeds the agreed limit.

Run it beside the current process before allowing it to change operational state. Compare the suggestion with the real evidence and the eventual decision. Inspect disagreements rather than averaging them away.

Expand only when the organisation can answer four questions without asking the development team:

• What evidence did the system use?
• What exactly did it suggest?
• Who checked and authorised the result?
• What happens when the evidence is not good enough?

That is less dramatic than trying to automate everything. It is also much easier to trust when the work matters.

Field Systems & Geospatial starts with the job, evidence and decision rather than assuming AI is the answer.