CHAPTER 01 / HUMAN CONTEXT
What is the AI system meant to help someone do?
People, purposes, and outcomes
An AI system starts with a human purpose: a task to support, a decision to inform, or an outcome to improve. A convincing output is useful only when it helps with that purpose.
Start with the task
A useful description names the user, the task and the conditions of success. “Help a maintenance planner find relevant equipment instructions” is easier to evaluate than “add AI.” The task determines which information is needed, which errors matter and when a person should review the result.
Judge the outcome
Speed and fluency are incomplete measures. Ask whether the output is correct enough for the task, whether its evidence can be checked and whether the user can recover from an error. Evaluation should include the setting in which people actually use the system.
Keep human responsibility visible
People choose the purpose, grant permissions, review important decisions and decide when to stop using a system. A model output does not transfer responsibility away from the organization deploying it. These questions connect the technical system to governance.
Keep this connection in mind.
Define success before choosing a model. The same technology can be helpful in one setting and unsuitable in another.
- People uses AI applications
- AI applications can support Human tasks
- People in organizational roles contributes to AI decisions and oversight
Sources for this introduction
- NIST · AI Risk Management Framework ↗
Human context, evaluation, accountability and risk management.
Draft teaching text, prepared with AI assistance. Larry’s editorial review is pending. Read the editorial approach.
EXPLORE RELATED AREAS