Expert demonstrations
Expert demonstrations capture how a capable person or system carries out a task. They may include instructions, observations, actions, and completed outputs. In machine learning, demonstrations can provide examples of desired behavior, such as responses to requests or sequences of actions in software or robotics.
How it works
Define the task and what competent performance means, then capture enough context to interpret the actions. A useful demonstration can include the starting state, available information, decision points, corrections, and result. Selection and quality checks matter because a recording can omit important context or include mistakes.
Methods differ across domains. Instruction-following research has used human-written demonstrations, while robot-learning research has used teleoperated action sequences. These are different representations with different requirements.
Why it matters for licensing
A recipient may need demonstrations that match a particular task, interface, or operating condition. Ordinary logs are not automatically expert demonstrations. Licensing also needs to address source content, participant permissions, and any sensitive information visible during the task.
Example
Fictional example: An experienced dispatcher handles a scheduling conflict in a test environment. The recording includes the constraints available at each step and the final schedule, allowing reviewers to distinguish a justified decision from an unexplained sequence of clicks.
Limitations and misconceptions
Expert behavior in one setting may not generalize to another. Demonstrations can encode local shortcuts or flawed practices, and training only on successful examples may leave a model unprepared for recovery. The term does not certify quality or imply Rancher offers such data.
Questions to ask
- What evidence establishes the demonstrator’s competence for this task?
- Are starting context, corrections, and outcomes captured?
- Which operating conditions and failure cases are missing?