CHAPTER 16 / 24

Risk management and implementation

Explore this chapter in the graph ↗

16.1

Visual hallucination and apparent authority

AI visual errors include distorted quantities, missing connections, reversed labels, misleading proportions, and confused sources, as well as incorrectly depicted objects. Professional appearance may make errors harder to challenge. ChartQA and MathVista define testable chart-understanding and visual-mathematical tasks.[24][26] Their scope remains partial, so benchmark performance cannot guarantee reliability across real graphics.

Research on fine spatial information also reports difficulties for vision-language models on simple geometric relationships.[25] Instruction should include basic but consequential checks: counting nodes, distinguishing intersection from connection, confirming arrow direction, and matching numbers. These checks may appear less sophisticated than generated summaries, yet a single incorrect edge can change an explanation. Confident prose cannot repair it.

Review should follow consequences. Draft teaching graphics can support experimentation but need checks before distribution. Internal exploratory diagrams can retain hypotheses but need stronger evidence before decisions. Executable workflows additionally need permissions and recovery. Severity, detectability, and reversibility help determine where human review, rules, or independent calculation are appropriate.

16.2

Data copyright and provenance responsibility

Visual materials may contain guest records, internal policies, unpublished research, brand assets, or third-party images. Before AI use, determine whether the selected service may receive the material, whether outputs may be shared, and whether de-identification is needed. Public or synthetic classroom data can avoid unnecessary personal information. Specific permissions and legal questions require applicable institutional rules; general guidance here is not a case-specific legal conclusion.

An AI-generated label does not remove source responsibility. Research numbers still need citations, adapted structures need attribution, and synthetic images must not impersonate field records. C2PA provides mechanisms concerning provenance and integrity, but a valid credential does not establish that depicted claims are true.[38] Trustworthy origin, unchanged files, and correct interpretation remain distinct.

Student projects can maintain a concise source register with purpose, link, processing, and sharing scope. Important sources appear in the report and full records remain in the portfolio. Small tasks can develop professional habits without making every assignment a legal audit. Knowing when to consult a material owner is more useful than relying on a general educational-use disclaimer afterward.

Three sources of risk in visual AI
Figure 17. Three sources of risk in visual AI

Conceptual design proposed in this report

Figures retain bilingual labels; the surrounding text explains them. Open for a larger view.

16.3

Dependence accessibility and organizational burden

AI may lower drawing barriers while creating new dependencies: learners cannot reconstruct basic relationships without it, instructors cannot maintain complex generated files, or institutions become tied to one format. Governance should preserve autonomy as well as prevent errors. Core structures need exportable text, tables, or open graphics, important activities need static or paper alternatives, and independent tasks should check central knowledge.

UNESCO guidance emphasizes human agency, inclusion, and educational purposes.[31] In practice, tools should serve objectives, users should know which decisions the system proposed, and people unable to use an interface should retain a viable route. Cost, connectivity, accounts, and devices are instructional conditions. Failure to log in should not be interpreted as low effort or ability.

Maintenance, training, and support are additional burdens. A convincing demonstration may still require weekly template repairs, screenshot updates, and compatibility work. Responsibility and time should be assigned. Without maintenance resources, a simpler, manually revisable system may be preferable. Sustainability depends on appropriate complexity.

16.4

A twelve week pilot and exit conditions

A twelve-week pilot can have three phases. The first four weeks define objectives, select a limited-consequence task, organize sources, and establish a baseline. The next four create minimal materials, test them, and record errors. The final four compare quality and time, revise specifications, and decide whether to continue, change, or stop. The pilot should permit findings that some tasks do not justify automation.

Exit conditions may include persistent important errors, excessive maintenance, weaker independent student performance, unresolved data permissions, or users' inability to understand outputs. Stopping one implementation does not invalidate visual thinking. Preserved structures and instructional experience can move to simpler tools, keeping educational aims independent of a platform.

Expansion should copy validated task patterns before adding domains and complexity. A stable policy case may extend to promotional rules and then more complex services. One successful demonstration does not establish that every course can be automated. Each extension rechecks evidence, authority, and assessment, supporting a system instructors can understand and maintain.

Twelve week pilot with continuation or exit
Figure 18. Twelve week pilot with continuation or exit

Conceptual design proposed in this report

Figures retain bilingual labels; the surrounding text explains them. Open for a larger view.
Table 16. Pilot risks and exit conditions
RiskSignalResponse
Factual errorRepeated misuse of sourcesSuspend and repair affected output
Learning substitutionGood products weaker independenceRestore attempts and transfer
Maintenance burdenRework exceeds saved timeReduce scope or replace
Unclear permissionUnauthorized data or actionConstrain and confirm ownership
Access barriersSome learners cannot participateProvide alternative access
IrreproducibilityImportant results cannot be rebuiltPreserve data semantics and versions
Report synthesis and design recommendations