1. Should every AI system in the organization be subject to risk management?
AI products or services that companies (AI developers and AI users) build or adopt vary widely in type. They also differ from existing IT systems, so organizations often struggle from the current-state survey onward when responding to the Framework Act on Artificial Intelligence (AI). In short, what the Framework Act requires of companies can be summarized as follows.
- Ensure the safety of AI
- Identify and manage in advance the risks arising from AI
- Ensure accountability and transparency for AI decisions
The Framework Act currently prescribes “what must be done,” and it defines high-impact AI as “an AI system that may have a significant effect on, or pose a risk to, human life, physical safety, and fundamental rights.” It does not, however, present specific criteria for “which AI is in scope.” The resulting difficulties in practice can be summarized as follows.
- Is managing every AI system in the same way inefficient?
- How large is the regulatory risk from omitting a system that should be managed?
- Does the absence of internal criteria create problems when responding to an audit?
Considering the intent of the Framework Act together with global standards and reference models, companies face the following questions.
- Which AI should be managed?
- How far should the regulatory scope extend?
- By what criteria should risk management targets be selected?
2. What are the proposed criteria for selecting AI subject to risk management?
The proposed criteria for selecting AI subject to risk management draw on the core content and intent of the AI Framework Act and on global standards and regulatory frameworks, including the NIST AI Risk Management Framework, the OECD AI Principles, and the European Commission AI Act.
- Judge by impact, not by technology
- Apply quantitative and qualitative perspectives together
- Align with the Framework Act and global practice
- Ensure explainability for audit and supervisory response
The proposed criteria for selecting AI subject to risk management are designed as concise questions and content that can be understood and judged intuitively, because they will be used by diverse organizations, including IT, non-IT, business, and non-business functions. It is recommended that companies customize these proposed criteria to their environment and apply an assessment policy based on them.
| # | Assessment criterion | Judgment question | Applicable cases (Yes/No) |
| 1 | Rights, safety, and impact | Does the AI outcome directly affect an individual’s rights (financial transactions, employment, education, and similar areas) or safety (body or property)? | [Yes] Credit scoring, medical diagnosis, hiring assessment [No] Document summarization, search assistance |
| 2 | Degree of involvement in decision-making | Is the AI output used directly for an actual decision, such as approval or rejection, or for an equivalent judgment, rather than as a simple reference? | [Yes] Automated loan approval, hiring filtering [No] Reference report |
| 3 | Sensitivity of the domain of use | Is the AI used in a domain that materially affects an individual’s rights, such as finance, healthcare, employment, or access control? | [Yes] Financial underwriting, biometrics [No] Marketing copy generation |
| 4 | Severity of harm | If an error occurs, can it cause significant harm to an individual or the organization, such as financial, legal, or service-disruption harm, and is that impact likely to spread repeatedly to many people? | [Yes] Large-scale customer service AI [No] Internal test model |
| 5 | Processing of personal and sensitive information | Does the AI use personal information or sensitive information for input, training, processing, or output? | [Yes] Customer data analysis, medical data AI [No] Experimental model using public data |
| 6 | Security and potential for misuse | Is the model structured so that an external attacker or an internal user can misuse it, cause data leakage, or induce malfunction? | [Yes] LLM(GenAI) API, chatbot AI service [No] Closed analytical model |
| 7 | Controllability | Is the structure such that review, suspension, or correction by an internal user is difficult, and tracing and explaining the result is difficult? | [Yes] Fully automated, black-box model [No] Human-in-loop system |
3. How should the proposed selection criteria be used?
These proposed selection criteria can be used when an operator assesses the current state. During ongoing operations, they can also be used at the planning stage of a newly developed AI product or service to determine whether it is subject to risk management.
These criteria serve only to distinguish whether an AI product or service should be treated as subject to risk management, so they are an important first step. For AI selected as subject to risk management, a specific risk assessment, application of protective measures, verification, and monitoring are then carried out during analysis and design.