Continuing with our analysis of the European Artificial Intelligence Regulation, today we will examine High-Risk systems. These systems are permitted but must comply with a series of requirements, as we will also see.
Annex III lists them as follows, and they are classified as high-risk when their output is relevant to a decision that could pose a risk to health, safety, or fundamental rights.
- a. Biometric identification systems (those that identify people without their active participation, remembering the existing prohibition for security forces mentioned earlier).
- b. Critical infrastructure management (such as traffic, electricity, or water management).
- c. Education and vocational training (such as managing access to education or planning academic development).
- d. Personnel selection and labor relations management.
- e. Managing access to essential public and private services (such as social benefits, emergency services, credit, or insurance).
- f. Activities of security forces (such as evaluating evidence or suspects).
- g. Migration, asylum, and border control (such as polygraphs or evaluating applications).
- h. Administration of justice and democratic processes.
The Commission has the authority to add or remove systems from these categories.
What are the requirements for developing or using these systems?
- Risk management system: They must have a risk management system for the AI system that specifically considers risks to health, safety, and fundamental rights related to its purpose.
- Governance and data management: They must ensure governance and management of training and testing data, ensuring best practices in design, collection, and preparation, ensuring its relevance and correctness, and avoiding biases that negatively impact individuals.
- Updated technical documentation: This must demonstrate that the required criteria are being met.
- Automatic activity logs: They must automatically log system activities.
- User information: Users must be informed about the system’s capabilities, equipment requirements, scope of application, accuracy level, usage conditions that may involve risks, human supervision mechanisms, etc.
- Human supervision: The system must allow human oversight during its use to minimize risks to health, safety, and fundamental rights, particularly residual risks after mitigation measures have been applied. Users must be able to monitor the systems and interpret their outputs. For remote biometric identification, the output must require verification by a human, possibly two.
Accuracy, robustness, and cybersecurity: The systems must provide an adequate level of accuracy, robustness, and cybersecurity. The cybersecurity measures must be appropriate and proportional to the system’s circumstances.
