The European commission has delivered a new risk-assessment framework for trustworthy Artificial Intelligence (AI) that can be accessed online. Importantly, it puts a practical spin on the Commissions earlier previously published guidelines. This brief covers the main aspects of the European Commission’s framework, outlining its key features whilst also bringing in some analysis from recent developments.
In developing the new risk framework, the European Commission have emphasised the importance of observing an AI’s overall impact on core human rights. The main elements of the framework do this, covering hotly topical rights, relating to non-discrimination, protection of personal data and freedom of expression.
The main areas covered by the framework include:
(A) Human Agency and oversight
(B) Technical Robustness and Safety
(C) Privacy and Data Governance
(F) Societal and Environmental well being
EXAMINING THE FRAMEWORK
(B) Human Agency and oversight
This covers whether the system might create over-reliance in users, or illegitimately manipulate users’ behaviour. Whilst AI won’t be capable of making ‘human’ errors, it will however still be fallible to both manufacturing and design errors, as well as ‘machine errors’, fuelled by the algorithmic system itself. This presents clear problems in areas where AI, might be used to substitute human agency in its entirety. An example of this being, stage 5-6 of self-driving cars, where the AI algorithm is said to have complete control without driver interference (for more information see SAEs 5 Stages of Automation and our previous post on self-driving car fundamentals).
(A) Technical Robustness and Safety
This covers the subject of security, including resilience to cyber-attacks, general safety as well as the potentially devastating effects of inaccuracy and unreliability.
(C) Privacy and Data Governance
This aspect questions the impact of the AI system on the rights to privacy, the right to physical, mental or moral integrity and the right to data protection, and what mechanisms are in place to minimise potential harm and deal with concerns.
This covers the traceability of decisions and how, why and where data has been used. An issue with this aspect arises when looking at self-driving car “Black-boxes”, which generate specialist outputs or decisions that cannot readily be explained to users.
(D) Diversity, Non-discrimination and fairness
This limb is particularly important for the AI designers of today, who have been spearheading recent developments. Recent events have lighted the importance of avoiding discrimination of all forms more than ever and even more so in the context of algorithmic bias as highlighted by a recent Ted talk ‘The Future is Biased’ by the president of the Law Society, Christina Blacklwas. AI systems should avoid unfair bias and its important that designers not only have a diverse end user in mind, but as an employment group, are also adequately represented by a diverse class of people. Ultimately, the end system should be fair regardless of age, gender, race or other characteristics.
(E) Societal and Environmental well being
Naturally, as with the development of any new complex system designed to help humanity, any potential and actual adverse impacts to society and the environment should be limited, as far as possible. More-so, as we are so inextricably linked to both. Whilst its true, that initially adoption of AI in many instances has the potential to impact large areas of employment, leading to job-losses as a result of large-scale automation. This is a foreseeable result that should be accounted for in governmental contingency plans with ready measures to counter any negative effects. Ultimately and most importantly, the overall benefits to society, should outweigh any negative impact predicted to arise.
Last, but most importantly, its going to be crucial that any AI system developed going forwards has a system of accountability for when unjust or unfair outcomes do occur. This should also ensure an adequate system of redress. In the UK for example, this can thankfully be seen with the ‘Automated and Electric Vehicles Act 2018’ which provides the system of redress for self-driving cars under a single insurance scheme. For more details, see Driverless cars Insurance laws receive Royal assent
Now that we have a checklist and framework put in place for assessing the suitability of present and future AI systems, what remains to be seen is how developers and employers of AI systems will choose to implement its philosophy going forwards. Nonetheless, it’s certain that the principles are a great first step that towards helping end users benefit from AI equally without being exposed to unnecessary risk.
Stay tuned for more great content