The figures in the CDI paper are based on a study, which we at CEPS conducted ... the CDI paper contains incorrect and spurious information concerning the prospective cost of the proposed AI Act.
Artificial Intelligence (AI) is set to disrupt businesses and all our lives in the 21st century, leading governments around the world
to publish strategies for tapping the potential of AI. The EU
definitely wants to ensure its place at the top table when it comes to
harnessing the vast opportunities promised by AI, leading to the
European Commission to publish its own comprehensive AI Strategy in April 2021. However, the regulation of these technologies is becoming an increasingly contentious political battleground.
Sound assessments of the costs and benefits of AI regulations are
therefore an essential prerequisite for an informed democratic debate. A
policy paper by the Centre for Data Innovation (CDI) recently published figures on the costs of the proposed EU AI Act, a key component of the EU’s AI Strategy. This CDI paper contained figures which has caused numerous headlines such as: “Europe’s proposed A.I. law could cost its economy $36 billion“.
The figures in the CDI paper are based on a study, which we at CEPS conducted (in cooperation with ICF and Wavestone) to support the European Commission’s Impact Assessment for the AI Act in late 2020 (hereafter, ‘the Study’).
Unfortunately, despite the detailed explanation contained within the
Study, the CDI paper contains incorrect and spurious information
concerning the prospective cost of the proposed AI Act. This is
unfortunate because it leads to grossly exaggerated cost figures. As the
CDI paper partly replicates numbers from our Study and recompiles them
in a misleading way, we are taking the opportunity to address the main
concerns raised by the CDI paper, in the hope that this will help
restore some clarity on the methodology and the main results of our
analysis.
The CDI’s cost extrapolation is factually incorrect
On page 7, the CDI states: “The Commission’s impact assessment[1]concludes
that the [AI Act] will cause an additional 17% of overhead on all AI
spending. By 2025, the [AI Act] will have cost the European economy more
than €30 billion.” The ‘17%’ estimate is copied from the
compliance cost assessment estimated in our Study. The 17% was then
probably multiplied with the estimated upper bound of all AI investment
between 2021-2025 to create the ‘more than €30 billion’ estimate. The
CDI paper thus seems to use the upper bound cost estimate of our Study
(all AI systems without the business-as-usual factor) without providing a
clear explanation for doing so. This is incorrect in two important
ways:
The main compliance costs from the AI Act do not apply to all AI investments, but to high-risk AI systems.
The proposed AI Act lists several areas in Art. 6 and the annexes, to
which the main requirements apply. These only constitute a small share
of all AI investment. The Study assumes that only 10% of AI systems will
be subject to paying the compliance cost. This is consistent with what
the European Commission also stated when presenting the proposed AI Act,
i.e. that they expect the portion of high-risk AI systems to be between
5%-15% of all AI systems (Commission Impact Assessment p. 69).
Even for most high-risk AI systems our estimated 17% compliance costs will not entirely apply.
The 17% only applies to companies that do not fulfil any of the
regulatory requirements as part of their ‘business-as-usual’ (BAU)
practice. Including this BAU factor is common practice in cost
assessments. It is safe to assume that companies already fulfil parts of
the requirements (e.g., ensuring that AI systems are accurate, as
written in the AI Act proposal, Art. 15). The compliance costs for
businesses following common best practices will therefore be reduced.
Given these two important flaws, the ‘more than €30 billion’ estimate
grossly exaggerates the cost of the AI Act, and by no means reflects
the content and outcome of our economic analysis. Our extrapolated costs
of compliance, assuming 10% coverage, are highlighted in the table
below.
The CDI’s definition of ‘high-risk sectors’ is too broad
The CDI later recognises that the AI Act mostly applies to high-risk AI and writes: “about
35 percent of sectors (by value) fall into the [AI Act‘s] ‘high risk’
category: education, finance/insurance, health, IT, technical/scientific
activities, social work, and critical infrastructure. Approximately
€3.4 trillion of economic activity is thus considered ‘high risk’ by the
[AI Act].” In fact, the AI Act does not consider entire sectors as
high-risk, but only specific applications within these sectors (see AI
Act proposal Annex III). In the education sector, for example, only
those applications where an AI system is used to assess students or/and
to determine access to education is considered high risk (Annex III,
3a&b). This means that probably a lot less than the CDI’s assumed
€3.4T of EU GDP can be considered as ‘high-risk’.
The CDI’s estimated costs for SMEs are too high
The CDI writes: “Based on the EU’s own impact assessment, a small
business […] can expect total compliance costs of up to €400,000 for
one high-risk AI product requiring a quality management system.”
The original numbers which the CDI copied from CEPS’ study are actually
different – we estimated that setting up an entirely new Quality
Management System (QMS) could cost between €193,000-€330,000 with an
additional estimated €71,400 for annual maintenance.[2]
These are indeed relevant costs, especially for SMEs. It is important
to note, however, that these costs are only fully incurred if a company
does not have a QMS in the first place. Costs are reduced, including for
new products, once the QMS is set up. Moreover, QMSs are only a
requirement for providers of high-risk AI systems (AI Act proposal Art.
16).
Final Remarks
Cost assessments are difficult and based on many assumptions.
Reasonable academic arguments can be made that the costs estimated by
the Study could be higher or lower for the actual final AI Act.
Researchers could challenge the ‘10% high-risk AI’ assumption on
different grounds and may disagree with our estimates of compliance
costs. Section 4.9 of the Study lists several caveats and spells out
clearly the assumptions that we made to reach our estimates.
The CDI paper, however, picks out numbers from our Study and
recompiles them in an incorrect and/or misleading way. Given the issues
outlined above, we have no choice but to highlight this careless
reference to our Study so as to avoid any further or future confusion or
misunderstandings regarding our findings and our conclusions reached.
[1]
Throughout the policy paper CDI incorrectly quotes ‘the Commission’s
impact assessment’. The cited page numbers do not exist in the
Commission’s impact assessment. CDI is actually referring to the
externally sub-contracted study to support the impact assessment, which
was authored by CEPS and our partners ICF and Wavestone.
[2] The CDI probably simply added our individual €330,000 and €71,400 figures to arrive at their ‘€400,000’ figure.
