REPORT on a comprehensive European industrial policy on artificial intelligence and robotics

30.1.2019 - (2018/2088(INI))

Committee on Industry, Research and Energy
Rapporteur: Ashley Fox
Rapporteurs for the opinion (*):Dita Charanzová, Committee on the Internal Market and Consumer Protection
Mady Delvaux, Committee on Legal Affairs
Michał Boni, Committee on Civil Liberties, Justice and Home Affairs
(*) Associated committees – Rule 54 of the Rules of Procedure

Procedure : 2018/2088(INI)
Document stages in plenary
Document selected :  
A8-0019/2019

MOTION FOR A EUROPEAN PARLIAMENT RESOLUTION

on a comprehensive European industrial policy on artificial intelligence and robotics

((2018/2088(INI))

The European Parliament,

–  having regard to its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics[1],

–  having regard to its resolution of 1 June 2017 on digitising European industry[2],

–  having regard to its resolution of 12 September 2018 on autonomous weapon systems[3],

–  having regard to its resolution of 11 September 2018 on language equality in the digital age[4],

–  having regard to the Commission proposal of 6 June 2018 establishing the Digital Europe programme for the period 2021-2027 (COM(2018)0434),

–  having regard to Council Regulation (EU) 2018/1488 of 28 September 2018 establishing the European High Performance Computing Joint Undertaking[5],

–  having regard to Rule 52 of its Rules of Procedure,

–  having regard to the report of the Committee on Industry, Research and Energy and the opinions of the Committee on the Internal Market and Consumer Protection, the Committee on Legal Affairs, the Committee on Civil Liberties, Justice and Home Affairs and the Committee on the Environment, Public Health and Food Safety (A8-0019/2019),

A.  whereas transparent, ethics-embedded artificial intelligence (AI) and robotics have the potential to enrich our lives and further our capabilities, for both individuals and the common good;

B.  whereas developments in AI are unfolding at a fast pace, and whereas AI has already played a part in our daily lives for a number of years; whereas AI and robotics are boosting innovation, leading to new business models and playing a key role in transforming our societies and digitalising our economies in many sectors, such as industry, healthcare, construction and transport;

C.  whereas the increasing integration of robotics in human systems requires strong policy guidance on how to maximise the benefits and minimise the risks for society and ensure a safe, equitable development of artificial intelligence;

D.  whereas artificial intelligence is one of the strategic technologies for the 21st century both globally and in Europe, bringing positive change for the European economy, enabling innovation, productivity, competitiveness and wellbeing;

E.  whereas around a quarter of all industrial robots and half of all professional service robots in the world are produced by European companies, and whereas the EU therefore already has important assets on which it should base its European industrial policy;

F.  whereas AI and robotics have the potential to reshape multiple industries and lead to greater efficiencies in production as well as making European industry and SMEs more competitive globally; whereas the availability of large-scale datasets and testing and experimentation facilities are of major importance for the development of artificial intelligence;

G.  whereas a common approach will facilitate the development of AI technologies for the benefit of society, while also addressing the challenges presented by these technologies in order to foster innovation, enhance the quality of AI-enabled products and services, improve consumer experience and trust in AI technologies and robotics, and avoid fragmentation of the internal market;

H.  whereas computing performance must be maintained at a leading level in the Union, which should provide opportunities for the EU supply industry and increase its effectiveness in turning technological developments into demand-oriented, application-driven products and services, leading to their uptake in large-scale and emerging applications underpinned by artificial intelligence;

I.  whereas a coordinated approach at European level is urgently needed for the EU to be able to compete with the massive investments made by third countries, especially the US and China;

J.  whereas on 25 April 2018[6] the Commission committed to proposing a European approach to artificial intelligence by developing draft AI guidelines in cooperation with stakeholders within the AI alliance, a group of artificial intelligence experts, in order to boost AI-powered applications and businesses in Europe;

K.  whereas existing rules and processes ought to be reviewed, and if necessary modified, to account for artificial intelligence and robotics;

L.  whereas the European framework for AI must be developed with full respect for the rights enshrined in the Charter of Fundamental Rights, and in particular with respect to the principles of data protection, privacy and security;

M.  whereas developments in artificial intelligence can and should be designed in such a way that they preserve the dignity, autonomy and self-determination of the individual;

N.  whereas in its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics, Parliament called on the Commission to propose a coherent legal framework regarding the development of robotics, including autonomous systems and smart autonomous robots;

O.  whereas the development of AI and robotics needs to include society as a whole; whereas, however, in 2017, rural areas remained largely excluded from the benefits of AI, as 8 % of homes were not covered by any fixed network, and 53 % were not covered by any ‘next-generation access’ technology (VDSL, Cable Docsis 3.0 or FTTP);

P.  whereas development of AI-enabled services and products requires connectivity, free flow of data and accessibility of data within the EU; whereas use of advanced data mining techniques in products and services may help to increase the quality of decision‐making, and hence consumer choice, and improve business performance;

Q.  whereas technological developments in smart products and services can benefit the knowledge economy, which is based on the quantity, quality and accessibility of information available, and can thus lead to better adaptation to consumer needs;

R.  whereas cybersecurity is vital for ensuring that data is not maliciously corrupted or misused to make AI function in a way that is detrimental to citizens or companies, which would undermine industry and consumer trust in AI; whereas developments in AI increase reliance on these systems for actions and decisions, which in turn necessitates high standards of cyber resilience in the EU to protect against cybersecurity breaches and failures;

S.  whereas the trend towards automation requires that those involved in the development and commercialisation of artificial intelligence applications build in security and ethics at the outset, thereby recognising that they must be prepared to accept legal liability for the quality of the technology they produce;

T.  whereas the building of a trusted ecosystem for the development of AI technology should be based on data policy architecture; whereas this implies the creation of smooth and simplified data collection and management programmes for educational research purposes to enable the development of AI in many domains: medical, financial, biological, energy, industrial, chemical or public sector; whereas a data-driven AI ecosystem could comprise pan-European initiatives launched on open standards and based on mutual recognition of certificates and transparent rules of interoperability;

U.  whereas the use of AI alone does not ensure truth or fairness, as bias can emerge in how the data is collected and how the algorithm is written and can stem from bias present in society; whereas data quality, together with algorithmic design and constant re-evaluation processes, should prevent bias;

V.  whereas AI and robotics should be developed and deployed in a human-centred approach with the aim of supporting humans at work and at home; whereas AI can also be used to avoid people having to do dangerous jobs;

W.  whereas further development and increased use of automated and algorithmic decision-making undoubtedly has an impact on the choices that an individual (such as a businessperson or an internet user) and an administrative, judicial or other public authority make in reaching a final decision of a consumer, business or authoritative nature; whereas safeguards and the possibility of human control and verification need to be built in to the process of automated and algorithmic decision-making;

X.  whereas machine learning also raises challenges in terms of ensuring non-discrimination, due process, transparency and understandability in decision-making processes;

Y.  whereas AI constitutes a significant tool for addressing global societal challenges, and whereas the Member States should, therefore, through their public policy, promote investment, make funds available for R&D and address barriers in the development and adoption of AI;

Z.  whereas commercial artificial intelligence platforms have moved from testing to real applications in health, the environment, energy and transport; whereas machine-learning techniques are at the heart of all main web platforms and big data applications;

AA.  whereas Europe’s researchers and companies are involved in a wide variety of blockchain topics, ranging from the supply chain, government services, finance, internet of things (IoT), healthcare, media, smart cities, energy and transport; whereas Europe is a strong player in important blockchain-related fields such as AI; whereas blockchain can play an important role in enhancing European innovation;

AB.  whereas cybersecurity technologies such as digital identities, cryptography or intrusion detection, and their application in areas such as finance, industry 4.0, energy, transportation, healthcare and e-government, are essential to safeguard security and trust in online activity and transactions by citizens, public administrations and companies alike;

AC.  whereas text and data mining serves as a foundation for AI and machine learning applications, and are vital for SMEs and start-ups because they allow them to access large quantities of data to train AI algorithms;

AD.  whereas AI could prove to be very energy intensive; whereas, in light of this, it is important that the use of AI advances in keeping with the EU’s existing energy efficiency and circular economy targets;

AE.  whereas AI should fully support all European languages to provide all Europeans with equal opportunities to benefit from modern AI developments within the multilingual European information society;

AF.  whereas, in industry and services associated with high technology, AI is key to turning Europe into a ‘start-up continent’ by exploiting the latest technologies to generate growth in Europe, in particular in the areas of health technology, healthcare services and programmes, drug discovery, robotic and robot-assisted surgery, treatment of chronic diseases, and medical imaging and records, as well as securing a sustainable environment and safe food production; whereas Europe is currently lagging behind North America and Asia in terms of research and patents in the field of artificial intelligence;

AG.  whereas the development of AI technologies may help to improve the lives of people with chronic illnesses and disabilities and address social challenges such as our ageing population by making health technology more precise and effective in providing healthcare;

AH.  whereas there is a broad catalogue of possible applications of AI and robotics in medical care, such as managing medical records and data, performing repetitive jobs (analysing tests, X-rays, CT scans, data entry), treatment design, digital consultation (such as medical consultation based on personal medical history and common medical knowledge), virtual nurses, medication management, drug creation, precision medicine (as genetics and genomics look for mutations and links to disease from the information in DNA), health monitoring and healthcare system analysis, among other applications;

AI.  whereas accessibility does not mean the same services and appliances for all; whereas the accessibility of AI and robotics is based on inclusive planning and design; whereas the user’s needs, wishes and experiences need to be the starting point of the design;

AJ.  whereas there are strong ethical, psychological and legal concerns about the autonomy of robots, their obvious lack of human empathy and their impact on the doctor-patient relationship, which have not yet been properly addressed at EU level, in particular as regards the protection of patients’ personal data, liability, and the new economic and employment relationships that will be brought about; whereas ‘autonomy’ as such can only by fully attributed to human beings; whereas there is a need for a robust legal and ethical framework for artificial intelligence;

AK.  whereas the introduction of artificial intelligence in the area of health in particular must always be based on the ‘man operates machine’ principle of responsibility;

1.  A society supported by artificial intelligence and robotics

1.1.  Labour in the era of artificial intelligence and robotics

1.  Stresses that automation combined with artificial intelligence will increase productivity and therefore increase output; notes that, as in previous technological revolutions, some jobs will be replaced but new jobs will also be created transforming lives and work practices; stresses that increased use of robotics and AI should also reduce human exposure to harmful and hazardous conditions and should also help to create more quality and decent jobs and improve productivity;

2.  Urges Member States to focus on retraining workers in the industries most affected by the automation of tasks; stresses that new education programmes should focus on developing the skills of workers so that they can seize job opportunities within the new jobs created by AI; encourages the development of digital literacy programmes in schools, the development of apprenticeships and vocational training priorities to help workers adapt to technological changes;

3.  Recommends that Member States, alongside private sector actors, identify the risks and develop strategies to ensure that relevant retraining and reskilling programmes are developed; underlines that companies themselves must invest in the training and reskilling of their existing workforce in order to meet their needs;

4.  Stresses that the development of robotics in the EU will have a strong impact on industrial relations; believes that this impact should be addressed in a balanced manner so as to promote reindustrialisation and allow workers to also enjoy the productivity gains;

5.  Notes that in the current industrial landscape there is a delicate balance between the owners and the workers; considers that advances in implementing AI in industry should be made with broad consultation of social partners, as the potential shift in the number of people working in the industry requires proactive policies to help workers adapt to the new demands and to ensure that the gains are broadly shared; notes that this requires re-thinking and re-designing labour market policies, social security schemes and taxation;

6.  Urges the Member States to address barriers to entry in the labour force such as excessive qualifications;

7.  Believes that digital literacy is one of the most important factors for future AI development and urges the Commission and the Member States to develop and pursue digital skills training and retraining strategies; notes that digital literacy can support wide and inclusive participation in data economy solutions and facilitate communication and cooperation with all stakeholders;

8.  Notes that, as citizens of all ages will be impacted, education curricula must be adapted, including through the establishment of new learning paths and the use of new delivery technologies; stresses that education aspects should be properly addressed; considers in particular the need for digital skills, including coding, to be included in teaching and training from the early school years to life-long learning;

1.2.  Malicious use of artificial intelligence and fundamental rights

9.  Highlights the fact that malicious or negligent use of AI could threaten digital security and physical and public safety, as it could be used to conduct large-scale, finely targeted and highly efficient attacks on information society services and connected machinery, as well as disinformation campaigns, and generally diminish the right of individuals to self-determination; stresses that the malicious or negligent use of AI might also pose a risk to democracy and fundamental rights;

10.  Calls on the Commission to propose a framework that penalises perception manipulation practices when personalised content or news feeds lead to negative feelings and distortion of the perception of reality that might lead to negative consequences (for example, election outcomes, or distorted perceptions on social matters such as migration);

11.  Stresses the importance of recognising, identifying and monitoring disruptive developments in and around the development of AI; encourages research into AI to also focus on the detection of accidentally or maliciously corrupted AI and robotics;

12.  Urges the Commission to take note of the social challenges arising from practices resulting from the ranking of citizens; stresses that citizens should not be subjected to discrimination on the basis of their ranking and that they should be entitled to ‘another chance’;

13.  Expresses great concern about the employment of AI applications, including facial and voice recognition, in ‘emotional surveillance’ programmes, i.e. monitoring the mental conditions of workers and citizens in order to increase productivity and preserve social stability, sometimes coupled with ‘social credit’ systems, as already seen in China, for instance; stresses that such programmes are inherently at odds with European values and norms protecting the rights and freedoms of individuals;

2.  The technological path towards artificial intelligence and robotics

2.1.  Research and development

14.  Recalls the fact that Europe has a world-leading AI research community, which accounts for 32 % of AI research institutions worldwide;

15.  Welcomes the Commission’s proposal on the digital Europe programme and the budget of EUR 2.5 billion pledged to artificial intelligence as well as increased funding under the Horizon 2020 programme; understands the importance of EU funding complementing Member State and industry research budgets for AI and the need for collaboration between public, private and EU research programmes;

16.  Supports the operational objectives of the Digital Europe Programme to build up and strengthen core artificial intelligence capacities in the Union, to make them accessible to all businesses and public administrations and to reinforce and network existing artificial intelligence testing and experimentation facilities in Member States;

17.  Encourages Member States to develop multi-stakeholder partnerships across industry and research institutes as well as joint AI centres of excellence;

18.  Stresses that AI research should invest not only in AI technology and innovation developments but also in AI-related social, ethical and liability areas; believes that any AI model deployed should have ethics by design;

19.  Stresses that, while encouraging progress for the benefit of society and the environment, AI research and other related activities should be conducted in accordance with the precautionary principle and fundamental rights; stresses that everyone involved in the development, implementation, dissemination and use of AI should consider and respect human dignity and the self-determination and wellbeing – both physical and psychological – of the individual and society at large, anticipate potential safety impacts and take due precautions proportionate to the level of protection, including the prompt disclosure of factors that might endanger the public or the environment;

20.  Stresses that a competitive research environment is also key to developing artificial intelligence; underlines the importance of supporting excellent research, including fundamental science and high-risk high-reward projects, and of fostering a European research area with attractive conditions for funding, mobility and access to infrastructure and technology across the Union, based on the principle of openness towards third countries and expertise from outside the Union, provided that it does not undermine EU cyber security;

21.  Underlines the fact that EU researchers continue to earn significantly less than their counterparts in the US and China, which is known to be the primary reason for them to leave Europe; calls on the Commission and the Member States to focus on attracting top talent to European companies, and on the Member States to create attractive conditions;

22.  Stresses that Europe must dedicate the new FET[7] flagship to artificial intelligence, with a particular emphasis on a human-centric approach and language technologies;

23.  Believes that artificial intelligence, machine learning and exponential leaps in data availability and cloud computing fuel research initiatives aimed at understanding biology at molecular and cellular level, guiding the development of medical treatments and analysing data streams to detect health threats, predicting disease outbreaks and counselling patients; notes that data mining and data-navigation techniques can be used to identify care gaps, risks, trends and patterns;

24.  Highlights that, where risks arise as an unavoidable and integral element of AI research, robust risk assessment and management protocols should be developed and complied with, taking into account that the risk of harm should be no greater than that encountered in ordinary life, (i.e. people should not be exposed to risks greater than or additional to those to which they are exposed in their normal lifestyles);

2.2.  Investments

25.  Notes the importance of greater investment in this field in order to remain competitive; recognises that while most of the investment and innovation in this area comes from private sector ventures, Member States and the Commission should also be encouraged to continue investing in research in this sector and outline their development priorities; welcomes the InvestEU proposal and other public-private partnerships that will foster private funding; considers that the coordination of private- and public-sector investment should be encouraged to ensure that development is focused;

26.  Stresses that investments in AI, which can be characterised by significant uncertainty, should be complemented by EU funding for example from the European Investment Bank (EIB) or the European Investment Fund (EIF), or through InvestEU and the European Fund for Strategic Investments (EFSI), schemes which can help with regard to risk sharing;

27.  Urges the Commission to not allow EU funding for weaponised AI; urges the Commission to exclude from EU funding companies that are researching and developing artificial consciousness;

28.  Recommends that the Commission ensure that the intellectual property of research conducted with EU funding remains in the EU and in European universities;

2.3.  Innovation, societal acceptance and responsibility

29.  Notes that all major technological advancements required a transition period, one where the majority of society needed to gain a deeper understanding of the technology and integrate it into their daily life;

30.  Notes that the future of this technology is contingent on societal acceptance and that greater emphasis must be placed on adequately communicating its benefits to ensure greater understanding of the technology and its applications; also notes that if society is not informed about AI technology, there will be less drive for innovation in this sector;

31.  Considers that public acceptance relies on how the public is informed about the opportunities, challenges and developments of artificial intelligence; recommends that the Member States and the Commission facilitate access to credible information addressing the main concerns about AI and robotics such as privacy, safety and transparency in decision-making;

32.  Welcomes the use of regulatory sandboxes to introduce, in cooperation with regulators, innovative new ideas, allowing safeguards to be built into the technology from the start, thus facilitating and encouraging its market entry; highlights the need to introduce AI-specific regulatory sandboxes to test the safe and effective use of AI technologies in a real-world environment;

33.  Notes that for greater societal acceptance of artificial intelligence, there must be assurances that the systems being used are safe and secure;

34.  Notes that Artificial intelligence and language technology can provide important applications to foster Europe’s unity in its diversity: automated translation, conversational agents and personal assistants, spoken-language interfaces for robots and the internet of things, smart analytics, automated identification of online propaganda, fake news, hate speech;

2.4.  Supporting conditions: connectivity, data accessibility and high-performance computing, and cloud infrastructure

35.  Stresses that the integration of robotics and AI technology within the economy and society require digital infrastructure that provides ubiquitous connectivity;

36.  Stresses that connectivity is a precondition for Europe to become part of the gigabit society and that AI is a clear example of the exponential growth of a demand for high-quality, fast, secure and pervasive connectivity; believes that the Union and the Member States should continue to foster measures to stimulate investment in and take-up of very high capacity networks in the EU;

37.  Underlines that a rapid, safe and secure development of 5G is essential to guarantee that the Union can reap the full benefits of AI and protect against cyber security threats, making it possible to renew and develop industries and services, which are the backbone of the European economy, and to support the emergence of new services, production and markets, which is essential in order to safeguard new jobs and a high level of employment;

38.  Recalls that the availability of high quality and meaningful data is essential for real competitiveness in the AI industry, and calls for public authorities to ensure ways of producing, sharing and governing data by making public data a common good while safeguarding privacy and sensitive data;

39.  Stresses the importance of the quality of data used in deep learning; notes that the use of low-quality, outdated, incomplete or incorrect data may lead to poor predictions and in turn discrimination and bias;

40.  Believes that the new set of rules governing the free flow of non-personal data in the Union allows for more and more data to become available for data-driven innovation, making it easier for SMEs and start-ups to develop innovative AI-enabled services and to enter new markets, while allowing citizens and businesses to benefit from better products and services;

41.  Notes that AI has the potential to increase efficiency, comfort and welfare in many sectors, if the established industrial stakeholders cooperate with AI developers; notes, furthermore, that a large volume of data that is not personal in nature is currently in the possession of stakeholders and through partnerships could be used to increase their efficiency; considers that, for that to become a reality, cooperation between the users and the developers of AI is a prerequisite;

42.  Stresses the importance of interoperability and data accuracy in order to ensure a high level of reliability and security standards in the new technologies;

43.  Believes that the success of AI applications, tailored to users across the EU, often requires extensive knowledge of local markets, as well as access to and use of adequate local data for the datasets training, system testing and validation, especially in sectors related to natural language processing; asks the Member States to encourage the availability of high-quality, interoperable and open public-sector and privately-held data;

44.  Stresses the need to ensure the utmost consistency with the EU’s big data policy;

45.  Welcomes measures to facilitate and support the exchange and sharing of data across borders;

46.  Notes that at present the sharing of data is well below its potential and that large quantities of data are underutilised;

47.  Recognises that there is a reluctance to share data and underlines the need for action to encourage it; notes that the lack of common standards also has a large role to play in the ability to share data;

48.  Welcomes regulations such as the Free Flow of Data Regulation and the importance it has in fields such as AI to allow for more effective and efficient processes;

49.  Recognises that greater market-based incentives need to be put in place to encourage access to and the sharing of data; notes the risk that data openness poses to investing in data in the first place;

50.  Calls for greater clarity on data ownership rules and the legal frameworks in place; notes that regulatory uncertainty has led to over-cautious responses from industry;

51.  Highlights the significance of European initiatives on cloud computing and high-performance computing, which will further enhance the development of deep-learning algorithms and the processing of big data; strongly believes that for these initiatives to be successful and relevant to the development of AI, the infrastructure needs to be open to both public and private entities based in the Union and elsewhere, and to be governed by least-restrictive access criteria;

52.  Welcomes the establishment of the European High-Performance Computing Joint Undertaking; underlines that supercomputing and data infrastructure are essential to ensure competitive innovation ecosystem for the development of AI technologies and applications;

53.  Highlights that cloud computing has a key role to play in driving the uptake of AI; underlines that access to cloud services allows private companies, public institutions, research and academic institutions, and users to develop and use AI in an efficient and economically viable way;

3.  Industrial policy

54.  Recalls that while AI and robotics already have long-established industrial applications, advancements in the field are expanding and providing wide and diverse applications in all human activities; believes that any regulatory framework must include flexibility that allows for innovation and free development of new technologies and uses for AI;

55.  Underlines that identifying the scope and applications for AI should be the result of a design process led by needs and guided by principles that take into account the intended result and the best path to achieve it, from an economic and social point of view; believes that the existence of clear policies at all stages of development will lead to fit-for-purpose implementation and address the risks and downsides;

56.  Recommends the use and promotion of public-private partnerships to explore solutions to key challenges such as building a data ecosystem and promoting the access, sharing and flow of data while safeguarding people’s rights to privacy;

57.  Stresses that a significant challenge to the future of AI systems is the inconsistent quality of software production technology, and underlines therefore the great need for standardisation in the construction and use of AI systems;

58.  Notes the work being conducted globally and recognises the need to work proactively with partners, especially at the OECD and G20, in shaping the direction this industry moves in to ensure that the EU remains competitive and safeguards equal access among nations, as well as sharing the benefits of AI development as widely as possible;

59.  Notes with concern that a number of non-European companies and entities from third countries are increasingly employing AI-based predictive models to provide services and extract the added value on EU markets, especially at local level, and to monitor and possibly influence political sentiment, thus posing potential threats to the technological sovereignty of EU citizens;

60.  Stresses that public support for AI should be focused on those strategic sectors in which EU industry has the greatest opportunities to play a leading role at a global level and which have added value in the general public interest;

3.1.  Priority sectors

3.1.1.  Public sector

61.  Highlights that there are a number of benefits to be gained from AI and robotics in the public sector, and welcomes greater investment in research and development to ensure that this thrives;

62.  Stresses that Member States should also invest in education and AI training programmes in order to assist public sector employees in adopting the use of AI and robotics; notes that there should also be information campaigns intended for the citizens who will use public sector services provided by AI systems and robotics in order to calm their fears over loss of control of their personal data and to establish trust;

63.  Stresses that public sector information represents an extraordinary source of data that can contribute to rapid progress and create a new strategy to embrace new digital technologies, especially artificial intelligence;

64.  Believes that public sector adoption of trustworthy artificial intelligence can strongly support the reform of public administration in decision-making and improve public services, as well as power the more widespread adoption of AI in other industries;

65.  Acknowledges the use of robotic process automation and the impact it has had in improving public sector processes; notes its interoperability with legacy systems;

66.  Asks the Member States to lead this digital transformation by positioning themselves as primary responsible users and buyers of AI technology; stresses in this context that Member States must adapt their data policies pertaining to public data collection, use, repositories or annotation, among other related subjects, to allow AI deployment in all public sectors;

67.  Emphasises the need to include the public in the AI development process; calls on the Commission, therefore, to publish any algorithms, tools or technology funded or co-funded by the public as open source;

68.  Believes that AI will be a great asset in terms of implementing the ‘once only’ principle, enabling databases and information from different sources to be combined, and thereby facilitating citizens’ interaction with public administrations;

69.  Calls on the Commission to guarantee the protection of citizens from any AI ranking decision systems in public administrations, similar to those planned to be used in China;

3.1.2.  Health

70.  Stresses that human contact is a crucial aspect of human care;

71.  Notes that AI and robotics have potential benefits in the care-giving sector as life expectancy increases, for instance helping doctors and nurses to have more time for high value activities(e.g. patient interaction);

72.  Notes the impact that AI has already had on wellbeing, prevention, diagnosis and research and its great potential for designing personalised care; considers that this ultimately leads to a more sustainable, efficient and outcome-based healthcare ecosystem;

73.  Notes that when AI is combined with human diagnosis, the error rate tends to be significantly lower than for diagnosis by human doctors alone[8];

74.  Stresses that the use of data in the health sector must be monitored carefully and ethically and must not in any way obstruct access to social protection or insurance;

75.  Believes that when AI is being used in implanted medical devices, the bearer should have the right to inspect and modify the source code used in the device;

76.  Special attention should be paid to the use of big data in health with the aim of maximising the opportunities it can bring – such as improving the health of individual patients, as well as the performance of Member States’ public health systems – without lowering ethical standards and without threatening the privacy or safety of citizens;

77.  Stresses, however, that the existing system for the approval of medical devices may not be adequate for AI technologies; calls on the Commission to closely monitor progress on these technologies and to propose changes to the regulatory framework if necessary in order to establish the framework for determining the respective liability of the user (doctor/professional), the producer of the technological solution, and the healthcare facility offering the treatment; points out that legal liability for damage is a central issue in the health sector where the use of AI is concerned; stresses the need therefore to ensure that users will not be led invariably to back the diagnostic solution or treatment suggested by a technological instrument for fear of being sued for damages if, on the basis of their informed professional judgement, they were to reach conclusions that diverged even in part;

78.  Calls on the Member States and the Commission to increase funding in health-related AI technologies in the public and private sectors; welcomes, in this context, the declaration of cooperation signed by 24 EU Member States and Norway with a view to boosting the impact of investments in AI at European level; calls on the Member States and the Commission to consider whether training programmes for medical and healthcare personnel should be updated and standardised on a Europe-wide basis so as to ensure high levels of expertise and a level playing field in the Member States as regards knowledge and use of the most advanced technological instruments in robotic surgery, biomedicine, and AI-based biomedical imaging;

79.  Calls on the Commission to work on strategies and policies that can position the EU as a world leader in the growing field of healthcare technology, while ensuring that patients have access to seamless and effective medical care;

80.  Recognises that better diagnostics could save millions of lives as, according to the World Health Organisation, 89 % of premature deaths across Europe are caused by non-communicable diseases;

81.  Highlights the contribution made by AI and robotics to innovative preventive, clinical and rehabilitation practices and techniques in the health sector, with particular reference to the benefits they have for patients with disabilities;

82.  Recognises that the increased use of sensors in the field of robotics has enhanced the scope of care-giving and enables patients to have more personalised treatment and services and receive care remotely from their own homes, while also generating more meaningful data;

83.  Recognises that, according to the Eurobarometer survey of May 2017[9], at present EU citizens still feel uncomfortable with the idea of robots being used in everyday healthcare; calls on the Commission and Member States to develop strategies and communication campaigns to raise awareness of the benefits of day-to-day use of robots; notes in particular the ambition of Japan’s Robot Strategy;

3.1.3.  Energy

84.  Notes that AI allows energy suppliers to move from preventive to predictive asset maintenance and to achieve more efficient energy production by improving reliability in particular, for renewables, and by identifying the most effective locations for new installations, thus ensuring better demand response management;

85.  Recognises that more accurate data produced by AI on the potential of renewable energy production will create higher investment certainty for businesses and individuals, thus accelerating the energy transition towards renewable energy sources and contributing to the Union’s long-term strategy for a climate neutral economy;

86.  Notes that solutions involving sensors are already being used to manage energy usage in houses and that this has resulted inconsiderable energy and monetary savings;

87.  Welcomes the potential of AI in modelling, identifying and mitigating the impact of human activity on climate; notes that, while increased digitalisation also brings new energy needs, it can also bring increased efficiency to energy-intensive sectors and provide for better understanding of processes, leading to their improvement;

88.  Stresses that with more a digitalised energy sector energy networks become larger and more exposed to cyber threats; calls on the Member States and the Commission to accompany the digital transformation in the energy sectors with measures, such as artificial intelligence, that improve cyber security;

3.1.4.  Transport

89.  Welcomes the ability of AI and robotics to greatly improve our transport systems through the introduction of autonomous trains and motor vehicles; calls for greater research and investment in this area to ensure its safe and effective development; highlights the tremendous opportunities for both larger tech companies and SMEs;

90.  Notes that by reducing human error in the transport sector the system can potentially become more efficient, with fewer accidents, thanks to clearer evaluations and the predictive nature of the technology, fewer delays, with the ability to map traffic patterns and run services on schedule, and greater savings, with fewer driver-related faults and streamlined internal processes;

91.  Notes that the prevalence of autonomous vehicles in the future poses risks to data privacy and technical failures and will shift the liability from the driver to the manufacturer, requiring insurance companies to shift how they incorporate risk into their underwriting;

92.  Notes that voice communication is increasingly used in interaction with vehicles and transport systems but these features are only available for a handful of European languages, so it should be ensured that all Europeans can use these possibilities in their mother tongue;

3.1.5.  Agriculture and the food chain

93.  Notes that AI has the potential to catalyse a disruptive transformation of the current food system towards a more diverse, resilient, regionally adapted and healthy model for the future;

94.  Notes the role that AI can play in efforts to help tackle food security issues, predict famine and foodborne-disease outbreaks, reduce food loss and waste and improve sustainable management of land, water and other environmental resources critical to ecosystem health;

95.  Highlights that AI can intervene at critical points along the food system value chain from production to consumption and enhance our capacity to fundamentally alter the way we produce, process and buy food by better informing land-use planning practices;

96.  Notes that AI can improve resource management and input efficiency, help reduce post-harvest waste and influence consumption choices;

97.  Notes that AI in the form of precision farming holds the potential for disruptive transformation of agricultural production, as well as broader land management, by improving land use planning, predicting land use change and monitoring crop health, while also having the potential to transform the prediction of extreme weather events;

98.  Notes that AI can radically alter the delivery of inputs, pest control and farm management, influence farming practices, alter the way insurance products are delivered and help predict and avoid future famine and severe acute malnutrition outbreaks;

99.  Notes that AI can lead to better decisions about how to manage farm systems and stimulate the development of decision-support and recommendation systems by improving farm efficiencies and farm health;

3.1.6.  Cybersecurity

100.  Notes that cybersecurity is an important aspect of AI, especially given the challenges for transparency in high level AI; considers that the technological perspective, including auditing of the source code, and requirements for transparency and accountability should be complemented by an institutional approach dealing with the challenges of introducing AI developed in other countries into the EU single market;

101.  Calls for the swift implementation of the Cybersecurity Act; notes that the development of EU certification schemes should ensure a more resilient development and deployment of safe AI and robotic systems;

102.  Considers that AI can simultaneously be a cybersecurity threat and the tool for fighting cyber-attacks; believes that the EU Agency for Network and Information Security (ENISA) should prepare an action plan on cybersecurity in the area of AI, which should assess and address threats and weaknesses specific to AI;

103.  Underlines the importance of strengthening the industrial base as a strategic component of secure AI development; emphasises that in order to ensure an ambitious level of cybersecurity, data protection and trusted ICT services, Europe must invest in its technological independence; stresses the urgent need for the EU to develop its own infrastructure, data centres, cloud systems and components, such as graphics processors and chips;

104.  Notes that as AI evolves and hackers become more sophisticated, it will be imperative to have strong cybersecurity solutions;

105.  Recognises that implementation of AI solutions in cybersecurity will make the forecasting, prevention and mitigation of threats possible;

106.  Highlights that while AI will be able to provide greater coverage for the detection of threats, it is imperative to have human interpretation of these threats in order to detect whether they are genuine or not;

107.  Calls on the Commission to explore the use of blockchain-based cybersecurity applications that improve the resilience, trust and robustness of AI infrastructures through disintermediated models of data encryption; calls on the Commission to explore the possibility of rewarding citizens for their data by means of tokens;

108.  Calls on the Commission to reinforce the EU’s cybersecurity capacity by further combining and coordinating efforts across Europe;

3.1.7.  SMEs

109.  Recognises the importance of SMEs for the success of AI; welcomes the Commission initiative to create an AI on-demand platform that will boost technology transfer and catalyse the growth of start-ups and SMEs; calls on the Commission to promote digital innovation hubs for AI that do not lead to the creation of additional administration layers but instead focus on accelerating investments in projects that have proved to be efficient;

110.  Notes that the costs of investing in AI leads to high barriers to entry for SMEs; recognises that widespread adoption of AI by consumers would de-risk this investment for SMEs;

111.  Highlights the need to promote both the adoption of AI by SMEs and the use of it by consumers;

112.  Stresses the importance of targeted measures to ensure that SMEs and start-ups are able to adopt and benefit from AI technologies; believes that impact assessments of the effects of new EU legislation on the technological development of AI should be mandatory, and that such impact assessments should also be considered at national level;

113.  Underlines that AI can be an enabler for SMEs but also increases the leverage of big early adopters and developers; points to the need, therefore, from a competition point of view, to ensure that new distortions are properly evaluated and addressed;

4.  Legal framework for artificial intelligence and robotics

114.  Calls on the Commission, with a view to fostering a regulatory environment favourable to the development of AI and in line with the principle of better regulation, to regularly re-evaluate current legislation to ensure that it is fit for purpose with respect to AI while also respecting EU fundamental values, and to seek to amend or substitute new proposals where this is shown not to be the case;

115.  Welcomes the setup of AI-based participative platforms allowing citizens to be successfully heard and to interact with governments by making proposals, including through participatory budgets and other instruments of direct democracy; stresses that bottom-up projects can foster citizen participation and help people make informed decisions in a more effective and democratic way;

116.  Notes that AI is a notion encompassing a wide range of products and applications, from automation, algorithms and narrow artificial intelligence to general artificial intelligence; considers that a comprehensive law or regulation on AI should be approached with caution, as sectoral regulation may provide policies that are general enough but also refined up to the level where they are meaningful for the industrial sector;

117.  Stresses that the policy framework must be designed to encourage the development of all kinds of AI and not only deep learning systems, which require a huge amount of data;

4.1.  An internal market for artificial intelligence

118.  Underlines the importance of the principle of mutual recognition in the cross-border use of smart goods, including robots and robotic systems; recalls that, when necessary, testing, certification and product safety should ensure that certain goods are safe by design and by default; notes in this context the importance of also working on the ethical aspects of AI;

119.  Underlines that EU legislation related to the implementation of the Digital Single Market strategy should remove barriers to the deployment of AI; calls on the Commission to evaluate where it is necessary to update policy and regulatory frameworks in order to build a single European market for AI;

120.  Recognises that robotics and AI technologies are increasingly used in autonomous vehicles, such as autonomous cars and civilian drones; notes that some Member States are already enacting or considering legislation in this area in particular, which could result in a patchwork of national legislation hampering the development of autonomous vehicles; calls, therefore, for a single set of Union rules that strikes the right balance between the interests of and potential risks for users, businesses and other concerned parties, while avoiding over-regulation in robotics and AI systems;

121.  Urges the Member States to modernise their professional training and education systems in order to take into account scientific progress and developments in the field of AI, in line with the Proportionality Test Directive[10] and the Professional Qualifications Directive[11], and to make EU professional services globally competitive in the coming decades;

122.  Stresses that AI applies to a variety of sectors where standardisation is of high relevance, such as smart manufacturing, robotics, autonomous vehicles, virtual reality, healthcare and data analysis, and believes that EU-wide standardisation for AI will foster innovation and guarantee a high level of consumer protection; acknowledges that, while a significant number of standards on issues such as safety, reliability, interoperability and security exist, further promotion and development of common standards for robotics and AI is necessary and should be part of the Union’s priorities; calls on the Commission, in cooperation with EU standardisation bodies, to continue to engage proactively with international standardisation bodies on improving standards in this field;

123.  Recalls that many policy aspects relevant for AI-enabled services, including rules on consumer protection and policy on ethics and liability, are covered by the existing regulatory framework on services, namely the Services Directive[12], the Professional Qualifications Directive and the e-Commerce Directive[13]; underlines, in this context, that humans must always be ultimately responsible for decision-making, especially for professional services such as the medical, legal and accounting professions; considers that reflection is needed on whether supervision by a qualified professional is necessary, with a view to protecting legitimate public interest objectives and providing high-quality services;

124.  Recognises the importance of improved digital services such as virtual assistants, chatbots and virtual agents, bringing unprecedented operational efficiencies, while duly acknowledging the need to develop a human-centric, market-driven AI to produce better and more reliable decisions in view of the limits to the autonomy of AI and robotics;

4.2.  Personal data and privacy

125.  Stresses that a high level of safety, security and privacy must be ensured with respect to data used for communication between people and robots and artificial intelligence; calls on the Commission and the Member States therefore to integrate the security and privacy by design principles in their policies related to robotics and artificial intelligence;

126.  Reiterates that the right to the protection of private life and the right to the protection of personal data as enshrined in Articles 7 and 8 of the Charter of Fundamental Rights and Article 16 of the Treaty on the Functioning of the European Union apply to all areas of robotics and artificial intelligence and that the Union legal framework for data protection must be fully complied with; underlines the responsibility of designers of robotic systems and artificial intelligence to develop products in such a way that they are safe, secure and fit for purpose and follow procedures for data processing compliant with existing legislation, confidentiality, anonymity, fair treatment and due process;

127.  Calls on the Commission to ensure that any Union legislation on artificial intelligence includes measures and rules which take into account the rapid technological evolution in this field, so as to ensure that Union legislation does not lag behind the curve of technological development and deployment; stresses the need for such legislation to be compliant with rules on privacy and data protection; calls for the review of rules, principles and criteria regarding the use of cameras and sensors in robots and with regard to artificial intelligence in accordance with the Union legal framework for data protection;

128.  Calls on the Commission to ensure that any future EU regulatory framework on AI guarantees the privacy and confidentiality of communications, personal data protection, including the principles of lawfulness, fairness and transparency, data protection by design and default, purpose limitation, storage limitation, accuracy, and data minimisation in full compliance with Union data protection law, as well as security, personal safety and other fundamental rights, such as the right to freedom of expression and information;

129.  Stresses that the right to privacy must always be respected and individuals must not be personally identifiable; underlines that an AI developer should always have clear, unambiguous and informed consent and that AI designers have a responsibility to develop and follow procedures for valid consent, confidentiality, anonymity, fair treatment and due process; stresses that designers must comply with any requests that any related data be destroyed and removed from any datasets;

130.  Recalls that Regulation (EU) 2018/1807[14] on the free flow of non-personal data states that ‘if technological developments make it possible to turn anonymised data into personal data, such data are to be treated as personal data, and Regulation (EU) 2016/679 is to apply accordingly’;

4.3.  Liability

131.  Welcomes the Commission’s initiative to create the Expert Group on Liability and New Technologies with the aim of providing the EU with expertise on the applicability of the Product Liability Directive[15] to traditional products, new technologies and new societal challenges (Product Liability Directive formation) and assisting the EU in developing principles that can serve as guidelines for possible adaptations of applicable laws at EU and national level relating to new technologies (New Technologies formation);

132.  Regrets, however, that no legislative proposal was put forward during this legislature, thereby delaying the update of the liability rules at EU level and threatening the legal certainty across the EU in this area for both traders and consumers;

133.  Notes that AI engineers or the companies employing them should remain accountable for the social, environmental and human health impacts that AI systems or robotics may have on present and future generations;

4.4.  Consumer protection and empowerment

134.  Underlines that consumer trust is essential to the development of AI and that AI-based systems deal with more and more consumer data, which makes them prime targets for cyber-attacks; also highlights the fact that AI must function in a way that is not detrimental to citizens and consumers, and considers that the integrity of the data and algorithms it relies on must therefore be ensured;

135.  Considers that AI technologies developed for both manufacturing and individual use should be subject to product safety checks by market surveillance authorities and consumer protection rules ensuring, where appropriate, minimum safety standards and addressing the risk of accidents resulting from interaction with, or working in proximity to, humans; believes that ethical issues and issues of data protection, including third-party and personal data, civil liability and cybersecurity, should be considered in any policy on AI;

4.5.  Intellectual property rights

136.  Recalls its above-mentioned resolution of 16 February 2017 in which it noted that there are no legal provisions that specifically apply to robotics, but that existing legal regimes and doctrines can be readily applied to robotics, although some aspects appear to call for specific consideration; renews the call made in that resolution on the Commission to support a horizontal and technologically neutral approach to intellectual property applicable to the various sectors in which robotics could be employed;

137.  Welcomes in this regard the communication from the Commission to the Institutions on Guidance on certain aspects of Directive 2004/48/EC of the European Parliament and of the Council on the enforcement of intellectual property rights (COM(2017)0708) but underlines the need to monitor the relevance and efficiency of rules on intellectual property rights to govern the development of AI; underlines, in this context, the importance of fitness checks;

5.  Ethical aspects

138.  Believes that artificial intelligence actions and applications should comply with ethical principles and relevant national, Union and international law;

139.  Calls for the creation of an ethical charter of best practice for AI and robotics that companies and experts should follow;

140.  Calls on the Commission and the Member States to promote strong and transparent cooperation between the public and private sectors and academia that would reinforce knowledge sharing, and to promote education and training for designers on ethical implications, safety, and respect for fundamental rights, as well as for consumers on the use of robotics and artificial intelligence, with a particular focus on safety and data privacy;

141.  Calls on the Commission to ensure that applications based on AI should not use data collected from various sources without first receiving the consent of the data subject; calls on the Commission to create a framework that makes sure that consent given by the data subject will generate data only for the intended purposes;

142.  Calls on the Commission to respect the right of citizens to an offline life and to ensure that there is no discrimination against citizens on whom no data has been recorded;

5.1.  Human-centric technology

143.  Stresses that ethical rules must be in place to ensure human-centric AI development, the accountability and transparency of algorithmic decision-making systems, clear liability rules and fairness;

144.  Welcomes the Commission’s initiative to set up the High-Level Expert Group on Artificial Intelligence, as well as the EU AI alliance network, with the aim of delivering ethical guidelines for AI; calls on the Commission to ensure the largest possible uptake of those ethical guidelines by the industry, academia and public authorities; recommends that Member States incorporate the guidelines into their national AI strategies and develop real accountability structures for industries and governments as they design and deploy AI;

145.  Considers that continuous follow-up on the implementation of the AI ethical guidelines and its impact on the development of human-centric AI is essential; calls on the Commission to analyse whether the voluntary ethical guidelines are sufficient to ensure that the inclusive, ethically embedded uptake of AI does not generate economic and social divides in EU societies, and suggest regulatory and policy measures if necessary;

146.  Notes the recent developments in monitoring and adapting to behavioural analytics; calls on the Commission to develop an ethical framework that limits its use; urges the Commission to create awareness and launch an information campaign on AI and its use as regards behavioural analytics;

5.2.  Embedded values in technology – ethical-by-design

147.  Points out that the guiding ethical framework should be based on the principles of beneficence, non-maleficence, autonomy and justice, on the principles and values enshrined in Article 2 of the Treaty on European Union and in the Charter of Fundamental Rights, such as human dignity, equality, justice and equity, non-discrimination, informed consent, private and family life and data protection, as well as on other underlying principles and values of Union law, such as non-stigmatisation, transparency, autonomy, individual responsibility and social responsibility, and on existing ethical practices and codes;

148.  Believes that Europe should take the lead on the global stage by deploying only ethically embedded AI; underlines that, to achieve this, the governance of ethics in AI must be ensured at different levels; recommends that the Member States establish AI ethics monitoring and oversight bodies and encourage companies developing AI to set up ethics boards and draw up ethical guidelines for their AI developers;

149.  Stresses that European standards for AI must be based on the principles of digital ethics, human dignity, respect for fundamental rights, data protection, and security, thus contributing to building trust among users; emphasises the importance of capitalising on the EU’s potential for creating a strong infrastructure for AI systems rooted in high standards of data and respect for humans; notes that transparency and explainability need to be embedded in the development of AI;

150.  Notes that automated weapons systems should continue to have a human-in-command approach to artificial intelligence;

5.3.  Decision-making – limits to the autonomy of artificial intelligence and robotics

151.  Stresses the difficulty and complexity of predicting the future behaviours of many complex AI systems and the emerging behaviours of interacting AI systems; asks the Commission to evaluate whether there is a need for specific regulations related to AI-enabled decision-making;

152.  Notes that artificial intelligence will remain a useful tool for collaboration in human action to improve its performance and reduce errors;

153.  Calls for people to have a right to know, a right of appeal and a right to redress when AI is used for decisions affecting individuals which carry a significant risk to an individual’s rights or freedom or may cause them harm;

154.  Stresses that algorithms in decision-making systems should not be deployed without a prior algorithmic impact assessment (AIA), unless it is clear that they have no significant impact on the life of individuals;

155.  Believes that artificial intelligence, especially systems with built-in autonomy, including the capability to independently extract, collect and share sensitive information with various stakeholders, and the possibility of self-learning or even evolving to self-modify, should be subject to robust principles; stresses that AI systems must not keep or disclose personal confidential information without explicit approval from the source of that information;

5.4.  Transparency, bias and explainability of algorithms

156.  Points out that while AI brings great benefits in automation and decision-making, it also carries an inherent risk when the algorithms are static and opaque; stresses, in this context, the need for greater transparency with regard to algorithms;

157.  Calls on the Commission, the Member States and the data protection authorities to identify and take all possible measures to prevent or minimise algorithmic discrimination and bias and to develop a strong common ethical framework for the transparent processing of personal data and automated decision-making to guide data usage and the enforcement of Union law;

158.  Underlines that any AI system must be developed with respect for the principles of transparency and algorithmic accountability allowing for human understanding of its actions; notes that in order to build trust in and enable the progress of AI, users must be aware of how their data, as well as other data and data inferred from their data, is used when they are communicating or interacting with an AI system or with humans supported by an AI system; believes that this will contribute to better understanding and confidence among users; stresses that the intelligibility of decisions must be an EU standard in accordance with Articles 13, 14 and 15 of the General Data Protection Regulation (GDPR)[16]; recalls that the GDPR already provides for a right to be informed about the logic involved in data processing; stresses that, in accordance with Article 22 of the GDPR, individuals have the right to obtain human intervention when a decision based on automated processing significantly affects them;

159.  Highlights that the Commission, the European Data Protection Board, national data protection authorities and other independent supervisory authorities should henceforth play a crucial role in the promotion of transparency and due process, legal certainty in general and, more specifically, concrete standards protecting fundamental rights and guarantees associated with the use of data processing and analytics; calls for closer collaboration among authorities charged with overseeing or regulating conduct in the digital environment; calls for adequate funding and staffing of such authorities;

160.  Acknowledges that machine learning algorithms are trained to learn by themselves, benefiting automation and decision-making; calls for AI ethics guidelines to address issues related to algorithmic transparency, explainability, accountability and fairness;

161.  Highlights the importance of the explainability of AI systems’ outputs, processes and values, making them understandable to non-technical audiences and providing them with meaningful information, which is necessary to evaluate fairness and gain trust;

162.  Points out that the lack of transparency with respect to these technologies and their applications raises a number of ethical issues;

163.  Notes that AI systems should be explainable to humans and should provide meaningful information so that feedback can be given; recognises that the strength of AI models is dependent on feedback and reassessment and encourages this process;

164.  Notes that citizens are concerned about not knowing when AI is being used and what information will be processed; recommends clear disclosure when AI is being used by citizens; stresses that, in order to maintain consumer trust, it is important that data transmitted remains secure;

165.  Considers that algorithmic accountability should be regulated by policymakers through impact assessments based on established parameters;

166.  Notes that disclosing the computer code itself will not solve the AI transparency issue, because it would not reveal the inherent biases that exist and would fail to explain the machine-learning process; underlines that transparency means not only transparency of code, but also of data and automated decision-making;

167.  Acknowledges that the disclosure of source code could lead to misuse and the gaming of algorithms;

168.  Highlights the importance of tackling developer bias, and thus the need for a diverse workforce in all branches of the IT sector, as well as safeguard mechanisms, to avoid biases based on gender and age embedded into AI systems;

169.  Acknowledges that revealing the code or trade secrets would also discourage companies from R&D of new code since their intellectual property would be at risk; notes that development of AI should encourage instead the interpretability of models and their interaction with the input and training data;

170.  Acknowledges that although transparency and explainability may reveal deficiencies, they do not guarantee reliability, security and fairness; considers accountability, therefore, to be integral to achieving trustworthy artificial intelligence, which can be attained via different means, such as AIAs, auditing and certification;

171.  Stresses the need for the development of protocols for the ongoing monitoring and detection of algorithmic bias;

172.  Points out that designers of algorithms should ensure that essential requirements such as fairness and explainability are adhered to from the beginning of the design phase and throughout the development cycle;

173.  Νotes the need for guidelines describing good development practices;

174.  Stresses the importance of showing lineage in order to be able to trace the history of the AI model; considers that this will improve understanding of the models and help establish trust based on their history;

175.  Stresses that the use of AI systems must be clearly identified in interactions with users;

176.  Stresses that the dissemination of artificial intelligence and robotics should take place while fully respecting human rights and that on no account should stereotypes against women, or any other form of discrimination, be reproduced in machines and robots;

177.  Points out that even high-quality training data can lead to a perpetuation of existing discrimination and injustice when not used carefully and consciously; notes that the use of low-quality, outdated, incomplete or incorrect data at different stages of data processing may lead to poor predictions and assessments and in turn bias, which can eventually result in infringements of the fundamental rights of individuals or purely incorrect conclusions or false outcomes; believes, therefore, that it is important in the age of big data to ensure that algorithms are trained on representative samples of high-quality data in order to achieve statistical parity; emphasises that even if accurate high-quality data is used, predictive analysis based on AI can only offer a statistical probability; recalls that, under the GDPR, the further processing of personal data for statistical purposes, including AI training, may only result in aggregate data which cannot be re-applied to individuals;

178.  Calls on the Commission to ensure that anyone who produces deepfake material or synthetic videos, or any other realistically made synthetic videos, explicitly states that they are not original;

179.  Notes that AI inherently relies on gathering large amounts of data, and often on the creation of new databases that are used to make assumptions about people; believes that emphasis should be placed on identifying and building response mechanisms for potential threats to ensure mitigation of negative effects;

180.  Reiterates that AI systems should not create or reinforce bias; underlines that, when developing and using algorithms, considerations around bias and fairness must be included at all stages, from design to implementation; underlines that the dataset and algorithm must be assessed and regularly tested to ensure accurate decision-making;

6.  Governance

6.1.  Coordination at Union level

181.  Calls on the Commission to work towards developing strong EU leadership to prevent duplication and fragmentation of efforts and ensure coherent national-level policies and the exchange of best practices for wider usage of AI;

182.  Welcomes the different national strategies developed by the Member States; welcomes the Commission’s Coordinated Plan on Artificial Intelligence, published on 7 December 2018; calls for better cooperation between the Member States and the Commission in this regard;

183.  Notes that a number of Member States already have their own national AI strategies and welcomes the fact that all Member States signed a Declaration on Cooperation on Artificial Intelligence in April 2018; welcomes also the upcoming coordinated plan on AI between the Commission and the Member States, but calls on all the parties involved to strive for the highest possible level of cooperation;

184.  Believes that enhanced cooperation between the Member States and the Commission is necessary in order to guarantee coherent cross-border rules in the Union which encourage collaboration between European industries and allow the deployment across the entire Union of AI which is consistent with the required levels of safety and security, as well as the ethical principles enshrined in Union law;

185.  Stresses that a harmonised, risk-based and progressive EU data policy framework would increase trust and support the path of AI in Europe, thereby ensuring the completion of the Digital Single Market and increasing the productivity of Europe-based businesses;

186.  Recommends that existing and future AI-related initiatives and pilot projects carried out by the Commission should be closely coordinated, possibly under the guidance of the proposed oversight mechanism, so as to realise synergy effects and ensure the creation of real added value while avoiding costly double structures;

187.  Calls on the Commission and the Member States to consider the creation of a European regulatory agency for AI and algorithmic decision-making tasked with:

•  Establishing a risk assessment matrix for classifying algorithm types and application domains according to their potential for a significant negative impact on citizens;

•  Investigating the use of algorithmic systems where a case of infringement of human rights is suspected (with evidence provided by a whistle-blower, for example);

•  Advising other regulatory agencies about algorithmic systems falling within their remit;

•  Enhancing the effectiveness of the tort liability mechanism as a means to regulate accountability of algorithmic systems by providing a contact point for citizens who are not familiar with legal procedures;

•  Auditing the AIAs of high-level impact systems to approve or reject the proposed uses of algorithmic decision-making in highly sensitive and/or safety-critical application domains (private health-care, for instance); the AIA for private sector applications could follow a very similar process to the one proposed for the public sector, with the possible difference that the various stages of public disclosure could be handled as confidential communication to the regulatory agency (under a non-disclosure agreement) in order to safeguard vital trade secrets;

•  Investigating suspected cases of rights violations by algorithmic decision-making systems, for both individual decision instances (singular aberrant outcomes, for example) and statistical decision patterns (discriminatory bias, for instance); investigations could be triggered following the lodging of complaints, or on the basis of evidence provided by whistle-blowers, investigative journalists or independent researchers (including NGOs and academics);

188.  Notes the ongoing work on AI by the International Organisation for Standardisation (ISO), and urges the Member States to coordinate their ISO members to ensure that European interests are represented accordingly in the development of standards in this area;

6.2.  International governance

189.  Welcomes the creation of the OECD AI Policy Observatory and calls for greater ambition in developing a roadmap for further cooperation;

190.  Stresses the different models being developed in third countries, specifically in the US, China, Russia and Israel, and highlights the values-based approach used in Europe and the need to work with international partners in bilateral and multilateral settings, for the ethical advancement and adoption of AI; recognises that this technology does not have any borders and requires cooperation beyond that of the EU Member States alone;

191.  Calls on the Commission to work at an international level to ensure the greatest consistency between international players and to advocate EU ethical principles worldwide;

192.  Underlines that AI is a technology with a global impact, providing shared benefits and posing similar challenges; points to the need for a global approach, as in the case of the economic system, and especially in the case of a technology that has a significant impact on the markets; stresses the need for AI to be put on the agenda of existing institutions and organisations, and calls for an assessment of the need for additional fora, which should be created if necessary;

°

°  °

193.  Instructs its President to forward this resolution to the Council and the Commission.

  • [1]  OJ C 252, 18.7.2018, p. 239.
  • [2]  OJ C 307, 30.8.2018, p. 163.
  • [3]  Texts adopted, P8_TA(2018)0341.
  • [4]  Texts adopted, P8_TA(2018)0332.
  • [5]  OJ L 252, 8.10.2018, p. 1.
  • [6]  COM(2018)0237.
  • [7]  Future and Emerging Technologies.
  • [8]  OECD Digital Economy Outlook 2017.
  • [9]  Special Eurobarometer 460.
  • [10]  Directive (EU) 2018/958 of the European Parliament and of the Council of 28 June 2018 on a proportionality test before adoption of new regulation of professions, OJ L 173, 9.7.2018, p. 25.
  • [11]  Directive 2013/55/EU of the European Parliament and of the Council of 20 November 2013 amending Directive 2005/36/EC on the recognition of professional qualifications and Regulation (EU) No 1024/2012 on administrative cooperation through the Internal Market Information System (‘the IMI Regulation’), OJ L 354, 28.12.2013, p. 132.
  • [12]  Directive 2006/123/EC of the European Parliament and of the Council of 12 December 2006 on services in the internal market, OJ L 376, 27.12.2006, p. 36.
  • [13]  Directive 2000/31/EC of the European Parliament and of the Council of 8 June 2000 on certain legal aspects of information society services, in particular electronic commerce, in the Internal Market (‘Directive on electronic commerce’), OJ L 178, 17.7.2000, p. 1.
  • [14]  Regulation (EU) 2018/1807 of the European Parliament and of the Council of 14 November 2018 on a framework for the free flow of non-personal data in the European Union, OJ L 303, 28.11.2018, p. 59.
  • [15]  Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products, OJ L 210, 7.8.1985, p. 29.
  • [16]  Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC, OJ L 119, 4.5.2016, p. 1.

OPINION of the Committee on the Internal Market and Consumer Protection (7.12.2018)

for the Committee on Committee on Industry, Research and Energy

on the comprehensive European industrial policy on Artificial Intelligence and Robotics
(2018/2088(INI)COM(2018)0237))

Rapporteur for opinion: Dita Charanzová

(*)  Associated committee – Rule 54 of the Rules of procedure

(Simplified procedure – Rule (50)2 of the Rules of Procedure)

SUGGESTIONS

The Committee on the Internal Market and Consumer Protection calls on the Committee on Industry, Research and Energy, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A.  Whereas developments in Artificial Intelligence (AI) are unfolding at a fast pace and whereas AI has already played a part in our daily lives for several years; whereas AI and robotics are boosting innovation, leading to new business models and playing a key role in transforming our societies and digitalising our economies in many sectors, such as industry, healthcare, construction and transport;

B.  Whereas China and the United States are the leading countries in AI technologies and the EU is now lagging substantially behind; whereas it will be impossible for the EU to catch up without a coordinated approach at European level; whereas a common approach to AI would allow the EU to better promote its values worldwide and to play a greater and more effective role on the global stage;

C.  Whereas around a quarter of all industrial robots and half of all professional service robots in the world are produced by European companies, and the EU therefore already has important assets on which it should base its European Industrial Policy;

D.  Whereas a common approach will facilitate the development of AI technologies for the benefit of society, while also addressing the challenges presented by these technologies in order to foster innovation, enhance the quality of AI-enabled products and services, improve consumer experience and trust in AI technologies and robotics, and avoid the fragmentation of the internal market;

E.  Whereas the development of AI and robotics needs to include society as a whole; whereas, however, in 2017, rural areas remained largely excluded from the benefits of AI, as 8 % of homes are not covered by any fixed network, and 53 % are not covered by any ‘next-generation access’ technology (VDSL, Cable Docsis 3.0 or FTTP);

F.  Whereas development of AI-enabled services and products requires connectivity, free flow of data and accessibility of data within the EU; whereas use of advanced data mining techniques in services and products may help to increase the quality of decision‐making, and hence consumer choice, and improve business performance;

G.  Whereas cybersecurity is vital for ensuring that data is not maliciously corrupted or misused to make AI function in a way that is detrimental to citizens or companies, which would undermine industry and consumer trust in AI; whereas developments in AI increase reliance on these systems for actions and decisions, which in turn necessitates high standards of cyber resilience in the EU to protect against cybersecurity breaches and failures;

H.  Whereas technological developments in smart products and services can benefit the knowledge economy, which is based on the quantity, quality and accessibility of information available, and can thus lead to better adaptation to consumer needs;

I.  Whereas consumers should be able to trust the technology they interact with, and whereas there is therefore a need to respond to ethical concerns over the development of AI and robotics; whereas the EU should base its response to these concerns on respect for the values enshrined in Article 2 of the Treaty on European Union and the EU Charter of Fundamental Rights; whereas the High-Level Expert Group on Artificial Intelligence established by the Commission will develop draft AI ethics guidelines by the end of 2018;

1.  Underlines that the first and most important condition in order for the EU to catch up on AI is that it act within a common framework;

2.  Stresses that uncoordinated actions in terms of R&D, diverging decisions in terms of regulation and actions at international level further distance the EU from this goal;

3.  Notes that a number of Member States already have their own national AI strategies and welcomes the fact that all Member States signed a Declaration on Cooperation on Artificial Intelligence in April 2018, as well as the upcoming coordinated plan on AI between the Commission and the Member States, but calls on all involved parties to aim at the highest possible level of cooperation;

A society supported by Artificial Intelligence and robotics

4.  Recalls the fact that Europe has a world-leading AI research community, which accounts for 32 % of AI research institutions worldwide;

5.  Underlines the fact that EU researchers continue to earn significantly less than their counterparts in the US and China and that it is known as the primary reason for them to leave Europe; calls on the Commission and the Member States to focus on attracting top talent to European companies, and on the Member States to create attractive conditions;

6.  Stresses the importance of targeted measures to ensure that small and medium-sized enterprises and start-ups are able to adopt and benefit from AI technologies; believes that impact assessments of the effects of new EU legislation on the technological development of AI should be mandatory and that such impact assessments should also be considered at national level;

7.  Calls on the Commission, in order to foster a regulatory environment that is favourable to the development of AI and in line with the principle of better regulation, to regularly re-evaluate the current legislation in order to ensure that it is fit for purpose regarding AI, while respecting EU fundamental values, and to seek to amend or substitute new proposals where this is shown not to be the case;

8.  Recommends further work to increase connectivity in neglected areas, such as rural areas, and neglected sectors, in order to have a genuinely inclusive digital transition;

9.  Highlights the fact that the wider introduction of AI technologies may replace certain professions, in particular those that are most likely to be automated, and create new ones; recalls that a significant proportion of the EU population – 37 % of the labour force – does not have basic digital skills; underlines that the Commission forecasts a significant shortage of ICT experts, predicting 750 000 job vacancies by 2020;

10.  Urges Member States to modernise their professional training and education systems in order to take into account scientific progress and developments in the field of AI, in line with the Proportionality Test Directive and the Professional Qualifications Directive, and to make EU professional services globally competitive in the coming decades;

The technological path towards artificial intelligence and robotics

11.  Believes that the new set of rules governing the free flow of non-personal data in the Union allows for more and more data to become available for data-driven innovation, making it easier for SMEs and start-ups to develop innovative AI-enabled services and to enter new markets, while allowing citizens and businesses to benefit from better products and services;

12.  Recalls that the Regulation on the free flow of non-personal data states that if technological developments make it possible to turn anonymised data into personal data, such data are to be treated as personal data, and Regulation (EU) 2016/679 is to apply accordingly;

13.  Acknowledges the potential of data sharing, in the context of the future development of deep learning, in particular as regards high value datasets, given their significant civic or socio-economic benefits and suitability for the creation of value-added services and applications;

14.  Underlines the importance of open non-personal data from public and private sources and believes that a regulatory framework, including the re-use of public sector information should support access to open data and interoperability; urges the Commission to consider establishing a secure and voluntary single gateway for facilitating the exchange of non-personal data on the EU level, in order to create simplified data collection;

15.  Underlines that consumer trust is essential to the development of AI and that AI-based systems deal with more and more consumer data, which makes them prime targets for cyber-attacks; also highlights the fact that AI has to function in a way that is not detrimental to citizens and consumers and considers that the integrity of the data and algorithms it relies on must therefore be secured;

16.  Stresses the importance of recognising, identifying and monitoring disruptive developments in and around the development of AI; encourages research into AI to also focus on the detection of accidentally or maliciously corrupted AI and robotics;

Industrial policy

17.  Considers that the Union should be better equipped against cyber-attacks; encourages the swift completion of the Strategy for Cybersecurity that should ensure the development and deployment of safe AI and robotics systems that are resilient to cyber-attacks, notably via the development of EU certification schemes; believes that ENISA should prepare an action plan on cybersecurity in the area of AI, which should assess and address threats and weaknesses specific to AI;

18.  Underlines that the development of a strong and competitive AI and Robotics industry cannot rely solely on public funds; highlights the importance of mobilising private funds to foster innovation and to achieve a developed AI and robotics industry in the EU;

19.  Recalls that while AI and robotics already have long-established industrial applications, advancements in the field are expanding and providing wide and diverse applications in all human activities; believes that any regulatory framework must include flexibility that allows for innovation and free development of new technologies and uses for AI; underlines that regulation should not hinder research and development in the private sector;

20.  Calls on the Commission to ensure that Digital Innovation Hubs do not lead to the creation of additional administration layers but focus on accelerating investments in projects that have proven their efficiency; calls on the Commission to focus on grants and financial assistance towards private R&D projects; underlines the need for greater use of public-private partnerships in AI;

21.  Believes that AI will be a great asset in terms of implementing the ‘once only’ principle, enabling the combination of databases and information from different sources, and thereby facilitating citizens’ interactions with public administrations;

Legal framework for artificial intelligence and robotics

An internal market for artificial intelligence

22.  Considers that Member States should ensure the creation of data based on the principle of ‘open by design and by default’, while ensuring a consistent level of protection of public interest objectives, such as public security or personal data protection, including where sensitive information related to critical infrastructure is concerned;

Responsible development and free movement of smart goods

23.  Underlines the importance of the mutual recognition principle in the cross-border use of smart goods, including robots and robotic systems; recalls that, when necessary, testing, certification and product safety should ensure that certain goods are safe by design and by default; notes in this context the importance of also working on the ethical aspects of AI;

24.  Stresses that AI applies to a variety of sectors where standardisation is of high relevance, such as smart manufacturing, robots, autonomous cars, virtual reality, healthcare and data analysis, and believes that EU-wide standardisation for AI will foster innovation and guarantee a high level of consumer protection; acknowledges that whereas a significant number of standards on issues such as safety, reliability, interoperability and security exist, further promotion and development of common standards for robotics and AI is necessary and should be part of the Union’s priorities; calls on the Commission, in cooperation with EU standardisation bodies, to continue to engage proactively with international standardisation bodies on improving standards in this field;

25.  Urges the Commission and the Member States to promote small-scale live testing of innovative products and technologies (so-called sand-box innovation), before such products are placed on the market;

26.  Recognises that robotics and AI technologies are increasingly used in autonomous vehicles, such as autonomous cars and civilian drones; notes that some Member States are already enacting or considering legislation in this area in particular, which could result in a patchwork of national legislation hampering the development of autonomous vehicles; calls, therefore, for a single set of Union rules that strikes the right balance between the interests of and potential risks for users, businesses and other concerned parties, while avoiding over-regulation in robotics and AI systems;

Free provision of AI-driven services

27.  Recalls that there is a continuous evolution in technologies for analysis, exploitation and processing of data, such as machine learning, AI and the Internet of Things, and that rapid technological evolution makes it possible to create new services and new applications, which are built upon the use, aggregation or combination of data;

28.  Recalls that many policy aspects relevant for AI-enabled services, including rules on consumer protection and policy on ethics and liability, are covered by the existing regulatory framework on services, namely the Services Directive, Professional Qualifications Directive and e-Commerce Directive;

29.  Underlines that humans must always be ultimately responsible for decision-making, especially for professional services such as the medical, legal and accounting professions; considers that reflection is needed on whether supervision by a qualified professional is necessary, with a view to protecting legitimate public interest objectives and providing high quality services;

30.  Recognises the importance of improved digital services such as virtual assistants, chatbots or virtual agents, bringing unprecedented operational efficiencies, while duly acknowledging the need to develop a human-centric and market-driven AI to produce better and more reliable decisions in view of the limits to the autonomy of AI and robotics;

Consumer protection and empowering consumers

31.  Considers that AI technologies developed for both manufacturing and individual use should be subject to product safety checks by market surveillance authorities and consumer protection rules ensuring, where appropriate, minimum safety standards and addressing the risk of accidents resulting from interaction with, or working in proximity to, humans; believes that ethical issues and issues of data protection, including third-party and personal data, civil liability and cybersecurity, should be considered in any policy on AI;

32.  Recalls that algorithms are the cornerstones of AI and robotics; calls on the Member States to ensure that only high quality data is used for AI, and in particular that it is up to date, accurate and reliable, in order to ensure consumer trust and acceptance; recalls in this regard the importance of explainability of decision-making processes of AI technologies and unbiasedness of algorithms, while respecting EU law in particular on trade secrets, and of creating review structures to remedy possible mistakes in decisions taken by AI;

33.  Stresses the importance of user-friendly AI in fostering consumer trust; believes that where final and permanent decisions are made by AI and not by humans, consumers should be made aware of this and be able to seek human review and redress; stresses that when interacting with an automated system, users should be informed about how to reach a human and how to ensure that a system’s decisions can be checked and corrected;

34.  Takes note of the Commission staff working document of 25 April 2018 on liability for emerging digital technologies (SWD(2018)0137); notes its conclusion that while the current liability framework is still stable and functioning, it should be reviewed by the expert group on liability in light of new AI technologies, in order to ensure it is fit for purpose and includes the perspectives of the consumer, innovators and companies operating in the EU;

35.  Stresses the need to educate citizens, and especially younger citizens, to understand how AI works; calls for greater empowerment of EU citizens and consumers by making AI more available to citizens, consumers and civil society; believes that this can be achieved by offering intensified and widespread education for a better understanding of AI.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Simplified procedure - date of decision

16.5.2018

Date adopted

4.12.2018

 

 

 

OPINION OF THE COMMITTEE ON LEGAL AFFAIRS ()

European Parliament

2014-2019

 

Committee on Legal Affairs

The Chair

12.12.2018

Mr Jerzy Buzek

Chair

Committee on Industry, Research and Energy

BRUSSELS

Subject:  Opinion on a comprehensive European industrial policy on artificial intelligence and robotics (2018/2088(INI))

Dear Mr Chair,

Under the procedure referred to above, the Committee on Legal Affairs, which I have the honour to chair, has been asked to submit an opinion to your committee. Mrs Mady Delvaux was appointed rapporteur for opinion. At its meeting of 20 November 2018, the committee decided to send the opinion in the form of a letter.

In accordance with the agreement reached on association under Rule 54 of the Rules of Procedure, the Committee on Legal Affairs holds exclusive competence for questions relating to liability and intellectual property rights and it holds shared competence for questions relating to embedded values in technology (“ethical-by-design”).

The Committee on Legal Affairs considered the matter at its meeting of 6 December 2018 and following exchanges between the coordinators via written procedure the opinion below was adopted at the meeting on 10 December 2018. At that meeting[1], it decided to call on the Committee on Industry, Research and Energy, as the committee responsible, to incorporate the following suggestions into its motion for a resolution.

I trust that this will make a useful contribution into the report drafted by your Committee.

Yours sincerely,

Pavel Svoboda

SUGGESTIONS

A.  whereas its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics (2015/2013(INL)) called on the Commission to propose a coherent legal framework regarding the development of robotics, including autonomous systems and smart autonomous robots;

1.  Urges that great attention has to be paid to the safety of products to ensure protection and trust of consumers in AI and robotics;

2.  Stresses that human-centric design when developing AI is crucial in order to ensure responsibility and accountability of decision-making process made by the machines;

3.  Stresses that the policy framework must be designed to encourage the development of all kinds of AI and not only deep learning systems, which requires a huge amount of data;

4.  Stresses that the use of data in health sector has to be monitored carefully and ethically and must not in any way obstruct the access to social protection or insurance;

5.  Believes that when AI is being used in implanted medical devices, the bearer should have the right to inspect and modify the source code used in the device;

6.  Recalls that the availability of quality data is essential for real competitiveness in the AI industry, and calls for public authorities to ensure ways of producing, sharing and governing data by making data a common good as open data;

7.  Welcomes the initiative of the Commission to create the expert group on Liability and New technologies to provide the EU with expertise on the applicability of the Product Liability Directive to traditional products, new technologies and new societal challenges (Product Liability Directive formation) and assist the EU in developing principles that can serve as guidelines for possible adaptations of applicable laws at EU and national level relating to new technologies (New Technologies formation);

8.  Regrets however that no legislative proposal was put forward during this legislature delaying the update of the liability rules at EU level and threatening the legal certainty across EU in this area for both traders and consumers;

9.  Notes the ongoing work of the International Organisation for Standardisation (ISO) on AI and urges the Member States to coordinate their ISO members to defend as best as possible the European values and interests in the development of standards;

10.  Recalls its above-mentioned resolution of 16 February 2018 in which it noted that there are no legal provisions that specifically apply to robotics, but that existing legal regimes and doctrines can be readily applied to robotics, although some aspects appear to call for specific consideration; renews the call made in that resolution on the Commission to support a horizontal and technologically neutral approach to intellectual property applicable to the various sectors in which robotics could be employed;

11.  Welcomes in this regard the communication from the Commission to the Institutions on Guidance on certain aspects of Directive 2004/48/EC of the European Parliament and of the Council on the enforcement of intellectual property rights but underlines the need to monitor the relevance and efficiency of rules on intellectual property rights to govern the development of AI; Underlines, in this context, the importance of fitness checks;

12.  Welcomes that the Commission has set up the High-Level Expert Group on Artificial Intelligence, comprising representatives from academia, civil society and industry to draft AI ethics guidelines on issues such as fairness, safety, transparency, the future of work, democracy and more broadly the impact on the application of the Charter of Fundamental Rights, including privacy and personal data protection, dignity, consumer protection and non-discrimination;

13.  Looks forward to the joint meeting with Member States foreseen for January 2019 following the publication of the draft ethical guidelines at the end of 2018 as announced in the communication from the Commission on 25 April 2018;

14.  Underlines the role the Parliament could play through the work done by STOA to represent citizens’ concerns in drafting these guidelines;

15.  Takes note of the coordinated action plan on AI presented by Vice-Commissioner Ansip in December 2018;

16.  Stresses that the development of AI and robotics is currently led by non-EU countries, and therefore underlines the importance of having a European common approach to defend the place of the EU in the development of AI;

17.  Welcomes in that sense the creation of the EU AI alliance with the aim of full mobilisation of a diverse set of participants, including businesses, consumer organisations, trade unions, and other representatives of civil society bodies; notes that the goal of this broad multi-stakeholder platform is to complement and support the work of the AI High Level Expert Group, in particular in preparing draft AI ethics guidelines, and ensuring competitiveness of the EU in the burgeoning field of AI.

  • [1]  The following were present for the final vote: Pavel Svoboda (Chair), Jean‑Marie Cavada, (Vice-Chair), Mady Delvaux (Vice-Chair and rapporteur for opinion), Joëlle Bergeron, Kostas Chrysogonos, Sergio Gaetano Cofferati, Mary Honeyball, Sajjad Karim, Sylvia‑Yvonne Kaufmann, António Marinho e Pinto, Julia Reda, Evelyn Regner, Axel Voss, Tiemo Wölken, Francis Zammit Dimech, Tadeusz Zwiefka, Kosma Złotowski, Luis de Grandes Pascual.

OPINION of the Committee on Civil Liberties, Justice and Home Affairs (11.12.2018)

for the Committee on Industry, Research and Energy

on a comprehensive European industrial policy on artificial intelligence and robotics
(2018/2088(INI))

Rapporteur (*): Michał Boni

(*) Associated committee – Rule 54 of the Rules of Procedure

SUGGESTIONS

The Committee on Civil Liberties, Justice and Home Affairs calls on the Committee on Industry, Research and Energy, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

–  having regard to the communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions on Artificial Intelligence for Europe (COM(2018)0237),

–  having regard to its resolution of 16 February 2017 with recommendations to the Commission on Civil Law Rules on Robotics[1],

A.  whereas artificial intelligence (AI) is one of the strategic technologies for the 21st century, both globally and in Europe, bringing positive change for the European economy and enabling innovation, productivity, competitiveness and wellbeing;

B.  whereas the European framework for AI must be developed on a basis of full respect for fundamental rights as enshrined in the Charter of Fundamental Rights of the European Union, and in particular for the principles of data protection, privacy and security;

C.  whereas human-centric AI and machine learning constitute an advantage for European society and can help address some of the most pressing societal challenges with tangible benefits for citizens;

1.  Calls on the Commission and other competent institutions to collaborate closely with researchers representing different disciplines in order to identify, support and leverage the potential positive effects of innovative uses of AI, to investigate, prevent, and mitigate potential harmful effects of malicious or negligent uses of AI, and to develop tools, policies, and norms appropriate to ensure the ethical governance of AI applications; stresses the need to develop knowledge exchange programmes, including cross-border programmes, and to facilitate joint strategy development between civil society organisations; notes that best practices should be identified in research areas with more mature methods for addressing dual-use concerns, such as security and privacy, and that they should be applied to the area of AI;

2.  Highlights the fact that malicious or negligent use of AI could threaten digital security and physical as well as public safety, as it could be used to conduct large-scale, finely-targeted and highly efficient attacks on information society services and connected machinery, as well as disinformation campaigns, and generally diminish the right of individuals to self-determination; stresses that the malicious or negligent use of AI might also pose risks to democracy and fundamental rights;

3.  Emphasises that the right to respect for private life and the protection of personal data, as enshrined in Articles 7 and 8 of the Charter of Fundamental Rights and Article 16 of the Treaty on the Functioning of the European Union (TFEU), apply to all areas of robotics and to all the other areas in which AI may be used, including big data analytics, machine learning and the Internet of Things, and that the Union’s legal framework for data protection must be fully complied with in the area of AI systems; reiterates its support for the Commission’s ‘FinTech Action plan: For a more competitive and innovative European financial sector’, which highlights the role that regulation can play in facilitating technological innovation and better understanding of innovative business models and market developments, by allowing regulators to supervise firms under the Union’s legal framework;

4.  Calls on the Commission to ensure that any future EU regulatory framework on AI guarantees the privacy and confidentiality of communications, personal data protection, including the principles of lawfulness, fairness and transparency, data protection by design and default, purpose limitation, storage limitation, accuracy and data minimisation, in full compliance with Union data protection law, as well as security, personal safety and other fundamental rights, including the right to freedom of expression and information; considers that any future regulatory framework should be assessed by the Commission on a regular basis, taking account of new technological developments;

5.  Stresses that European standards for AI must be based on the principles of digital ethics, human dignity, respect for fundamental rights, data protection and security, aiming at incorporating these principles by design and thus contributing to building trust among users; emphasises the importance of capitalising on the EU’s potential for creating a strong infrastructure for AI systems rooted in high standards of data protection and respect for humans; stresses the need to systematically invest in developing the future expertise needed, particularly in training researchers and facilitating knowledge sharing between various disciplines - including applied ethics - specialising in AI technology, robotics, and related fields;

6.  Calls on the Commission to take a strong stance against the development and use of autonomous weapon systems powered by AI within any EU framework;

7.  Welcomes the Commission’s initiative to establish the European AI Alliance, to be tasked with developing comprehensive AI ethical guidelines and policy recommendations; highlights the importance of active participation in global alliances and AI fora to enable knowledge sharing and a comprehensive understanding of concerns relating to data privacy, transparency and security in developing and deploying AI systems; welcomes, in this regard, the work of the High-Level Expert Group of Artificial Intelligence (AI HLEG), which serves as the steering group for the work of the AI Alliance; calls on the Commission to ensure participation of data protection authorities in this high-level group, to complete the work in an open and transparent manner and to publish the guidelines; moreover, invites the Commission to continue working towards an EU-wide approach on AI which will be human-centric and human- governed, and to actively promote high EU standards in the field of AI at international level;

8.  Underlines that any AI system must be developed with respect for the principles of transparency and for algorithmic accountability allowing for human understanding of its actions; notes that in order to build trust in and enable the progress of AI, users must be aware of how their data, as well as other data and data inferred from their data, is used and must know when they are communicating or interacting with an AI system or with humans supported by an AI system; believes that this will contribute to better understanding and confidence among users; stresses that the intelligibility of decisions must be an EU standard in accordance with Articles 13, 14 and 15 of the GDPR; recalls that the GDPR already foresees a right to be informed about the logic involved in data processing; stresses that, according to Article 22 of the GDPR, individuals have the right to obtain human intervention when a decision based on automated processing significantly affects them;

9.  Highlights that the Commission, the European Data Protection Board, national data protection authorities and other independent supervisory authorities should henceforth play a crucial role in the promotion of transparency and due process, legal certainty in general and, more specifically, of concrete standards protecting fundamental rights and guarantees associated with the use of data processing and analytics; calls for closer collaboration among the authorities charged with overseeing or regulating conduct in the digital environment; calls for adequate funding and staffing of such authorities;

10.  Highlights the importance of tackling developer bias, and thus of the need for a diverse workforce in all branches of the IT sector, as well as safeguard mechanisms to avoid biases based on gender and age being embedded into AI systems;

11.  Highlights the importance of establishing a regulatory framework to control algorithms and their impact, including, therefore, the possibility of engaging independent auditors for algorithms (or even software watchdogs, or a regulator that can investigate AI automated decisions;

12.  Stresses the importance of the quality and accuracy, as well as the representative nature of data used in the development and deployment of algorithms, as their standard relies on the data used to train them; points out that even high-quality training data can lead to the perpetuation of existing discrimination and injustice when not used carefully and consciously; notes that the use of low-quality, outdated, incomplete or incorrect data at different stages of data processing may lead to poor predictions and assessments and in turn to bias, which can eventually result in infringements of the fundamental rights of individuals or purely incorrect conclusions or false outcomes; believes, therefore, that it is important in the age of big data to ensure that algorithms are trained on representative samples of high-quality data in order to achieve statistical parity; emphasises that even if accurate high-quality data is used, predictive analysis based on AI can only offer a statistical probability; recalls that under the GDPR, the further processing of personal data for statistical purposes, including AI training, may only result in aggregate data which cannot be reapplied to individuals;

13.  Calls on the Commission, the Member States and the data protection authorities to identify and take all possible measures to prevent or minimise algorithmic discrimination and bias and to develop a strong common ethical framework for the transparent processing of personal data and automated decision-making, to guide data usage and the enforcement of Union law;

14.  Stresses that, while encouraging progress for the benefit of society and the environment, AI research and other related activities should be conducted in accordance with the precautionary principle and fundamental rights; stresses that everyone involved in the development, implementation, dissemination and use of AI should consider and respect human dignity as well as the self-determination and wellbeing - both physical and psychological - of the individual and society at large, anticipate potential safety impacts, and take due precautions proportionate to the level of protection, including the prompt disclosure of factors that might endanger the public or the environment; underlines the need for a coordinated approach to ensure that risk-based evaluations are carried out frequently for AI systems and their components; highlights the importance of designing and enforcing a comprehensive set of auditing rules and guidelines for the development and deployment of AI systems, as well as safety and flagging mechanisms aimed at mitigating their infringement risks;

15.  Underlines that the following principles should be applied for overall strategies on AI and robotics:

a) Robots and artificial intelligence are multi-use tools. Robots and artificial intelligence should not be designed solely or primarily to kill or harm humans. Individual rights and fundamental freedoms must be guaranteed, in particular human integrity (physical and mental), human dignity and identity. The primacy of the human being over the sole interest of science or society is underlined;

b) Humans are responsible agents. Lawmakers should make sure that emerging technologies comply with existing laws and fundamental rights;

c) Robots and artificial intelligence are products that should be designed to be safe, secure and fit for purpose, in the same way as other products;

d) Robots and artificial intelligence are manufactured artefacts. They should not be designed in a deceptive way to exploit vulnerable users; instead, their machine nature should be transparent;

e) A person having legal responsibility for a robot or artificial intelligence system should be designated. In the event of gross negligence regarding safety and security, manufacturers should be held responsible even where user agreements include non-liability clauses;

16.  Emphasises the need to include the public in the AI development process; calls on the Commission, therefore, to publish any algorithms, tools and technology funded or co-funded by the public as open source;

17.  Highlights that where risks arise as an unavoidable and integral element of AI research, robust risk assessment and management protocols shall be developed and complied with, taking into account that the risk of harm shall be no greater than that encountered in ordinary life (i.e. people shall not be exposed to risks greater than or additional to those to which they are exposed in their normal lifestyles).

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

10.12.2018

 

 

 

Result of final vote

+:

–:

0:

30

0

1

Members present for the final vote

Martina Anderson, Monika Beňová, Michał Boni, Cornelia Ernst, Romeo Franz, Nathalie Griesbeck, Jussi Halla-aho, Monika Hohlmeier, Sophia in ‘t Veld, Dietmar Köster, Juan Fernando López Aguilar, Roberta Metsola, Claude Moraes, Péter Niedermüller, Ivari Padar, Giancarlo Scottà, Birgit Sippel, Csaba Sógor, Helga Stevens, Bodil Valero, Harald Vilimsky, Josef Weidenholzer

Substitutes present for the final vote

Marek Jurek, Jean Lambert, Angelika Mlinar, Maite Pagazaurtundúa Ruiz, Barbara Spinelli, Axel Voss

Substitutes under Rule 200(2) present for the final vote

Lucy Anderson, Margrete Auken, Anthea McIntyre

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

30

+

ALDE

Nathalie Griesbeck, Sophia in ‘t Veld, Angelika Mlinar, Maite Pagazaurtundúa Ruiz

ECR

Jussi Halla-aho, Marek Jurek, Anthea McIntyre, Helga Stevens

ENF

Giancarlo Scottà

GUE/NGL

Martina Anderson, Cornelia Ernst, Barbara Spinelli

PPE

Michał Boni, Monika Hohlmeier, Roberta Metsola, Csaba Sógor, Axel Voss

S&D

Lucy Anderson, Monika Beňová, Dietmar Köster, Juan Fernando López Aguilar, Claude Moraes, Péter Niedermüller, Ivari Padar, Birgit Sippel, Josef Weidenholzer

VERTS/ALE

Margrete Auken, Romeo Franz, Jean Lambert, Bodil Valero

0

-

 

 

1

0

ENF

Harald Vilimsky

Key to symbols:

+  :  in favour

-  :  against

0  :  abstention

OPINION of the Committee on the Environment, Public Health and Food Safety (21.11.2018)

for the Committee on Industry, Research and Energy

on a comprehensive European industrial policy on artificial intelligence and robotics
(2018/2088(INI))

Rapporteur: Bolesław G. Piecha

SUGGESTIONS

The Committee on the Environment, Public Health and Food Safety calls on the Committee on Industry, Research and Energy, as the committee responsible, to incorporate the following suggestions into its motion for a resolution:

A.  whereas artificial intelligence (AI) will become a technological, economic, social and psychological revolution, with obvious ethical repercussions, not only in Europe but also all over the world;

B.  whereas the increasing integration of robotics in human systems requires strong policy guidance on how to maximise the benefits and minimise the risks for society and ensure a safe, equitable development of artificial intelligence;

C.  whereas the market maturity of artificial intelligence represents, ethically and morally, the biggest revolution since the steam engine and the computer, one which needs to be discussed in depth at a social level;

D.  whereas, in industry and services associated with high technology, AI is key to turning Europe into a ‘start-up continent’ by exploiting the latest technologies to generate growth in Europe, in particular in the areas of health technology, healthcare services and programmes, drug discovery, robotic and robot-assisted surgery, treatment of chronic diseases, and medical imaging and records, as well as securing a sustainable environment and safe food production; whereas Europe is currently lagging behind North America and Asia in terms of research and patents in the field of artificial intelligence;

E.  whereas the development of AI technologies may help to improve the lives of people with chronic illnesses and disabilities and address social challenges such as our ageing population by making health technology more precise and effective in providing healthcare;

F.  whereas there is a broad catalogue of possible applications of AI and robotics in medical care, such as: managing medical records and data, performing repetitive jobs (analysing tests, X-rays, CT scans, data entry), treatment design, digital consultation (such as medical consultation based on personal medical history and common medical knowledge), virtual nurses, medication management, drug creation, precision medicine (as genetics and genomics look for mutations and links to disease from the information in DNA), health monitoring, healthcare system analysis, etc.;

G.  whereas accessibility does not mean the same services and appliances for all; whereas the accessibility of AI and robotics is based on inclusive planning and design; whereas the user’s needs, wishes and experiences need to be the starting point of the design;

H.  whereas there are strong ethical, psychological and legal concerns about the autonomy of robots, their obvious lack of human empathy and their impact on the doctor-patient relationship which have not yet been properly addressed at EU level, in particular as regards the protection of patients’ personal data, liability, and the new economic and employment relationships that will be brought about; whereas ‘autonomy’ as such can only by fully attributed to human beings; whereas there is a need for a robust legal and ethical framework for artificial intelligence;

I.  whereas the introduction of artificial intelligence in the area of health in particular must always be based on the ‘man operates machine’ principle of responsibility;

J.  whereas the risks of AI have not been studied enough;

1.  Underlines that any revolutionary technological progress, particularly in the field of AI and robotics, should serve human wellbeing;

2.  Stresses that AI systems and robots are innovative technological means to improve people’s lives, grow the economy, and address challenges pertaining to health, the environment, climate change and food safety, among many other aspects, with a human agent always being responsible for their use;

3.  Calls on the Commission and the Member States to draw up a fully comprehensive EU Action Plan aimed at creating a European ‘artificial intelligence ecosystem’ to fully exploit socio-economic development opportunities while thoroughly addressing issues of safety, security, prevention of harm and mitigation of risks, human moral responsibility, governance and regulation of AI and robots, environmental sustainability, and disposal of AI devices[1]; stresses that this ‘ecosystem’ should be open to the needs of very small enterprises (VSEs) and small and medium-sized enterprises (SMEs) and established in association with all relevant stakeholders, in particular patient groups, but also taking into account the concerns of other groups such as welfare associations, churches and faculties of philosophy;

4.  Special attention should be paid to the use of ‘big data’ in health with the aim of maximising the opportunities it can bring – such as improving the health of individual patients as well as the performance of Member States’ public health systems – without lowering ethical standards and without threatening the privacy or safety of citizens;

5.  Warns strongly against any attempt to equip machines employing artificial intelligence with some kind of ‘personality’, thereby exonerating producers and the people who operate them from responsibility;

6.  Stresses, however, that the existing system for the approval of medical devices may not be adequate for AI technologies; calls on the Commission to closely monitor progress on these technologies and to propose changes to the regulatory framework if necessary in order to establish the framework for determining the respective liability of the user (doctor/professional), the producer of the technological solution, and the healthcare facility offering the treatment; points out that legal liability for damage is a central issue in the health sector where the use of AI is concerned; stresses the need therefore to ensure that users will not be led invariably to back the diagnostic solution or treatment suggested by a technological instrument for fear of being sued for damages if, on the basis of their informed professional judgement, they were to reach conclusions that diverged even in part;

7.  Calls on the Commission and the Member States to allocate greater funding to AI and robotic research and encourage AI experts and European and non-European companies to create innovative jobs enhancing research on public health issues; maintains, however, that an overall employment strategy needs to go hand in hand with the advance of AI in the professional healthcare context;

8.  Calls on the Commission and the Member States to foster talent in the fields of artificial intelligence and robotics by taking practical measures (for example, setting up incubators and supporting start-ups) to increase the number of trained personnel, giving particular attention to the presence of women and to attracting students from other disciplines;

9.  Calls on the Council and the Commission to take all legal measures necessary to prohibit methods of euthanasia and selection by means of the increasing use of optimising human-machine interfaces in the human body;

10.  Calls on the Commission to allow access to grants under EU framework programmes and other forms of funding for research into AI;

11.  Calls on the Commission and the Member States to draw up far-reaching plans aimed at attracting key stakeholders and main players from the fields of IT, mathematics, physics, medical information technology, clinical psychology, bioengineering and pharmaceutics to open AI biomedical and bionic research centres all across Europe; calls on the Commission and the Member States, however, to take the risks of AI (such as their impact on the doctor-patient relationship and the dehumanisation of medical care) seriously and to support research into the risks and ethical concerns related to AI, which could help to determine whether and under what conditions the use of a given technological solution might be considered appropriate and in accordance with the fundamental principles of human dignity and equality and could hence be authorised;

12.  Calls on the Member States and the Commission to increase funding in health-related AI technologies in the public and private sectors; welcomes, in this context, the declaration of cooperation signed by 24 EU Member States and Norway with a view to boosting the impact of investments in AI at European level; calls on the Member States and the Commission to consider whether training programmes for medical and healthcare personnel should be updated and standardised on a Europe-wide basis so as to ensure high levels of expertise and a level playing field in the Member States as regards knowledge and use of the most advanced technological instruments in robotic surgery, biomedicine, and artificial intelligence-based biomedical imaging.

INFORMATION ON ADOPTION IN COMMITTEE ASKED FOR OPINION

Date adopted

20.11.2018

 

 

 

Result of final vote

+:

–:

0:

48

2

0

Members present for the final vote

Marco Affronte, Pilar Ayuso, Zoltán Balczó, Catherine Bearder, Ivo Belet, Biljana Borzan, Paul Brannen, Nessa Childers, Birgit Collin-Langen, Seb Dance, Mark Demesmaeker, Bas Eickhout, Francesc Gambús, Gerben-Jan Gerbrandy, Jens Gieseke, Julie Girling, Sylvie Goddyn, Françoise Grossetête, Benedek Jávor, Karin Kadenbach, Urszula Krupa, Giovanni La Via, Jo Leinen, Peter Liese, Valentinas Mazuronis, Susanne Melior, Miroslav Mikolášik, Massimo Paolucci, Gilles Pargneaux, Bolesław G. Piecha, John Procter, Julia Reid, Frédérique Ries, Annie Schreijer-Pierik, Adina-Ioana Vălean, Jadwiga Wiśniewska

Substitutes present for the final vote

Cristian-Silviu Buşoi, Nicola Caputo, Michel Dantin, Martin Häusling, Esther Herranz García, Gesine Meissner, Tilly Metz, Ulrike Müller, Sirpa Pietikäinen, Carlos Zorrinho

Substitutes under Rule 200(2) present for the final vote

Mercedes Bresso, Innocenzo Leontini, Olle Ludvigsson, Ana Miranda

FINAL VOTE BY ROLL CALL IN COMMITTEE ASKED FOR OPINION

48

+

ALDE

Catherine Bearder, Gerben Jan Gerbrandy, Valentinas Mazuronis, Gesine Meissner, Ulrike Müller, Frédérique Ries

ECR 

Mark Demesmaeker, Urszula Krupa, Bolesław G. Piecha, John Procter, Jadwiga Wiśniewska

EFDD

Sylvie Goddyn

NI

Zoltán Balczó

PPE

Pilar Ayuso, Ivo Belet, Cristian‑Silviu Buşoi, Birgit Collin‑Langen, Michel Dantin, Francesc Gambús, Jens Gieseke, Françoise Grossetête, Esther Herranz García, Giovanni La Via, Innocenzo Leontini, Peter Liese, Miroslav Mikolášik, Sirpa Pietikäinen, Annie Schreijer‑Pierik, Adina‑Ioana Vălean

S&D

Biljana Borzan, Paul Brannen, Mercedes Bresso, Nicola Caputo, Nessa Childers, Seb Dance, Karin Kadenbach, Jo Leinen, Olle Ludvigsson, Susanne Melior, Massimo Paolucci, Gilles Pargneaux, Carlos Zorrinho

VERTS/ALE

Marco Affronte, Bas Eickhout, Martin Häusling, Benedek Jávor, Tilly Metz, Ana Miranda

2

-

EFDD

Julia Reid

PPE

Julie Girling

0

0

 

 

Key to symbols:

+  :  in favour

-  :  against

0  :  abstention

  • [1]  See in particular the European Group on Ethics in Science and New Technologies statement on Artificial Intelligence, Robotics and ‘Autonomous’ Systems of March 2018.

INFORMATION ON ADOPTION IN COMMITTEE RESPONSIBLE

Date adopted

14.1.2019

 

 

 

Result of final vote

+:

–:

0:

49

1

4

Members present for the final vote

Bendt Bendtsen, Jonathan Bullock, Cristian-Silviu Buşoi, Jerzy Buzek, Edward Czesak, Jakop Dalunde, Fredrick Federley, Ashley Fox, Igor Gräzin, Theresa Griffin, András Gyürk, Rebecca Harms, Eva Kaili, Barbara Kappel, Seán Kelly, Jeppe Kofod, Jaromír Kohlíček, Peter Kouroumbashev, Zdzisław Krasnodębski, Miapetra Kumpula-Natri, Christelle Lechevalier, Paloma López Bermejo, Edouard Martin, Tilly Metz, Angelika Mlinar, Csaba Molnár, Nadine Morano, Angelika Niebler, Morten Helveg Petersen, Miroslav Poche, Carolina Punset, Julia Reda, Paul Rübig, Massimiliano Salini, Algirdas Saudargas, Sven Schulze, Neoklis Sylikiotis, Patrizia Toia, Evžen Tošenovský, Vladimir Urutchev, Kathleen Van Brempt, Henna Virkkunen, Flavio Zanonato, Carlos Zorrinho

Substitutes present for the final vote

Pilar Ayuso, Michał Boni, Gunnar Hökmark, Rupert Matthews, Clare Moody, Caroline Nagtegaal, Răzvan Popa, Dennis Radtke, Giancarlo Scottà, Davor Škrlec

FINAL VOTE BY ROLL CALL IN COMMITTEE RESPONSIBLE

49

+

ALDE

Fredrick Federley, Igor Gräzin, Angelika Mlinar, Caroline Nagtegaal, Morten Helveg Petersen, Carolina Punset

ECR

Edward Czesak, Ashley Fox, Zdzisław Krasnodębski, Rupert Matthews, Evžen Tošenovský

ENF

Barbara Kappel, Christelle Lechevalier, Giancarlo Scottà

PPE

Pilar Ayuso, Bendt Bendtsen, Michał Boni, Cristian-Silviu Buşoi, Jerzy Buzek, András Gyürk, Gunnar Hökmark, Seán Kelly, Nadine Morano, Angelika Niebler, Dennis Radtke, Paul Rübig, Massimiliano Salini, Algirdas Saudargas, Sven Schulze, Vladimir Urutchev, Henna Virkkunen

S&D

Theresa Griffin, Eva Kaili, Jeppe Kofod, Peter Kouroumbashev, Miapetra Kumpula-Natri, Csaba Molnár, Clare Moody, Miroslav Poche, Răzvan Popa, Patrizia Toia, Kathleen Van Brempt, Flavio Zanonato, Carlos Zorrinho

VERTS/ALE

Jakop Dalunde, Rebecca Harms, Tilly Metz, Julia Reda, Davor Škrlec

1

-

EFDD

Jonathan Bullock

4

0

GUE/NGL

Jaromír Kohlíček, Paloma López Bermejo, Neoklis Sylikiotis

S&D

Edouard Martin

Key to symbols:

+  :  in favour

-  :  against

0  :  abstention

Last updated: 31 January 2019
Legal notice - Privacy policy