[Pending Profile Approval] The Emerging Legal Challenges for AI

The emerging legal challenges for AI

The blowback against AI is growing:

Thanks to the efforts of Rage Against the Machines’s Tom Morello and others led a campaign that forced some of the world’s largest music festivals —Coachella, Bonnaroo, and SXSW — to go on the record and state clearly that they have no plans to use facial recognition technology at their events and forced Ticketmaster, which had invested in facial recognition startup Blink Identity, to distance itself from the surveillance startup it had boasted about partnering with just a year earlier.

The social justice organization Mijente launched a campaign against Palantir alleging the US government used Palantir’s AI to facilitate hundreds of arrests of immigrants and the separation of immigrant families

A class action lawsuit claims that the State of Michigan’s Unemployment Insurance Agency deprived thousands of Michigan’s unemployment claimants from their health care because it administered unemployment insurance through automated programs that were defectively designed, implement, and/or maintained.  There have also been problems with AI systems employed for benefits screening in Indiana, USA and Australia.

As more governments and legal professionals and researchers embrace AI, they need also be aware of the potential dangers of placing blind faith in the impartiality, reliability and infallibility of AI.

AI systems are subject to both biases inherent in the algorithms employed - as different sets of engineers bring very different biases and assumptions to the creation of algorithms - and the data sets used. Given that different AI systems operate with different algorithms and, in many cases, on different datasets, it is not surprising that different AI systems can produce different results.

This observation was reinforced by researchers who compared the results of the same search entered into the same jurisdictional case databases of Casetext, Fastcase, Google Scholar, Lexis Advance, Ravel, and Westlaw and obtained widely divergent results.

Law-makers are starting to acknowledge the implications of AI biases. Under the EU’s General Data Protection Regulation (GDPR), companies operating in the EU will have to utilise algorithms that do not take into account characteristics such as gender, race or religion.

AI systems must also be able to correctly interpret users’ inputs. Though AI systems have made significant progress in understanding human language, there are still significant challenges to be overcome – especially where non-English languages are involved as a system must not only provide correct translations of individual words but also distinguish the meaning of compound words. For example, an AI may need to know that the Chinese word for ‘Canada” is a combination of the characters for “to add”, “to hold” and “big” (加拿大) that, when read sequentially, phonetically approximates the English word Canada and that the specific grouping of 加拿大 needs to be translated as ‘Canada’ rather than as individual component characters.

The system must also be able to correctly perceive the relationship between words, as this is key to understanding the entire meaning of a law or regulation. For instance, while the Chinese sentence 網下和網上投資者獲得配售後, 應當按時足額繳付認購資金⁷⁰- can be translated as: “After offline and online investors receive the placement, they should pay the subscribed-to funds on time and in full”, a semantic mistake made by a machine could confuse dependency of the clauses and translate this as requiring the funds to pay the investors once placement has been completed.

Unpredictability

AI systems can also be unpredictable; In October 2014, a bot tasked to buy random items from the web bought 10 pills of ecstasy from the dark web (while the Swiss police arrested the robot they released the programmers from any wrongdoing) and  Microsoft's TAY chatbot famously had to be shut down soon after its release because it began tweeting offensive comments.

Moreover, AI developers cannot always explain how their systems reach their conclusions - a research group at Mount Sinai Hospital in New York applied deep learning to the hospital's database of patient records, creating a program named Deep Patient. When tested on new records, Deep Patient proved proficient at predicting diseases. Without any expert instruction, it discovered patterns hidden in the hospital data that seemed to indicate when people were on the way to a wide range of ailments, including liver cancer and even schizophrenia. Its developers, however, had no idea how Deep Patient learned to do this.

Deep learning machines can self-reprogram to the point that even their programmers are unable to understand the internal logic behind AI decisions. In this context, it is difficult to detect hidden biases and to ascertain whether they are caused by a fault in the computer algorithm or by flawed datasets

Algorithmic transparency

There also have been calls for greater algorithmic transparency, i.e., to oblige companies to release some mandatory information on their AI algorithms, in order to detect potential bias.⁹⁹ In the US, these claims are usually confronted with the observation that algorithms have proprietary nature and are protected under trade secret law.¹⁰⁰ In the European Union, a first step towards transparency has been taken with the aforementioned General Data Protection Regulation. This instrument provides European citizens with a right of explanation, i.e., to be informed about the reasons behind any algorithmic decision affecting them.¹⁰¹ Again, this measure aims at avoiding automated decisions based on discriminatory parameters, such as race, gender or religion.¹⁰²

Others emphasize that transparency alone is inadequate to solve the problem of AI discrimination. Indeed, both the complexity of neural networks and the size of the datasets on which they are trained make AI internal logic inaccessible to human scrutiny.¹⁰³ Moreover, greater transparency neither solves the problems of data bias nor of the quality of the overall results returned by a legal AI.