Alston applauded the decision, commenting: "By applying universal human rights standards, this Dutch court is setting a standard that can be applied by courts elsewhere. The litigation and its outcome are likely to inspire activists in other countries to file similar legal challenges to address the risks of emerging digital welfare systems."
"Governments that have relied on data analytics to police access to social security -- such as those in the US, the UK, and Australia -- should heed the court's warning about the human rights risks involved in treating social security beneficiaries as perpetual suspects," the civil rights group added.
Dutch court rules AI benefits fraud detection system violates EU human rights
A Dutch court has ruled that an automated surveillance system using artificial intelligence (AI) to detect welfare fraud violates the European Convention on Human Rights, and has ordered the government to cease using it immediately. The judgement comes as governments around the world are ramping up use of AI in administering welfare benefits and other core services, and its implications are likely to be felt far beyond the Netherlands.
States worldwide are turning to technology to make the welfare state more efficient and mitigate welfare fraud. In the Netherlands, the state used a digital welfare fraud detection system called Systeem Risico Indicatie. The SyRI was a system that used personal data from different sources and uncovered fraud. In 2020, a Dutch court decided the SyRI legislation was unlawful because it did not comply with the right to privacy under the European Convention of Human Rights. This is among the first time a court invalidated a welfare fraud detection system for breaching the right to privacy. We analyze the judgment and its implications in a full paper; below are some of our main points.
In 2020, a Dutch court decided the SyRI legislation was unlawful because it did not comply with the right to privacy under the European Convention of Human Rights. According to the court, the SyRI system did not strike a fair balance between fraud detection and privacy. The court's most important reasons were the SyRI system was too opaque, it collected too much data and the purposes for collecting the data were not clear and specific enough.
We show in the paper the immediate effects of the judgment are limited. The judgment does not say much about fraud detection and automated decision-making in general. A court might approve a similar system if the government ensures more transparency.
Still, the SyRI judgment is important. The judgment reminds policymakers that fraud detection must happen in a way that respects data protection principles and human rights, such as the right to privacy. The judgment also confirms the importance of transparency about how personal data is used.
In a closely watched decision, a court in the Netherlands recently halted a welfare fraud detection system, ruling that it violates human rights. The decision is likely to bring closer scrutiny to these systems worldwide, although Americans have fewer legal protections than their European counterparts.
Abstract:In the transition to a data-driven society, organizations have introduced data-driven algorithms that often apply artificial intelligence. In this research, an ethical framework was developed to ensure robustness and completeness and to avoid and mitigate potential public uproar. We take a socio-technical perspective, i.e., view the algorithm embedded in an organization with infrastructure, rules, and procedures as one to-be-designed system. The framework consists of five ethical principles: beneficence, non-maleficence, autonomy, justice, and explicability. It can be used during the design for identification of relevant concerns. The framework has been validated by applying it to real-world fraud detection cases: Systeem Risico Indicatie (SyRI) of the Dutch government and the algorithm of the municipality of Amersfoort. The former is a controversial country-wide algorithm that was ultimately prohibited by court. The latter is an algorithm in development. In both cases, it proved effective in identifying all ethical risks. For SyRI, all concerns found in the media were also identified by the framework, mainly focused on transparency of the entire socio-technical system. For the municipality of Amersfoort, the framework highlighted risks regarding the amount of sensitive data and communication to and with the public, presenting a more thorough overview compared to the risks the media raised.Keywords: ethics; algorithms; artificial intelligence; responsible AI; fairness; ethical framework; fraud detection
The regulation of AI has also been a matter for the courts, as in the example of the Netherlands in February 2020. The District Court of the Hague took decisive action in terms of privacy rights in its ruling on the SyRI case (NJCM v. the Netherlands), concluding that the System Risk Indication System, a legal instrument that the Dutch government uses to detect fraud in areas such as benefits, allowances and taxes, violates article 8 of the European Convention on Human Rights (ECHR). This court ruling shows that AI systems need to embed effective safeguards against privacy and human rights intrusions.
2ff7e9595c
टिप्पणियां