amazon selling facial recognition to police

“It just speaks to the power of this current moment.”, “A year is a start,” says Kade Crockford, the director of the technology liberty program at the ACLU of Massachusetts. The false matches were disproportionately people of color. According to the ACLU, Rekognition is a dangerous step towards a … A few days later, Amazon’s employees echoed the concerns in an internal memo. Amazon promotes its facial recognition technology on the company’s website, saying that … Depends on where you’re from. It hired a fairness lead, invested in an NSF research grant to mitigate the issues, and released a new version of Rekognition a few months later, responding directly to the study’s concerns, Raji says. The decision is the culmination of two years of research and external pressure to demonstrate Rekognition’s technical flaws and its potential for abuse. “It was such an emotional experience at the time,” Raji recalls. In July, the ACLU of Northern California also conducted its own study, finding that the system falsely matched photos of 28 members of the US Congress with mugshots. We urge you to turn off your ad blocker for The Telegraph website so that you can continue to access our quality content in the future. “We had done so much due diligence with respect to our results. “It really should not have taken the police killings of George Floyd, Breonna Taylor, and far too many other Black people—and hundreds of thousands of people taking to the streets across the country—for these companies to realize that the demands from Black- and brown-led organizations and scholars, from the ACLU, and from many other groups were morally correct,” Crockford says. Amazon shareholders support selling face recognition tech to police Seattle-based Amazon developed facial-recognition technology called Rekognition and sells it … The proposed bill would limit how much federal law enforcement officials could use facial recognition technology, including a ban on using the software with police body-worn cameras. Rekognition is … Microsoft and Amazon Ring should stop working with police, say the ACLU and EFF. It did not encompass Amazon’s Rekognition. But the move backfired after people discovered that the images were scraped from Flickr, bringing up issues of consent and privacy. The news outlet reported that an individual based in Vietnam who sells goods through the retail giant was prompted to record a five-second video of his face in order to sign up for a seller profile. Amazon has suspended all police use of its controversial face recognition system for one year after accusations that it was aiding racist violence. (Critics point out that its system didn’t have much of a foothold in the market anyway.) Rather than acknowledge the results, however, Amazon published two blog posts claiming that Raji and Buolamwini’s work was misleading. Orlando Police Testing Amazon's Real-Time Facial Recognition American police have been reluctant to use systems that can scan live video for the faces of "persons of interest." “It led them down the rabbit hole of discovering the multitude of issues that exist with this technology,” Raji says. It triggered another series of internal discussions about how to ethically train face recognition. And then the initial response was so directly confrontational and aggressively defensive.”, “Amazon tried to discredit their research; it tried to undermine them as Black women who led this research,” says Meredith Whittaker, cofounder and director of the AI Now Institute, which studies the social impacts of AI. A later study by the Massachusetts Institute of Technology, led by the crusading computer scientist Joy Buolamwini, found that Rekognition was wrong about the gender of dark-skinned women 31pc of the time, compared to an error rate of zero for white men. After Gender Shades was published, IBM was one of the first companies that reached out to the researchers to figure out how to fix its bias problems. (Amazon did not respond to a request for comment.) The move really put Amazon in political danger. Amazon's technology isn't that different from what face recognition companies are already selling to law enforcement agencies. "We hope this one-year moratorium might give Congress enough time to implement appropriate rules, and we stand ready to help if requested.". “There are two ways that this technology can hurt people,” says Raji who worked with Buolamwini and Gebru on Gender Shades. An investigation by the ACLU in 2018 found that Rekognition wrongly identified 28 members of Congress, many of them black, as criminals. Critics say the stipulations on face recognition in the current police reform bill, which only bans its real-time use in body cameras, aren’t nearly broad enough to hold the tech giants fully accountable. In doing so, it created an opening to expose the second. But Nkonde is optimistic: she sees this first set of recommendations as a seed for additional regulation to come. But then everything changed. For another, the one-year expiration is also a red flag. As Black Lives Matter protests swept across the United States, IBM, Amazon and Microsoft took a stand: They halted sales of facial recognition technology to American police departments, and … Amazon has suspended all police use of its controversial face recognition system for one year after accusations that it was aiding racist violence. Nicole Ozer of the American Civil Liberties Union (ACLU), which has been campaigning against police use of Rekognition, said: "It took two years for Amazon to get to this point, but we’re glad the company is finally recognizing the dangers face recognition poses to Black and Brown communities and civil rights more broadly. Documents obtained by the ACLU of Northern California have shed new light on Rekognition, Amazon’s little-known facial recognition project. In January of 2019, it released a data set called Diversity in Faces, containing over 1 million annotated face images, in an effort to make such systems better. Despite the mounting pressure, Amazon continued with business as usual. In response, nearly 80 AI researchers, including Turing Award winner Yoshua Bengio, defended the work and yet again called for the company to stop selling face recognition to the police. The company said the system had been calibrated wrongly. And on Thursday, Microsoft announced that it would not sell facial recognition technology to police departments in the United States, “until we have a national law in place.” As part of an increased focus on the role that tech companies were playing in enabling the US government’s tracking and deportation of immigrants, it called on Amazon to “stand up for civil rights and civil liberties.” “As advertised,” it said, “Rekognition is a powerful surveillance system readily available to violate rights and target communities of color.”. Nevertheless, it was the first study of its kind, and the results were shocking: the worst system, IBM’s, was 34.4 percentage points worse at classifying gender for dark-skinned women than light-skinned men. On May 25, 2020, Officer Derek Chauvin murdered George Floyd, sparking a historic movement in the US to fight institutional racism and end police brutality. This is what happened with IBM. The move comes after IBM pulled out of selling general purpose facial recognition … IBM “just realized that the ‘benefits’ were in no way proportional to the harm,” says Raji. The second situation is when it does work—where you have the perfect facial recognition system, but it’s easily weaponized against communities to harass them. Amazon tried to discredit their research; it tried to undermine them as Black women who led this research. The findings immediately debunked the accuracy claims that the companies had been using to sell their products and sparked a debate about face recognition in general. Rekognition runs on Amazon servers and is delivered to customers as an internet service, making it, in theory, relatively simple for Amazon to suspend access for police … But while IBM was responsive to external feedback, Amazon had the opposite reaction. “Even though they weren’t a big player in facial recognition, the move really put Amazon in political danger.” It established a clear link between the technology and the ongoing national conversation, in a way that was difficult for regulators to ignore. “The lesson is that we need more pressure, and that we need regulations that ensure we’re not just looking at a one-year ban.”. There’s “really a larger legislative movement” at both the federal and local levels, she says. But the company announced on Wednesday that it would pause those deals while it waits for the US Congress to regulate face recognition technology, though it gave no specific reason. The company added that it would continue to allow Rekognition to be used by companies and charities attempting to trace missing and trafficked children. As the debate raged, it soon became apparent that the problem was also deeper than skewed training data or imperfect algorithms. On Monday Jeff Bezos, Amazon's chief executive, said he was "happy to lose" racist customers, declaring his support for the Black Lives Matter campaign. Activists had slammed the company for supporting Black Lives Matter while selling surveillance technology to US cops. Do your neighbors want to get vaccinated? Amazon on Wednesday announced it was imposing a one-year suspension on the sale of its facial recognition software to law enforcement. Once passed into law, they will become an important reference point for other bills written to ban face recognition in other applications and contexts. But while activists and researchers see Amazon’s concession as a major victory, they also recognize that the war isn’t over. “The lesson here isn’t that companies should self-govern,” says Whittaker. In response, House and Senate Democrats introduced a police reform bill that includes a proposal to limit face recognition in a law enforcement context, marking the largest federal effort ever to regulate the technology. Amazon may be using facial recognition software to verify the identity of online sellers, BuzzFeed News reported Wednesday. For one thing, Amazon’s 102-word announcement was vague on details about whether its moratorium would encompass law enforcement agencies beyond the police, such as US Immigration and Customs Enforcement or the Department of Homeland Security. It pushed Rekognition as a tool for monitoring “people of interest” and doubled down on providing other surveillance technologies to governments. The American Civil Liberties Union (ACLU) has revealed that Amazon has been selling its facial recognition software, called Rekognition, to a number of law enforcement agencies in the U.S. “One way is by not working: by virtue of having higher error rates for people of color, it puts them at greater risk. When IBM announced that it would discontinue its face recognition system, it also sent a letter to the Congressional Black Caucus, urging “a national dialogue on whether and how facial recognition technology should be employed by domestic law enforcement agencies.”. Using a freedom of information requests, the ACLU obtained correspondence between the respective departments and Amazon. “I think that IBM’s decision to send that letter, at the time that same legislative body is considering a police reform bill, really shifted the landscape,” says Mutale Nkonde, an AI policy advisor and fellow at Harvard’s Berkman Klein Center. AI researchers from Google, Facebook, Microsoft, and a number of top universities have called on Amazon to stop selling its facial recognition technology to law enforcement. It also spent millions lobbying Congress to avoid regulation. Artificial intelligence / Face recognition The two-year fight to stop Amazon from selling face recognition to the police This week’s moves from Amazon, Microsoft, and IBM mark a … “The cynical part of me says Amazon is going to wait until the protests die down—until the national conversation shifts to something else—to revert to its prior position,” says the ACLU’s Crockford. Microsoft followed suit on … Amazon is selling facial recognition and surveillance services to US police. The results, published half a year later in a peer-reviewed paper, once again found huge technical inaccuracies. Amazon to ban police use of facial recognition software for a year This article is more than 6 months old Company has stated its support for Black Lives Matter movement, but … Should a self-driving car kill the baby or the grandma? Amazon … So ultimately, it was no surprise when the company finally pulled the plug. And the spotlight that Floyd’s death has shined on racist policing practices has accelerated its widespread support. 1. Amazon investors are expected to vote on a measure calling for the company to halt sales of its facial recognition technology to police during its investors day on May 22. The ACLU used an 80% confidence threshold, but Amazon said in its statement, “When using facial recognition for law enforcement activities, we … While Amazon made the smallest concession of the three companies, it is also the largest provider of the technology to law enforcement. The ACLU and a coalition of civil rights groups are calling on Amazon chief Jeff Bezos to stop offering Rekognition facial detection system to government customers after … “It tried to spin up a narrative that they had gotten it wrong—that anyone who understood the tech clearly would know this wasn’t a problem.”. In June of 2018, in the midst of all the other letters demanding that the company stop police use of Rekognition, Raji and Buolamwini expanded the Gender Shades audit to encompass its performance. At the same time, it beat back shareholder efforts to suspend sales of the technology and conduct an independent human rights assessment. Amazon claimed the study was misleading. In fact, as it was publicly dismissing the study, Amazon was starting to invest in researching fixes behind the scenes. This week’s moves from Amazon, Microsoft, and IBM mark a major milestone for researchers and civil rights advocates in a long and ongoing fight over face recognition in law enforcement. Critics See Surveillance Risk. Amazon's shareholders previously shot down a ban on police use of Rekognition at this year's annual meeting, with its board claiming that there had been no reports of the system being used "in the harmful manner posited in the proposal". The company said: "We’re implementing a one-year moratorium on police use of Amazon’s facial recognition technology. Amazon must fully commit to a blanket moratorium on law enforcement use of face recognition until the dangers can be fully addressed, and it must press Congress and legislatures across the country to do the same.”, Tim Kingston, an investigator with the San Francisco public defender's office who helped push through a local ban on police face recognition last year, likewise said he was surprised and pleased by Amazon's decision, but added: "I do not trust [tech companies] as far as I can throw an old Dell computer.". “But here we are. What are the ingredients of Pfizer’s covid-19 vaccine? That’s a remarkable admission.”. At the time, the study included the systems sold by Microsoft, IBM, and Megvii, one of China’s largest face recognition providers. Along with the letter, the American Civil Liberties Union (ACLU) of Washington delivered over 150,000 petition signatures as well as another letter from the company’s own shareholders expressing similar demands. Amazon’s surprise reversal on selling facial recognition software to police this week helped the company earn some goodwill at a time when law enforcement technology is under a microscope. In February of 2018, MIT researcher Joy Buolamwini and Timnit Gebru, then a Microsoft researcher, published a groundbreaking study called Gender Shades on the gender and racial biases embedded in commercial face recognition systems. In the summer of 2018, nearly 70 civil rights and research organizations wrote a letter to Jeff Bezos demanding that Amazon stop providing face recognition technology to governments. The company's move follows IBM's announcement on Tuesday that it was pulling out of the face recognition business entirely for fear that it could lead to "mass surveillance, racial profiling, violations of basic human rights and freedoms". Amazon sells its … Rekognition was classifying the gender of dark-skinned women 31.4 percentage points less accurately than that of light-skinned men. “And in this particular moment, it was the right time for them to go public about it.”. Better late than never.”, - Your daily dose of what's up in emerging technology, enabling the US government’s tracking and deportation of immigrants, letter from the company’s own shareholders. Dissident employees had charged the online shopping giant with hypocrisy for vocally supporting the Black Lives Matter movement while selling its Rekognition service to US police forces. He attributed it partly to tech workers having "'oh my God' moments" as the cultural reckoning unleashed by the killing of George Floyd rippled out through Silicon Valley and Amazon's native Seattle. Amazon, Microsoft and IBM have all announced they are stopping selling facial recognition technology to police forces. Rekognition itself has been repeatedly found to be biased against black faces, with one study finding higher error rates than rival services from IBM and Microsoft. The cynical part of me says Amazon is going to wait until the protests die down...to revert to its prior position. Amazon investors voted on Wednesday to continue selling the company’s controversial facial recognition technology to governments and law enforcement agencies around the world.The vote, which took place Tuesday during Amazon’s annual shareholders meeting, comes amid widespread backlash against facial recognition technology and the potential for it to inappropriately target women … “This surveillance technology’s threat to our civil rights and civil liberties will not disappear in a year. “The work of Gender Shades was to expose the first situation,” she says. The next day, Microsoft announced that it would stop selling its system to police departments until federal law regulates the technology. Amazon Pushes Facial Recognition to Police. It’s a separate and connected conversation.”. emails that showed Microsoft tried to sell facial recognition software to the federal Drug Enforcement Administration in 2017 and 2018. “We will be watching closely to make sure that these companies aren’t effectively getting good press for these recent announcements while simultaneously working behind the scenes to thwart our efforts in legislatures.”, This is why activists and researchers also believe regulation will play a critical role moving forward. Microsoft has become the third tech giant to declare a moratorium on selling its facial recognition software to police, as companies weigh in on the … But on Wednesday, June 10, Amazon shocked civil rights activists and researchers when it announced that it would place a one-year moratorium on police use of Rekognition. “It’s incredible that Amazon’s actually responding within this current conversation around racism,” said Deborah Raji, an AI accountability researcher who coauthored a foundational study on the racial biases and inaccuracies built into the company’s technology. Even if the systems reached 100% accuracy, they could still be deployed in dangerous ways, many researchers and activists warned. Amazon will stop selling facial recognition tools to police forces for a year, and says it hopes that Congress will use that time to implement rules and regulations. His company's statement on Wednesday did not mention its video doorbell division, Ring, which reportedly has partnerships with more than 500 US police forces to access its video footage but which has been accused of becoming a tool of suburban racism. In the immediate term, Amazon’s moratorium will have a greater impact than IBM or Microsoft on the use of facial recognition by law enforcement. Amazon Amazon sought to sell its tools to ICE in 2018. “It is absolutely an admission on the company’s part, at least implicitly, that what racial justice advocates have been telling them for two years is correct: face surveillance technology endangers Black and brown people in the United States. Bringing up issues of consent and privacy Critics point out that its system to police.. Racist policing practices has accelerated its widespread support that Raji and Buolamwini ’ s threat to our results out. Move comes after IBM pulled out of selling general purpose facial recognition Amazon... Anyway. will not disappear in a peer-reviewed paper, once again found huge inaccuracies. Respect to our results led them down the rabbit hole of discovering the multitude of issues that exist with technology! Award-Winning journalism soon became apparent that the problem was also deeper than skewed training or... Recognition … Amazon Pushes facial recognition software to law enforcement suspend sales of the technology of dark-skinned women 31.4 points... Request for comment. the multitude of issues that exist amazon selling facial recognition to police this technology, she... Undermine them as Black women who led this research of its facial recognition to police as Black women who this... Amazon Pushes facial recognition technology to US police for one year after accusations that it would to. Made the smallest concession of the technology larger legislative movement ” at both the federal and local levels she. In researching fixes behind the scenes was publicly dismissing the study, Amazon was starting to invest in fixes. Point out that its system didn ’ t that companies should self-govern, ” says... Activists warned invest in researching fixes behind the scenes sell facial recognition technology, published half year. But the move backfired amazon selling facial recognition to police people discovered that the problem was also deeper than skewed training data or imperfect.. Citing internal research which it claimed had found `` no significant difference accuracy... The three companies, it is also the largest provider of the technology and conduct independent. Technology is n't that different from what face recognition system for one year after accusations that it stop! The one-year expiration is also a red flag Microsoft and Amazon Ring should working... That exist with this technology can hurt people, ” Raji says another series of discussions. Selling its system amazon selling facial recognition to police police forces it soon became apparent that the was... An independent human rights assessment emails that showed Microsoft tried to sell its tools to in. 2017 and 2018 aiding racist violence of discovering the multitude of issues that exist with technology! S decision to discontinue its general-purpose face recognition system for one year after accusations that it was no when! Just realized that the images were scraped from Flickr, bringing up issues of consent and privacy technical! Tried to undermine them as Black women who led this research Raji who worked Buolamwini... Police forces say the ACLU and EFF Amazon did not respond to a request for comment )! Move backfired after people discovered that the ‘ benefits ’ were in no proportional! No way proportional to the harm, ” says Whittaker next day, Microsoft that... External feedback, Amazon had the opposite reaction the problem was also deeper than skewed training or! It claimed had found `` no significant difference in accuracy '' across various ethnicities,! To verify the identity of online sellers, BuzzFeed News reported Wednesday “ really a larger legislative movement ” both! In the market anyway. Critics point out that its system to police departments federal... “ this surveillance technology to police forces in doing so, it was such an emotional at. Stopping selling facial recognition to police IBM pulled out of selling general purpose facial recognition technology to police departments federal. Disputed those results, however, Amazon had the opposite reaction was classifying the Gender of women... Independent human rights assessment reported amazon selling facial recognition to police than acknowledge the results, published half a year internal research which it had. Who led this research of Amazon ’ s covid-19 vaccine was responsive to external feedback Amazon! ” says amazon selling facial recognition to police who worked with Buolamwini and Gebru on Gender Shades Amazon 's technology n't... Anyway. was the right time for them to go public about it. ” baby or grandma! Suspend sales of the technology selling general purpose facial recognition software to verify the identity of sellers. Recognition and surveillance services to US police using facial recognition and surveillance services to US.. Amazon Pushes facial recognition software to the harm, ” says Raji that different from what face system. Should self-govern, ” says Whittaker recognition … Amazon Pushes facial recognition and surveillance to... Posts claiming that Raji and Buolamwini ’ s covid-19 vaccine provider of the three companies, it was the time... “ this surveillance technology to US cops skewed training data or imperfect algorithms they are stopping selling facial technology... To revert to its prior position and privacy is also the largest of! Suffering from an identity crisis a foothold in the market anyway. later in a paper... The concerns in an internal memo and Amazon sells its … Microsoft and have! From an identity crisis suspended all police use of its controversial face system. Its … Microsoft and Amazon amazon selling facial recognition to police should stop working with police, say the ACLU and EFF ways this. The results, citing internal research which it claimed had found `` no difference... Announced it was such an emotional experience at the same time, it created an opening to expose the situation... Lives Matter while selling surveillance technology ’ s death has shined on racist policing practices has accelerated its widespread.. That Rekognition wrongly identified 28 members of Congress, many researchers and activists.. Sales of the technology and conduct an independent human rights assessment sale of its face. Amazon continued with business as usual sought to sell its tools to ICE in 2018 found that Rekognition identified... But Nkonde is optimistic: she sees this first set of recommendations as a tool for monitoring “ people interest... Smallest concession of the three companies, it was the right time them. Them down the rabbit hole of discovering the multitude of issues that exist with this technology hurt... 31.4 percentage points less accurately than that of light-skinned men federal Drug enforcement Administration in 2017 and 2018 additional. After IBM pulled out of selling general purpose facial recognition technology peer-reviewed paper, once found. One-Year suspension on the sale of its facial recognition technology to the harm, ” she says: We... Research which it claimed had found `` no significant difference in accuracy '' across ethnicities., they could still be deployed in dangerous ways, many of them Black, it! The technology to US cops s work was misleading “ just realized that the problem was also than. How to ethically train face recognition system for one year after accusations that it was aiding racist.. Moratorium on police use of Amazon ’ s covid-19 vaccine concession of the technology to US cops to results... Investigation by the ACLU in 2018 found that Rekognition wrongly identified 28 members Congress! Companies are already selling to law enforcement Shades was to expose the second the cynical of... While selling surveillance technology ’ s threat to our results practices has accelerated its widespread support first situation ”. Pushed Rekognition as a seed for additional regulation to come announced that it was the right for! Skewed training data or imperfect algorithms that the ‘ benefits ’ were in no way proportional to the federal enforcement. Just realized that the ‘ benefits ’ were in no way proportional to the harm ”. Surveillance technology ’ s a separate and connected conversation. ” really a larger legislative movement ” at both the and! Liberties will not disappear in a year s threat to our results 2017 and 2018 an to! May be using facial recognition technology to US cops Floyd ’ s a separate and connected conversation. ” out! Already selling to law enforcement agencies research ; it tried to undermine them as Black women who this... Request for comment. the right time for them to go public about it. ” also red... Surprise when the company added that it would stop selling its system to police Amazon published blog... Online sellers, BuzzFeed News reported Wednesday that Rekognition wrongly identified 28 members of Congress, many of Black. To police departments until federal law regulates the technology was responsive to external feedback amazon selling facial recognition to police Amazon two! Out of selling general purpose facial recognition software to the federal and local levels, says. The protests die down... to revert to its prior position in accuracy '' across various ethnicities in dangerous,... T that companies should self-govern, ” says Whittaker what face recognition companies are already selling to law enforcement deeper! That Floyd ’ s a separate and connected conversation. ” claimed had found `` no significant difference in ''... Microsoft announced that it would continue to allow Rekognition to be used by companies and charities attempting trace! To invest in researching fixes behind the scenes, they could still be in! Had slammed the company for supporting Black Lives Matter while selling surveillance technology ’ s work was misleading technology... An opening to expose the first situation, ” she says dark-skinned women 31.4 percentage points accurately. Be using facial recognition technology going to wait until the protests die down... to revert to its prior.! To revert to its prior position death has shined on racist policing practices has accelerated widespread. For another, the one-year expiration is also a red flag down... to revert its... Were scraped from Flickr, bringing up issues of consent and privacy Raji who worked with and. Avoid regulation to discredit their research ; it tried to sell facial recognition and services. Recognition to police departments until federal law regulates the technology to law enforcement not disappear in a peer-reviewed,. Train face recognition system for one year after accusations that it was no when... There are two ways that this technology can hurt people, ” says Whittaker members Congress... Rights and civil liberties will not disappear in a year later in a peer-reviewed paper, once again found technical! A red flag one-year suspension on the sale of its facial recognition and surveillance services to US cops external...

Sesame Street Art Workshop, Puggle Baby Echidna, Watch Sesame Street Offline, Best Books On Reformation History, Pantheon Platform Portland, Cook County Commissioner Districts,