FBI Wish List: An App That Can Recognize the Meaning of Your Tattoos

We’ve long known that the FBI is heavily invested in developing face recognition technology as a key component in its criminal investigations. But new records, obtained by EFF through a Freedom of Information Act (FOIA) lawsuit, show that’s not the only biometric marker the agency has its eyes on. The FBI’s wish list also includes image recognition technology and mobile devices to attempt to use tattoos to map out people’s relationships and identify their beliefs.

EFF began looking at tattoo recognition technology in 2015, after discovering that the National Institute for Standards & Technology (NIST), in collaboration with the FBI, was promoting experiments using tattoo images gathered involuntarily from prison inmates and arrestees. The agencies had provided a dataset of thousands of prisoner tattoos to some 19 outside groups, including companies and academic institutions, that are developing image recognition and biometric technology. Government officials instructed the groups to demonstrate how the technology could be used to identify people by their tattoos and match tattoos with similar imagery.

Our investigation found that NIST was targeting people who shared common beliefs, with a heavy emphasis on religious imagery.  NIST researchers, we discovered, had also bypassed basic oversight measures. Despite rigid requirements designed to protect prisoners who might be used as subjects in government research, the researchers failed to seek sign-off from the in-house watchdog before embarking on the project.

Following our report, NIST stopped responding to EFF’s FOIA requests. The agency also rushed to retroactively alter its documents to downplay the nature of the research. In a statement issued to the press, NIST denied our findings, claiming that its goal was simply to evaluate the effectiveness of tattoo recognition algorithms and “not about the many complex law enforcement policies or approaches that may be related to images of tattoos.”

This claim rings especially hollow now that the FBI has released email communications and slide presentations with NIST in response to our FOIA suit. 

Read the FBI records provided in response to our FOIA litigation. 

Among the records were two previously withheld presentations from FBI divisions focused on deciphering tattoos that were delivered at a special event organized by NIST for participants in the research project. 

What the FBI “Wants” and “Needs”

 While the FBI has long used a rudimentary system called TAG-IMAGE to use tattoos to identify people, according to the presentation, “It does NOT [sic] attempt to answer the question ‘What does it mean?’” Officials told the roomful of government, academic, and corporate researchers assembled by NIST that the FBI is seeking to establish “affiliation” using tattoo images.  This means identifying “gang membership, terrorist relevance, location, [and] symbols description/meaning.”

One particularly alarming presentation by the chief of the FBI’s Cryptanalysis & Racketeering Records Unit was titled, “Tattoo, Graffiti, and Symbol Recognition: A Codebreakers Perspective.” It began with a graphic of exclusively religious symbols, including Christian, Jewish, Hindu, and Taoist iconography.

 A subsequent slide was even more ominous. The headline read: “We want a one stop database to tell us what a symbol means.”

The presentation went onto explain that the FBI currently uses open-source resources to decipher the meanings of symbols and tattoos. These include the Anti-Defamation League’s Hate on Display database,  the University of Michigan’s Science Fiction and Fantasy “Dictionary of Symbolism,” and the U.S. Patent and Trademark Office’s website. 

The heavily redacted presentation included blacked-out slides that would have shown the extent of the FBI’s current efforts as well as several individual examples that posed the question, “What does this tattoo mean?”

Further in the presentation, the FBI further elaborated on its goals: “The Technology We Need: web accessible user populated database with instant i2i matching” (i2i stands for “image to image”). The slide included a photograph of a New York Police Department officer tinkering with a smartphone.

This technology is not science fiction: several years ago the Department of Homeland Security promoted a controversial program at Purdue University to crowdsource gang graffiti and tattoo images for a mobile recognition app. A prototype technology was deployed in Indiana and Illinois and introduced to the FBI in 2014. However, very little information is available about this pilot program after 2016.

Tattoos and Meaning

Most biometric technology, such as face, iris, or finger print recognition, is designed to establish the identity of otherwise unknown suspects or victims. Tattoo recognition is different in that not only can tattoos be used for identification, they can also reveal further information about the individual. For example a tattoo could say a lot about a person’s politics, religious beliefs, who their family members are, or even their favorite recording artist.

It’s this utility that raises civil liberties concerns, particularly in an age where law enforcement may be using tattoos to add individuals to gang databases or prioritize immigrants for deportation proceedings. For example, while the FBI says that the database could link members of a particular gang, it is also true that it could be used to compile a list of individuals who subscribe to a particular religion.

It’s also important to note the risk of erroneous interpretations of tattoos, whether by human or technology. A swastika tattoo, for example, could refer to the Neo-Nazi movement, but it’s also a symbol used by Native Americans. A six-pointed star is often associated with Judaism, but it can also signify a particular street gang.

In one high profile case, federal officials attempted to deport a Mexican man after accusing him of having a gang-affiliated tattoo. A judge found that claim to be unfounded [PDF] after prosecutors were unable to counter a well-respected gang expert’s testimony that he had “never seen a gang member with a similar tattoo nor would [he] attribute this tattoo to have any gang-related meaning.”

Even if a person has a gang tattoo, it does not mean they are part of that gang: often people who leave gangs cannot afford to have tattoos removed. In some cases, gang members have forcibly tattooed women against their will. 

NIST’s research itself also illustrates how dangerous this technology can be: none of the third parties were able to produce better than 15% accuracy in matching tattoos based on imagery. 

More Records on the Way

The new documents from the FBI begin to bare the truth about the agency’s tattoo recognition plans, however we are still waiting on the government to fully provide the records we requested. For example, EFF has demanded NIST and the FBI provide a list of the 19 companies and research institutions that received copies of the images collected from inmates.

Tattoo recognition technology is still in its early stages, but as we see an increased interest from federal agencies to use tattoos as an excuse to persecute immigrants, it’s more important than ever to expose this technology before it reaches maturation.