FOIA Uncovers Part of U.K. Shadow Regulation on Search Engines and Copyright

Last month we wrote about the adoption of a new secret agreement between copyright holders and the major search engines, brokered by the U.K. Intellectual Property Office, aimed at making websites associated with copyright infringement less visible in search results. Since the agreement wasn’t publicly available, we simultaneously issued a request under the U.K.’s Freedom of Information Act (FOIA), asking for a copy of the text. Today we received it.1

The agreement requires search engines to:

expand efforts to more effectively use [copyright infringement] notices to demote domains demonstrated to be dedicated to infringement, and work collaboratively with rights holders to consider other technically reasonable, scalable avenues empirically demonstrated to help materially reduce the appearance of illegitimate sites in the top search rankings.

Even before this agreement, Google had already begun to factor the Digital Millennium Copyright Act notices issued against websites into its search ranking algorithm, and Google has confirmed to us that the agreement won’t cause it to do anything differently than it was already doing. However the difference is that its independent efforts to demote links associated with copyright infringement are now taking place under an explicit threat of government regulation if it doesn’t make good enough progress by 1 June.

It’s important to know, then, what amounts to “good enough”. And, wouldn’t you know it, we can’t answer that question because the metrics for measuring progress under the agreement were redacted. The response to our FOIA request explains:

Disclosure of this agreement in full would compromise its effectiveness in combatting copyright piracy, resulting in the perpetuation of commercial detriment of legitimate copyright holders.  In effect, disclosure of the exempt information could assist persons intent on circumventing an agreed anti-piracy measure.

The redacted information is also exempt under section 31(1)(a), which relates to the prevention or detection of crime.  Disclosure of the redacted information could compromise the effectiveness of the Code as a measure to prevent or reduce the likelihood of copyright theft.

What we can be sure of, though, is that the more search engines tighten the criteria that demote websites from the top rankings, the more legitimate websites will trigger a false positive against these criteria, and be unfairly demoted. The U.K. agreement actually recognizes this, stating:

A whitelist process would need to be created to exclude legitimate sites that could be caught within this lower threshold. For an agreed sample of searches using neutral queries in conjunction with artist or content name, the aggregate results should be as follows…

The remainder of that paragraph, though, was also redacted—and a footnote (we would guess added by the copyright lobbyists) notes “Any such process will need to include a mechanism for challenging entries which are not clearly legitimate websites”.

For a company that previously sustained a record 500m settlement for failing to kow-tow to the demands of rights holders, there are obvious reasons why Google has played along with this process so far. However it must be very careful that its acquiescence to this shadowy regulation doesn’t escalate into a series of capitulations to copyright holder demands. You can read the full text of the agreement that we obtained below.

• 1. We were not the only people to do make such an FOIA request. TJ McIntyre from Digital Rights Ireland also made on and received the same information in return.