Tracking the HTTP 451 Response Code

May 15, 2022

Since 2016, the HTTP 451 code has been an adopted standard (just like 404 not found or 500 internal server error) and should be returned when a server denies access to some content as a result of a legal request. This project uses the Censys internet scanning project to automatically perform searches for publicly accessible servers that are providing HTTP access and returning a response code of 451 gather daily statistics, and save copies of the fully HTTP response body.

You can view up-to-date data, visualizations, and additional details about the project at https://http451.info

This status code indicates that the server is denying access to the resource as a consequence of a legal demand.

The use of the 451 status code implies neither the existence nor nonexistence of the resource named in the request. That is to say, it is possible that if the legal demands were removed, a request for the resource still might not succeed. -- RFC 7725

Despite its name describe it as “world wide,” the Web is not, and perhaps never has been, truly global. The individual nation-state still matters and exercises its power within online spaces—even at the level of the protocol—to control where and how media can move. The Hypertext Transfer Protocol (HTTP) is the major technological backbone to the World Wide Web and describes the technical standards for computers to follow and exchange hypertext documents with each other. A recently adopted HTTP standard, the “451 – Unavailable for Legal Reasons” response code, shows that legal structures of the nation operate in online spaces and represents the continued restrictions on the flow of media through the Web. Though the extent of its actual implementation remains difficult to determine, the existence of the HTTP 451 status code represents the intertwined nature of law, technology, and cultural practices and prompts us to reconsider just how “worldwide” the World Wide Web is. I argue that the 451 code shows that the Web has not eliminated the significance of national borders and in fact has enabled entirely new fine-grained control over how media does and does not move. A secondary goal of this project is to introduce “full stack analysis” as a model of Web history research in which the protocols and technical underpinnings of the Web are confronted alongside the immediately apparent text to piece together a more expansive view of online media. By studying the implementation of HTTP 451 status codes across multiple levels of the Web stack, I show how law and regulation continue to operate online.

To find examples of current HTTP 451 implementations and to view the full HTTP conversation, I wrote a series of python scripts to automatically perform queries on the Censys dataset. Censys is an information security research project which performs daily port scans on the entire IPv4 address space. In (marginally more) simple terms, Censys queries every publicly accessible IP address and checks for common running services and indexes the results into a searchable database. The Censys data includes details on HTTP servers, including the response code and response body that they return. I use the Censys API to automatically perform searches, for publicly accessible servers that are providing HTTP access and returning a response code of 451 gather daily statistics, and save copies of the fully HTTP response body. These scripts are ongoing, and in addition to the examples I discuss below, up-to-date data is publicly accessible at this webpage. By collecting information from multiple layers of the Web stack, I am able to view both the page content that a user might view and details about the underlying protocol too.