Web Science and Digital Libraries Research Group

HTTP  and HTML are the reigning champs in terms of delivering content from the Web to your computer, typically though a Web browser. Content and data available over HTTP could generally be categorized to be mostly within the surface web. This, however, only constitutes a small portion of what content is available on the complete Web. Some protocols, such as FTP, are no longer supported by browsers and accessed by way of more specialized programs. Modern formats, such as IPFS , also exist but have limited adoption and often still require external software. While content available over these non-HTTP protocols is still "on the Net", the extent to which it is archived remains murky. A Trip Down Internet Lane Originally, I was inspired to write this blog post because I was exploring different representations of content on the web I remembered a gem from my younger days on the Internet. That gem was the ASCII Star Wars animation that you could watch over your terminal by ty...

Figure 1: Screenshot from TweetedAt service showing timestamp for a  deleted tweet from @SpeakerPelosi Memento Link:  http://web.archive.org/web/20170905173455/https:/twitter.com/nancypelosi?lang=da On  May 11, 2019,  Derek Willis from Politwoops shared a list of deleted tweet IDs  which could not be attributed to any Twitter handle followed by them. We tried multiple techniques to find the list of deleted tweet IDs in the web archives, but we were unsuccessful in finding any of the tweet IDs in web archives within the time range of our analysis. During our  investigation, we learned of Snowflake, a service used to generate unique IDs by Twitter. We used Snowflake to extract the timestamp from the deleted tweet IDs. Of  the 107 deleted tweet IDs shared with us only seven of them were in the time range of our initial analysis . In this post, we describe TweetedAt , a web service and library to extract the timestamps for post-Snowflake I...