2018-04-24: Why we need multiple web archives: the case of blog.reidreport.com

This story started in December, 2017 with Joy-Ann Reid (of MSNBC) apologizing for "insensitive LGBT blog posts" that she wrote on her blog many years ago when she was a morning radio talk show host in Florida.   This apology was, at least in some quarters, (begrudgingly) accepted.   Today's update was news that Reid and her lawyers had in December claimed that either her blog, and/or the Internet Archive's record of the blog had been hacked (Mediaite, The Intercept).  Later today, the Internet Archive issued a blog post to deny the claim that it was hacked, stating:

This past December, Reid’s lawyers contacted us, asking to have archives of the blog (blog.reidreport.com) taken down, stating that “fraudulent” posts were “inserted into legitimate content” in our archives of the blog. Her attorneys stated that they didn’t know if the alleged insertion happened on the original site or with our archives (Reid’s claim regarding the point of manipulation is still unclear to us).

...

At some point after our correspondence, a robots.txt exclusion request specific to the Wayback Machine was placed on the live blog. That request was automatically recognized and processed by the Wayback Machine and the blog archives were excluded, unbeknownst to us (the process is fully automated). The robots.txt exclusion from the web archive remains automatically in effect due to the presence of the request on the live blog.   

Checking the Internet Archive for robots.txt, we can see that on 2018-02-16 blog.reidreport.com had a standard robots.txt page that blocked the admin section of WordPress, but by 2018-02-21 they had a version that blocked all robots, and as of today (2018-04-24) they had a version that specifically blocked only the Internet Archive's crawler ("ia_archiver").  As of about 5pm EDT, the robots.txt file had been removed (probably because of the Internet Archive's blog post calling out the presence of the robots.txt; cf. a similar situation in 2013 with the Conservative Party in the UK), but it may take a while for the Internet Archive to register its absence.

2018-04-25 update: Thanks to Peter Sterne for pointing out that www.blog.reidreport.com/robots.txt still exists, even though blog.reidreport.com/robots.txt does not.  They technically can be two different URLs though the convention is for them to canonicalize to the same URL (which is what the Wayback Machine does).  HTTP session info provided below, but the summary is that robots.txt is still in effect and the need for other web archives is still paramount. 

% curl -i http://blog.reidreport.com/robots.txt HTTP/1.1 404 Not Found Server: nginx/1.12.2 Date: Tue, 24 Apr 2018 21:55:18 GMT Content-Type: text/html Transfer-Encoding: chunked Connection: keep-alive Vary: Accept-Encoding X-Endurance-Cache-Level: 2

Until the Internet Archive begins serving blog.reidreport.com again, this is a good time to remind everyone that there are web archives other than the Internet Archive.  The screen shot above shows the Memento Time Travel service, which searches about 26 public web archives.  In this case, it found mementos (i.e., captures of web pages) in five different web archives: Archive-It (a subsidiary of the Internet Archive), Bibliotheca Alexandrina (the Egyptian Web Archive), the National Library of Ireland, the archive.is on-demand archiving service, and the Library of Congress.  For a machine readable service, below I list the TimeMap (list of mementos) generated by our MemGator service; the details aren't important but it is the source of the URLs that will appear next.  

Beginning with the original tweets by @Jamie_Maz (2017-11-30 thread, 2018-04-18 thread), I scanned through the screen shots (no URLs were given) and looked for screen shots that had definitive datetimes (most images did not have them).  The datetimes are (with ones for which we have evidence in bold, and the ones that we inferred by matching text are maked with "(inferred)"):

2005-04-25

2005-07-16

2005-07-21

2006-01-20 (inferred)

2006-06-05

2006-06-13 (inferred)

2006-10-03

2006-12-23

2007-02-21

2008-07-04

2008-10-16

2009-01-15

(update: because of canonicalization errors, some of the URLs are not being excluded; see below)

Most of those dates are pretty early in web archiving times, when the Internet Archive was the only archive commonly available, and many (all?) of the mementos in other web archives were surely originally crawled by the Internet Archive, even if on a contract basis (e.g., for the Library of Congress).  Nonetheless, with multiple copies geographically and administratively dispersed throughout the globe, an adversary would have had to hack multiple web archives and alter their contents (cf. lockss.org), or have hacked the original site (blog.reidreport.com) approximately 12 years ago for adulterated pages to have been hosted at all the different web archives.  While both scenarios are technically possible, they are extraordinarily unlikely.  

While we don't know the totality of the hacking claims, we can offer three archived web pages, hosted at the Library of Congress web archive (webarchive.loc.gov), that corroborate at least some of the claims @Jamie_Maz.

2006-01-20

30/x Joy seemed very interested in Brokeback Mountain, but wouldn't watch it bc it featured two men hooking up.
 
She can't understand who is going to see it since she imagines everyone would be turned off by it. pic.twitter.com/ogBnPyDSUF

— Not a bot (@Jamie_Maz) April 18, 2018

Evidence for this tweet can be found at (approximately midway): http://webarchive.loc.gov/all/20060125004941/http://blog.reidreport.com/ 

2018-04-25 update: the above image is for the left-hand image in the tweet; the right-hand image in the tweet can be found about 1/3 of the way down at: https://web.archive.org/web/20070222030051/https://blog.reidreport.com/labels/Tim%20Hardaway.html

2006-06-05

19/x Joy - I am not a gay marriage supporter pic.twitter.com/wbf2QSkx7n

— Not a bot (@Jamie_Maz) April 18, 2018

Evidence for this tweet can be found at (approximately 2/3 down): http://webarchive.loc.gov/all/20060608144033/http://blog.reidreport.com/

2006-06-13

I'm not sure this evidence maps directly to one of tweets, but it fits the general theme of anti-Charlie Crist: http://webarchive.loc.gov/all/20060615134635/http://blog.reidreport.com/

This memento also exists at archive.is; it is a copy of the Internet Archive's copy but it is not blocked by robots.txt because it is in another archive: http://archive.is/20060615134635/http://blog.reidreport.com/

2006-10-03

10/x Of course there are even more posts about Charlie Crist.

For some strange reason Joy posts a link to an article that claims Crist was involved in gay sex parties with Mark Foley. pic.twitter.com/qNazjxYQ7K

— Not a bot (@Jamie_Maz) April 18, 2018

Evidence for this tweet can be found at (approximately midway): http://webarchive.loc.gov/all/20061010125903/http://blog.reidreport.com/

2008-10-16

7/x Joy calls Crist "Miss Charlie" and again declares his potential wedding to a women is a fraud and a "veep marketing strategy". pic.twitter.com/ZMCbEfURfn

— Not a bot (@Jamie_Maz) November 30, 2017

Evidence for this tweet can be found at (approximately 1/3 down): http://webarchive.loc.gov/all/20081018020856/http://blog.reidreport.com/ 

In summary, of the many examples that @Jamie_Maz provides, I can find five copies in the Library of Congress's web archive.  These crawls were probably performed on behalf of the Library of Congress by the Internet Archive (for election-based coverage); even though there are many different (and independent) web archives now, in 2006 the Internet Archive was pretty much the only game in town.  Even though these mementos are not independent observations, there is no plausible scenario for these copies to have been hacked in multiple web archives or at the original blog 10+ years ago.  There may be additional evidence in the other web archives, but I haven't exhaustively searched them.

We don't know the full details of what Reid's lawyers alleged, so perhaps there are details that we don't know.  But the analysis from the Internet Archive crawl engineers, plus evidence in separate web archives suggest that the claim has no merit.

The case of blog.reidreport.com is another example of why we need multiple web archives.  

--Michael

Thanks to Prof. Michele Weigle and John Berlin for bringing this issue to my attention and uncovering some of the examples.   

Memento TimeMap for blog.reidreport.com:

% curl -i https://memgator.cs.odu.edu/timemap/link/http://blog.reidreport.com/ HTTP/1.1 200 OK Access-Control-Allow-Origin: * Access-Control-Expose-Headers: Link, Location, X-Memento-Count, X-Generator Content-Type: application/link-format Date: Tue, 24 Apr 2018 20:39:32 GMT X-Generator: MemGator:1.0-rc7 X-Memento-Count: 174 Transfer-Encoding: chunked <http://blog.reidreport.com/>; rel="original", <https://memgator.cs.odu.edu/timemap/link/http://blog.reidreport.com/>; rel="self"; type="application/link-format", <http://webarchive.loc.gov/all/20051213063757/http://blog.reidreport.com/>; rel="first memento"; datetime="Tue, 13 Dec 2005 06:37:57 GMT", <http://webarchive.loc.gov/all/20051227071134/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 27 Dec 2005 07:11:34 GMT", <http://webarchive.loc.gov/all/20060109233824/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 09 Jan 2006 23:38:24 GMT", <http://webarchive.loc.gov/all/20060111221738/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 11 Jan 2006 22:17:38 GMT", <http://webarchive.loc.gov/all/20060113221754/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 13 Jan 2006 22:17:54 GMT", <http://webarchive.loc.gov/all/20060117040021/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 17 Jan 2006 04:00:21 GMT", <http://webarchive.loc.gov/all/20060125004941/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 25 Jan 2006 00:49:41 GMT", <http://webarchive.loc.gov/all/20060130212707/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 30 Jan 2006 21:27:07 GMT", <http://webarchive.loc.gov/all/20060207043723/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 07 Feb 2006 04:37:23 GMT", <http://webarchive.loc.gov/all/20060214021136/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 14 Feb 2006 02:11:36 GMT", <http://webarchive.loc.gov/all/20060602120119/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 02 Jun 2006 12:01:19 GMT", <http://webarchive.loc.gov/all/20060608144033/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 08 Jun 2006 14:40:33 GMT", <http://archive.is/20060615134635/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 15 Jun 2006 13:46:35 GMT", <http://webarchive.loc.gov/all/20060615134635/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 15 Jun 2006 13:46:35 GMT", <http://webarchive.loc.gov/all/20060929093509/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 29 Sep 2006 09:35:09 GMT", <http://webarchive.loc.gov/all/20061010125903/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 10 Oct 2006 12:59:03 GMT", <http://webarchive.loc.gov/all/20061019213357/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 19 Oct 2006 21:33:57 GMT", <http://webarchive.loc.gov/all/20061119124609/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 19 Nov 2006 12:46:09 GMT", <http://webarchive.loc.gov/all/20061219122832/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 19 Dec 2006 12:28:32 GMT", <http://webarchive.loc.gov/all/20070102040834/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 02 Jan 2007 04:08:34 GMT", <http://webarchive.loc.gov/all/20070114015213/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 14 Jan 2007 01:52:13 GMT", <http://webarchive.loc.gov/all/20070513093553/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 13 May 2007 09:35:53 GMT", <http://webarchive.loc.gov/all/20071217225456/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 17 Dec 2007 22:54:56 GMT", <http://webarchive.loc.gov/all/20080113230146/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 13 Jan 2008 23:01:46 GMT", <http://webarchive.loc.gov/all/20080214153440/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 14 Feb 2008 15:34:40 GMT", <http://webarchive.loc.gov/all/20080829145325/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 29 Aug 2008 14:53:25 GMT", <http://webarchive.loc.gov/all/20080904170937/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 04 Sep 2008 17:09:37 GMT", <http://webarchive.loc.gov/all/20080913110633/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 13 Sep 2008 11:06:33 GMT", <http://webarchive.loc.gov/all/20080922195742/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 22 Sep 2008 19:57:42 GMT", <http://webarchive.loc.gov/all/20080926154752/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 26 Sep 2008 15:47:52 GMT", <http://webarchive.loc.gov/all/20081002223753/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 02 Oct 2008 22:37:53 GMT", <http://webarchive.loc.gov/all/20081009210202/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 09 Oct 2008 21:02:02 GMT", <http://webarchive.loc.gov/all/20081018020856/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 18 Oct 2008 02:08:56 GMT", <http://webarchive.loc.gov/all/20081026032823/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 26 Oct 2008 03:28:23 GMT", <http://webarchive.loc.gov/all/20081101231444/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 01 Nov 2008 23:14:44 GMT", <http://webarchive.loc.gov/all/20081107190850/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 07 Nov 2008 19:08:50 GMT", <http://webarchive.loc.gov/all/20081114192933/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 14 Nov 2008 19:29:33 GMT", <http://webarchive.loc.gov/all/20081129222646/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 29 Nov 2008 22:26:46 GMT", <http://webarchive.loc.gov/all/20090807192202/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 07 Aug 2009 19:22:02 GMT", <http://webarchive.loc.gov/all/20090906034348/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 06 Sep 2009 03:43:48 GMT", <http://archive.is/20091123072635/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 23 Nov 2009 07:26:35 GMT", <http://wayback.archive-it.org/all/20091123072635/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 23 Nov 2009 07:26:35 GMT", <http://wayback.archive-it.org/all/20100608130917/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 08 Jun 2010 13:09:17 GMT", <http://archive.is/20100908150601/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 08 Sep 2010 15:06:01 GMT", <http://wayback.archive-it.org/all/20100908150601/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 08 Sep 2010 15:06:01 GMT", <http://webarchive.loc.gov/all/20101017180828/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 17 Oct 2010 18:08:28 GMT", <http://webarchive.loc.gov/all/20101021204435/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 21 Oct 2010 20:44:35 GMT", <http://archive.is/20101023143957/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 23 Oct 2010 14:39:57 GMT", <http://wayback.archive-it.org/all/20101023143957/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 23 Oct 2010 14:39:57 GMT", <http://webarchive.loc.gov/all/20101029010331/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 29 Oct 2010 01:03:31 GMT", <http://webarchive.loc.gov/all/20101104233918/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 04 Nov 2010 23:39:18 GMT", <http://webarchive.loc.gov/all/20101111205248/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 11 Nov 2010 20:52:48 GMT", <http://webarchive.loc.gov/all/20101118125239/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 18 Nov 2010 12:52:39 GMT", <http://webarchive.loc.gov/all/20101125130403/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 25 Nov 2010 13:04:03 GMT", <http://webarchive.loc.gov/all/20101202211357/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 02 Dec 2010 21:13:57 GMT", <http://archive.is/20101203223309/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 03 Dec 2010 22:33:09 GMT", <http://wayback.archive-it.org/all/20101203223309/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 03 Dec 2010 22:33:09 GMT", <http://webarchive.loc.gov/all/20101204130037/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 04 Dec 2010 13:00:37 GMT", <http://archive.is/20101210220416/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 10 Dec 2010 22:04:16 GMT", <http://wayback.archive-it.org/all/20101210220416/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 10 Dec 2010 22:04:16 GMT", <http://wayback.archive-it.org/all/20101218022503/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 18 Dec 2010 02:25:03 GMT", <http://wayback.archive-it.org/all/20101225011455/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 25 Dec 2010 01:14:55 GMT", <http://wayback.archive-it.org/all/20110101102929/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 01 Jan 2011 10:29:29 GMT", <http://webarchive.loc.gov/all/20110102124225/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 02 Jan 2011 12:42:25 GMT", <http://wayback.archive-it.org/all/20110110192123/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 10 Jan 2011 19:21:23 GMT", <http://wayback.archive-it.org/all/20110115141029/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 15 Jan 2011 14:10:29 GMT", <http://wayback.archive-it.org/all/20110129081021/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 29 Jan 2011 08:10:21 GMT", <http://wayback.archive-it.org/all/20110131235456/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 31 Jan 2011 23:54:56 GMT", <http://wayback.archive-it.org/all/20110202022338/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 02 Feb 2011 02:23:38 GMT", <http://wayback.archive-it.org/all/20110205153552/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 05 Feb 2011 15:35:52 GMT", <http://webarchive.loc.gov/all/20110208002106/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 08 Feb 2011 00:21:06 GMT", <http://webarchive.loc.gov/all/20110219173553/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 19 Feb 2011 17:35:53 GMT", <http://wayback.archive-it.org/all/20110304213316/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 04 Mar 2011 21:33:16 GMT", <http://webarchive.loc.gov/all/20110306074027/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 06 Mar 2011 07:40:27 GMT", <http://wayback.archive-it.org/all/20110307144706/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 07 Mar 2011 14:47:06 GMT", <http://webarchive.loc.gov/all/20110310140543/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 10 Mar 2011 14:05:43 GMT", <http://webarchive.loc.gov/all/20110311192705/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 11 Mar 2011 19:27:05 GMT", <http://wayback.archive-it.org/all/20110321170236/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 21 Mar 2011 17:02:36 GMT", <http://webarchive.loc.gov/all/20110324213816/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 24 Mar 2011 21:38:16 GMT", <http://wayback.archive-it.org/all/20110329053124/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 29 Mar 2011 05:31:24 GMT", <http://webarchive.loc.gov/all/20110330170039/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 30 Mar 2011 17:00:39 GMT", <http://webarchive.loc.gov/all/20110406223119/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 06 Apr 2011 22:31:19 GMT", <http://webarchive.loc.gov/all/20110414011942/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 14 Apr 2011 01:19:42 GMT", <http://wayback.archive-it.org/all/20110416100848/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 16 Apr 2011 10:08:48 GMT", <http://webarchive.loc.gov/all/20110420154544/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 20 Apr 2011 15:45:44 GMT", <http://webarchive.loc.gov/all/20110427201727/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 27 Apr 2011 20:17:27 GMT", <http://webarchive.loc.gov/all/20110504135920/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 04 May 2011 13:59:20 GMT", <http://webarchive.loc.gov/all/20110520045229/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 20 May 2011 04:52:29 GMT", <http://webarchive.loc.gov/all/20110527183951/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 27 May 2011 18:39:51 GMT", <http://webarchive.loc.gov/all/20110602135315/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 02 Jun 2011 13:53:15 GMT", <http://webarchive.loc.gov/all/20110608090012/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 08 Jun 2011 09:00:12 GMT", <http://wayback.archive-it.org/all/20110610113620/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 10 Jun 2011 11:36:20 GMT", <http://webarchive.loc.gov/all/20110615131117/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 15 Jun 2011 13:11:17 GMT", <http://webarchive.loc.gov/all/20110622113849/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 22 Jun 2011 11:38:49 GMT", <http://webarchive.loc.gov/all/20110702040134/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 02 Jul 2011 04:01:34 GMT", <http://webarchive.loc.gov/all/20110706231737/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 06 Jul 2011 23:17:37 GMT", <http://webarchive.loc.gov/all/20110713173024/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 13 Jul 2011 17:30:24 GMT", <http://webarchive.loc.gov/all/20110721092604/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 21 Jul 2011 09:26:04 GMT", <http://webarchive.loc.gov/all/20110728205032/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 28 Jul 2011 20:50:32 GMT", <http://wayback.vefsafn.is/wayback/20110729092410/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 29 Jul 2011 09:24:10 GMT", <http://webarchive.loc.gov/all/20110804054817/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 04 Aug 2011 05:48:17 GMT", <http://wayback.archive-it.org/all/20110805152639/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 05 Aug 2011 15:26:39 GMT", <http://webarchive.loc.gov/all/20110811051914/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 11 Aug 2011 05:19:14 GMT", <http://wayback.archive-it.org/all/20110811052415/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 11 Aug 2011 05:24:15 GMT", <http://webarchive.loc.gov/all/20110817225634/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 17 Aug 2011 22:56:34 GMT", <http://webarchive.loc.gov/all/20110824095445/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 24 Aug 2011 09:54:45 GMT", <http://wayback.archive-it.org/all/20110910220909/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 10 Sep 2011 22:09:09 GMT", <http://wayback.archive-it.org/all/20111127124934/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 27 Nov 2011 12:49:34 GMT", <http://wayback.vefsafn.is/wayback/20111128190833/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 28 Nov 2011 19:08:33 GMT", <http://webarchive.loc.gov/all/20120216191131/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 16 Feb 2012 19:11:31 GMT", <http://webarchive.loc.gov/all/20120810235103/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 10 Aug 2012 23:51:03 GMT", <http://webarchive.loc.gov/all/20120818051223/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 18 Aug 2012 05:12:23 GMT", <http://webarchive.loc.gov/all/20120824003655/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 24 Aug 2012 00:36:55 GMT", <http://webarchive.loc.gov/all/20120830031237/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 30 Aug 2012 03:12:37 GMT", <http://webarchive.loc.gov/all/20120905212620/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 05 Sep 2012 21:26:20 GMT", <http://webarchive.loc.gov/all/20120920043905/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 20 Sep 2012 04:39:05 GMT", <http://webarchive.loc.gov/all/20120928205435/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 28 Sep 2012 20:54:35 GMT", <http://webarchive.loc.gov/all/20121005090212/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 05 Oct 2012 09:02:12 GMT", <http://webarchive.loc.gov/all/20121012142652/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 12 Oct 2012 14:26:52 GMT", <http://webarchive.loc.gov/all/20121106214550/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 06 Nov 2012 21:45:50 GMT", <http://webarchive.loc.gov/all/20121113213424/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 13 Nov 2012 21:34:24 GMT", <http://webarchive.loc.gov/all/20121122035116/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 22 Nov 2012 03:51:16 GMT", <http://webarchive.loc.gov/all/20121128012655/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 28 Nov 2012 01:26:55 GMT", <http://webarchive.loc.gov/all/20121206073847/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 06 Dec 2012 07:38:47 GMT", <http://wayback.archive-it.org/all/20121208104825/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 08 Dec 2012 10:48:25 GMT", <http://webarchive.loc.gov/all/20121209112553/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 09 Dec 2012 11:25:53 GMT", <http://webarchive.loc.gov/all/20121212152112/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 12 Dec 2012 15:21:12 GMT", <http://webarchive.loc.gov/all/20121219201542/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 19 Dec 2012 20:15:42 GMT", <http://archive.is/20121222083528/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 22 Dec 2012 08:35:28 GMT", <http://webarchive.loc.gov/all/20121228062056/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 28 Dec 2012 06:20:56 GMT", <http://webarchive.loc.gov/all/20130103131928/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 03 Jan 2013 13:19:28 GMT", <http://archive.is/20130104121910/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 04 Jan 2013 12:19:10 GMT", <http://webarchive.loc.gov/all/20130105083857/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 05 Jan 2013 08:38:57 GMT", <http://webarchive.loc.gov/all/20130109094417/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 09 Jan 2013 09:44:17 GMT", <http://webarchive.loc.gov/all/20130116233957/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 16 Jan 2013 23:39:57 GMT", <http://webarchive.loc.gov/all/20130123222346/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 23 Jan 2013 22:23:46 GMT", <http://wayback.archive-it.org/all/20130308150801/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 08 Mar 2013 15:08:01 GMT", <http://webarchive.loc.gov/all/20130309023350/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 09 Mar 2013 02:33:50 GMT", <http://wayback.archive-it.org/all/20130420082637/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 20 Apr 2013 08:26:37 GMT", <http://wayback.archive-it.org/all/20130420090721/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 20 Apr 2013 09:07:21 GMT", <http://wayback.archive-it.org/all/20130420193756/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 20 Apr 2013 19:37:56 GMT", <http://wayback.archive-it.org/all/20130422073707/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 22 Apr 2013 07:37:07 GMT", <http://wayback.archive-it.org/all/20130608121808/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 08 Jun 2013 12:18:08 GMT", <http://wayback.archive-it.org/all/20130807093321/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 07 Aug 2013 09:33:21 GMT", <http://wayback.archive-it.org/all/20130908144236/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 08 Sep 2013 14:42:36 GMT", <http://wayback.archive-it.org/all/20130928001144/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 28 Sep 2013 00:11:44 GMT", <http://wayback.archive-it.org/all/20131019034011/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 19 Oct 2013 03:40:11 GMT", <http://wayback.archive-it.org/all/20131020005113/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 20 Oct 2013 00:51:13 GMT", <http://wayback.archive-it.org/all/20131020081955/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 20 Oct 2013 08:19:55 GMT", <http://wayback.archive-it.org/all/20131101001723/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 01 Nov 2013 00:17:23 GMT", <http://wayback.archive-it.org/all/20131208032237/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 08 Dec 2013 03:22:37 GMT", <http://wayback.archive-it.org/all/20131209191158/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 09 Dec 2013 19:11:58 GMT", <http://wayback.archive-it.org/all/20131220170105/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 20 Dec 2013 17:01:05 GMT", <http://wayback.archive-it.org/all/20131224221904/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 24 Dec 2013 22:19:04 GMT", <http://wayback.archive-it.org/all/20140104201727/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 04 Jan 2014 20:17:27 GMT", <http://wayback.archive-it.org/all/20140110101150/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 10 Jan 2014 10:11:50 GMT", <http://wayback.archive-it.org/all/20140125081153/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 25 Jan 2014 08:11:53 GMT", <http://wayback.archive-it.org/all/20140225000347/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 25 Feb 2014 00:03:47 GMT", <http://wayback.archive-it.org/all/20140308212113/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 08 Mar 2014 21:21:13 GMT", <http://wayback.archive-it.org/all/20140608121032/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 08 Jun 2014 12:10:32 GMT", <http://wayback.archive-it.org/all/20140909053110/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 09 Sep 2014 05:31:10 GMT", <http://wayback.archive-it.org/all/20150808054942/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 08 Aug 2015 05:49:42 GMT", <http://wayback.archive-it.org/all/20180216091405/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 16 Feb 2018 09:14:05 GMT", <http://wayback.archive-it.org/all/20180217235122/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 17 Feb 2018 23:51:22 GMT", <http://wayback.archive-it.org/all/20180218200012/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 18 Feb 2018 20:00:12 GMT", <http://wayback.archive-it.org/all/20180219203551/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 19 Feb 2018 20:35:51 GMT", <http://wayback.archive-it.org/all/20180220214848/http://blog.reidreport.com/>; rel="memento"; datetime="Tue, 20 Feb 2018 21:48:48 GMT", <http://wayback.archive-it.org/all/20180221220248/http://blog.reidreport.com/>; rel="memento"; datetime="Wed, 21 Feb 2018 22:02:48 GMT", <http://wayback.archive-it.org/all/20180222222322/http://blog.reidreport.com/>; rel="memento"; datetime="Thu, 22 Feb 2018 22:23:22 GMT", <http://wayback.archive-it.org/all/20180223195912/http://blog.reidreport.com/>; rel="memento"; datetime="Fri, 23 Feb 2018 19:59:12 GMT", <http://wayback.archive-it.org/all/20180224210358/http://blog.reidreport.com/>; rel="memento"; datetime="Sat, 24 Feb 2018 21:03:58 GMT", <http://wayback.archive-it.org/all/20180225185618/http://blog.reidreport.com/>; rel="memento"; datetime="Sun, 25 Feb 2018 18:56:18 GMT", <http://wayback.archive-it.org/all/20180226193717/http://blog.reidreport.com/>; rel="memento"; datetime="Mon, 26 Feb 2018 19:37:17 GMT", <http://wayback.archive-it.org/all/20180227193459/http://blog.reidreport.com/>; rel="last memento"; datetime="Tue, 27 Feb 2018 19:34:59 GMT", <https://memgator.cs.odu.edu/timemap/link/http://blog.reidreport.com/>; rel="timemap"; type="application/link-format", <https://memgator.cs.odu.edu/timemap/json/http://blog.reidreport.com/>; rel="timemap"; type="application/json", <https://memgator.cs.odu.edu/timemap/cdxj/http://blog.reidreport.com/>; rel="timemap"; type="application/cdxj+ors", <https://memgator.cs.odu.edu/timegate/http://blog.reidreport.com/>; rel="timegate"

2018-04-25 update: As noted above, Peter Sterne brought to my attention that the non-standard URL of www.blog.reidreport.com/robots.txt still exists (and is blocking "ia_archiver") even though the more standard blog.reidreport.com/robots.txt is 404. 

% curl -i http://www.blog.reidreport.com/robots.txt HTTP/1.1 200 OK X-Robots-Tag: noindex, nofollow Content-Type: text/plain; charset=UTF-8 Expires: Wed, 25 Apr 2018 13:52:11 GMT Date: Wed, 25 Apr 2018 13:52:11 GMT Cache-Control: private, max-age=86400 Last-Modified: Tue, 06 Mar 2018 16:05:16 GMT X-Content-Type-Options: nosniff X-XSS-Protection: 1; mode=block Server: GSE Accept-Ranges: none Vary: Accept-Encoding Transfer-Encoding: chunked User-agent: ia_archiver

Another 2018-04-25 update: The NYT has covered the story ("MSNBC Host Joy Reid Blames Hackers for Anti-Gay Blog Posts, but Questions Mount"), and there was an interview with Reid's computer security expert ("Should We Believe Joy Reid’s Blog Was Hacked? This Security Consultant Says We Should"), Jonathon Nichols.  

 I embed a statement from Nichols (released by Erik Wemple), and a tweet from Nichols clarifying that they were not suggesting that Wayback Machine's mementos were hacked, but rather the hacked blog was crawled by the Internet Archive.  

This is where it's important to note that there maybe a discrepancy between the posts that Nichols is concerned with and those that @Jamie_Maz surfaced.  There is (semi-)independent evidence of @Jamie_Maz's pages, with the ultimate implication that for those pages to have been the result of a hack, blog.reidreport.com would have had to been hacked as many as 12 years ago -- and for nobody to have noticed at the time.  

Reid (& Nichols) could always unblock the Internet Archive and share the evidence of the hack. 

Here's the statement of security consultant Jonathan Nichols regarding the claims of blog-hacking by MSNBC's Joy Reid. pic.twitter.com/wGAui8Mfa5

— ErikWemple (@ErikWemple) April 25, 2018

1) WayBack was hacked
2) Joy was hacked
3) we said "Yo! Does your hack look like our hack!?"
4) PS: That's literally the industry standard (I've personally done it plenty of times)
5) We THEN got new data that showed it wasn't a hack of any archive.

— Jonathan Nichols (@wvualphasoldier) April 25, 2018

Yet another 2018-04-25 update: Apparently there are some holes in the http vs. https canonicalization wrt robots.txt blockage, allowing some of posts to surface.  Here's an example (via @YanceyMc):
https://web.archive.org/web/20060225041734/https://blog.reidreport.com/2005/10/harriet-miers-and-lesbian-hair-check.html

that page was captured in 2012; here's a version captured in 2006 (page originally authored in 2005; according to blogger):https://t.co/hrXwyC9wGH

here's a copy of that copy in @archiveis:https://t.co/kRgDN7nY4f https://t.co/103n66HKe1

— Michael L. Nelson (@phonedude_mln) April 25, 2018

Also, @wvualphasoldier deleted his tweets then protected his account, so that's the reason the above embed no longer formats correctly. (2018-04-26 update: @wvualphasoldier has now unprotected his account, but his earlier tweets about Joy Reid are still deleted.)

Yet, Yet Another 2018-04-25 update:

Thanks to Prof. Weigle and Mat Kelly for providing examples of some of the URLs that are slipping through the robots.txt exclusion.

Here's one: https://web.archive.org/web/20060805055643/https://blog.reidreport.com

and another: https://web.archive.org/web/20050728132003/https://blog.reidreport.com:443/

Which has the following information that I thought I saw in the original @Jamie_Maz tweets but now I can't find it, so perhaps I'm misremembering.  It certainly fits the overall theme.  Edit: it's in this post:

4/x Joy writes a blog post about John Roberts’s son, links to some articles, and makes joking comments about him.

Again a post that serves no purpose and has nothing to do with politics or Republicans’ position on gay marriage. pic.twitter.com/iW5Wb1z1dX

— Not a bot (@Jamie_Maz) April 18, 2018

2018-04-26 update: I saw this last night but I'm adding it now.  In Erik Wemple's article "Is MSNBC’s Joy Reid the victim of malicious ‘screenshot manipulation’?", he links to a PDFs to Google and the Internet Archive from Reid's lawyers.  The letters do not have the attachments, but from the text of the letters I was able to infer some of the references they discuss, such as "...making sexual innuendos about Senator Orrin Hatch..." and "Things people say when they're on the Fox News Channel".  Those two quotes can be found at both:

http://webarchive.loc.gov/all/20060111221738/http://blog.reidreport.com/
http://web.archive.org/web/20060111221738/https://blog.reidreport.com/

Picking this apart further, the post "Things people say when they're on the Fox News Channel" is one of about 12 posts Reid made on January 10, 2006.  The above mementos at LC and IA were archived on January 11, 2006 (specifically, 2006-01-11T22:17:38Z).  Assuming the timestamp of "11:28" is EST (Reid was in Florida at the time), the difference between EST and GMT is 5 hours.  The interval between the posting time and the archiving time is pretty small, less than 30 hours:

2006-01-11T22:17:38Z archive time

2006-01-10T16:28:00Z posting time (I've assumed "00" for seconds)
-----------------------------
29 hours 49 minutes 38 seconds, or 1 day, 5 hours, 49 minutes, 38 seconds.

Call it 30 hours.  It's possible that the blog's timezone was GMT, so the interval would become 35 hours.  It's also possible the timezone was PST (Blogger was a Google service in 2006, so there's a good chance the machines were in CA), then the window shrinks to 27 hours.  But EST and 30 hours is a pretty good guess.

Blogger does allow you to change creation dates on blog posts, so it is possible to go into a blog "today" and author a post that was "created" in 2006.  But the Internet Archive saw what it saw on 2006-01-11T22:17:38Z, so if the blog was hacked and posts were backdated to appear as 2006-01-10, then a hacker would have to have logged into the site and posted content without Reid or her readers noticing.  The archived page shows six posts on January 11, 2006, 12 posts on January 10, and 11 posts on January 09.  Even if we accept the premise that some of those posts are fraudulent, it is clear that Reid was a prolific blogger and regularly interacted with the blog (history note: blogging was very popular before Twitter & Facebook all but replaced it some years later).  I don't know what kind of readership Reid's blog had in 2006, but at the very least she was interacting with it many times per day, and would have had occasion to notice posts that she did not author.

Reid's lawyers also mention the post "Best Love Life EVER: Celebrity Wife-Swap Edition", which is also available at the above linked archived pages.  It purports to be published on January 11, 2006 at 9:23am, the first post (of six) of the day. Repeating the same analysis as above:

2006-01-11T22:17:38Z archive time

2006-01-11T14:23:00Z posting time (I've assumed "00" for seconds)
-----------------------------
7 hours 54 minutes 38 seconds

This establishes a boundary of 8 hours between when the allegedly fraudulent post was purportedly made and when the page was archived.  The last post of the day was 4:51pm ("The hypothetical situation room").  Repeating the above analysis again:

2006-01-11T22:17:38Z archive time

2006-01-11T21:51:00Z last posting time (I've assumed "00" for seconds)
-----------------------------
0 hours 26 minutes 38 seconds

In other words, the window for the last time that Reid interacted with the blog and when the blog was archived is less than 30 minutes.  It's entirely possible that Reid pressed "publish" and did not look back at the blog, and just moved on to the next task.  It's also possible that an adversary logged into the blog, posted the content before the 2006-01-11T22:17:38Z archival time, and changed the creation date to be earlier in the morning, before the presumably legitimate content was published.  But also keep in mind that such an adversary would not know in advance the time that the Internet Archive would visit the page (IA's "save page now" did not exist in 2006).  

In summary, while we can't rule out an external adversary logging in and inserting fraudulent content right before archiving time, it would take an extraordinary set of circumstances for the content to appear before the page was archived and not be noticed by Reid (or brought to her attention by her readers).

(also, apologies if I've made datetime arithmetic or TZ conversion errors; corrections welcome)

2018-04-27 update: I've deduced why the number of comments for the top-level pages (at least the few that I've looked at) do not match some people's expectations.  In short, the html page and the corresponding javascript page that holds the index for which posts have what number of comments are out of sync.  In one example I looked at, the html page is crawled on 2006-01-11 and the js index is crawled on 2006-02-07.  When the js runs and can't find its post id in the js index, it assumes zero comments and prints the string "comments?".

I'm including a twitter thread here in lieu of a proper write-up. 

"Why aren't there contemporary comments?" is a common question re: #JoyReid archived pages (see below)

A: "Temporal Violations" between html (at https://t.co/3r8BvyNmXa) and javascript index for comments (at https://t.co/v4OfWpb0MO)

(thread w/ 1 example below) https://t.co/uhGHlEA2QS

— Michael L. Nelson (@phonedude_mln) April 28, 2018