2019-09-10: Where did the archive go? Part 2: National Library of Ireland

In the previous post, we provided some details about changes in the archive Library and Archives Canada. After they upgraded their web archive replay system, we were no longer able to find 49 out of 351 mementos (archived web pages). In part 2 of this four part series, we focus on the movement of a collection from the National Library of Ireland (NLI).

In May 2018, we discovered that 979 mementos from the NLI collection that were originally archived at the European Archive (europarchive.org) were moved to the Internet Memory Foundation archive (internetmemory.org). Then in September 2018, we found that the collection of mementos had been moved to Archive-It (archive-it.org). We found that 192 mementos, out of 979, can not be found in Archive-It (i.e., missing mementos).

For example, the memento from the European Archive:

http://collection.europarchive.org/nli/20141013204117/http://www.defense.gov/

has been moved to the Internet Memory Foundation (IMF) archive at:

http://collections.internetmemory.org/nli/20141013204117/http://www.defense.gov/

before it ended up at Archive-it:

http://wayback.archive-it.org/10702/20141013204117/http://www.defense.gov/

The representations of the three mementos are illustrated in the figure below.

There were no changes in the 979 mementos (other than their URIs) when they moved from the European Archive to the IMF archive (even the archival banner remained the same as the figure above shows), but we found some significant changes upon the move from IMF to Archive-It which we will focus on in this post.

We refer to the archive from which mementos were moved (i.e., internetmemory.org) as the "original archive", and we use the "new archive" to refer to the archive to which the mementos were moved (i.e., archive-it.org). A memento is identified by a URI-M as defined in the Memento framework.

Our observations about changes in the NLI collection (from IMF to Archive-It) are as follows:

Observation 1: The functionality of the original archival banner is gone

Users of the European Archive and IMF were able to navigate through available mementos via the custom archival banner (marked in red in the top two screenshots in the figure above). Via this banner, the original archive allows users to view the available mementos and the representation of a selected memento in the same page. Archive-It, on the other hand, now uses the standard playback banner (marked in red in the bottom screenshot in the figure above). This new archive's banner contains information to inform users that they are viewing an "archived" web page. This banner also contains multiple links. One of the links will take you to a web page in archive-it.org that shows all available mementos in the archive as shown in the figure below:

Observation 2: The original archive is no longer reachable

After mementos were moved from internetmemory.org, the archive became unreachable as the following cURL session shows:

$ date
Tue May 21 08:03:51 EDT 2019

$ curl http://www.internetmemory.org
curl: (7) Failed to connect to www.internetmemory.org port 80: Operation timed out


In addition to IMF, the European Archive (europarchive.org) also is no longer maintained---it was shut down and the domain name was purchased by another entity and is now spam. 



The movement of mementos from these two archives will affect link integrity across web resources that contain links to mementos from the European Archive or IMF. As mentioned in the previous post, there actions that can be performed by original archives to maintain link integrity via "follow-your-nose" from the old URI-Ms to the corresponding URI-Ms in the new archive. 


For example, the archive Library and Archives Canada changed its domain name name from collectionscanada.gc.ca to webarchive.bac-lac.gc.ca (described in part 1), and because the archive still controls the original domain name collectionscanada.gc.ca, the archive could (even though it currently does not) redirect requests of URI-M in collectionscanada.gc.ca to the new archive webarchive.bac-lac.gc.ca. For instance, if the original archive uses the Apache web server, the mod_rewrite rule can be used to perform automatic redirects:

# With mod_rewrite
RewriteEngine on
RewriteRule   "^/webarchives/(\d{14})/(.+)" http://webarchive.bac-lac.gc.ca:8080/wayback/$1/$2  [L,R=301]

But these practices become impractical in the case of the European Archive and IMF because:

• The archives no longer exist (e.g., the European Archive and IMF),  so there is not a maintained web server available to issue the redirects. 
• Even if it still existed, the archive might decide to not issue redirects for former customers in order to increase lock-in.

In the upcoming post, we will describe the movement of mementos from the Public Record Office of Northern Ireland (PRONI) web archive to Archive-It. The PRONI organization still controls and maintains the original domain name webarchive.proni.gov.uk, so it is possible for PRONI to issue redirects to the new URI-Ms in Archive-It.

Observation 3: Not all mementos are available in the new archive

As defined in the previous post, a missing memento occurs when the values of the Memento-Datetime, the URI-R, and the final HTTP status code of the memento from the original archive are not identical to the values of the corresponding memento from the new archive. In this study, we found 192 missing mementos (out of 979) that can not be retrieved from the new archive. Instead, the new archive responds with other mementos that have different values for the Memento-Datetime, the URI-R, or the HTTP status code. We give two examples of missing mementos. The first example shows a memento can not be found in the new archive  with the same Memento-Datetime as it was in the original archive. When requesting the URI-M:

http://collections.internetmemory.org/nli/20121221162201/http://bbc.co.uk/news/

from the original archive (internetmemory.org) on September 03, 2018, the archive responded with "200 OK" with the representation shown in the top screenshot in the figure below. The Memento-Datetime of this memento was Fri, 21 Dec 2012 16:22:01 GMT. Then, we requested the corresponding URI-M:

http://wayback.archive-it.org/10702/20121221162201/http://bbc.co.uk/news/


from the new archive (archive-it.org). As shown in the cURL session below, the request redirected to another URI-M:

http://wayback.archive-it.org/10702/20121221163248/http://www.bbc.co.uk/news/

As shown in the figure below, the representations of both mementos are identical (except for the archival banners), we consider the memento from the original archive as missing because both mementos have different values of the Memento-Datetime (i.e., Fri, 21 Dec 2012 16:32:48 GMT in the new archive) for a delta of about 10 minutes. Even though the 10 minute delta might not be semantically significant (apparently just a change in the canonicalization of the URI-R, with bbc.co.uk redirecting to www.bbc.co.uk), we do not consider it to be the same since the values of the Memento-Datetime are not identical.

$ curl --head --location --silent http://wayback.archive-it.org/10702/20121221162201/http://bbc.co.uk/news/ | egrep -i "(HTTP/|^location:|^Memento-Datetime)"

HTTP/1.1 302 Found
Location: /10702/20121221163248/http://www.bbc.co.uk/news/
HTTP/1.1 200 OK
Memento-Datetime: Fri, 21 Dec 2012 16:32:48 GMT

The second example shows a memento that has different values of the Memento-Datetime and URI-R compared to the corresponding values from the original archive. When requesting the memento:

http://collections.internetmemory.org/nli/20121223122758/http://www.whitehouse.gov/

on September 03, 2018, the original archive returned "200 OK" for an archival "403 Forbidden" as the WARC record below shows:


WARC/1.0
WARC-Type: response
WARC-Target-URI: http://collections.internetmemory.org/nli/content/20121223122758/http://www.whitehouse.gov/
WARC-Date: 2018-09-03T16:31:30Z
WARC-Record-ID: <urn:uuid:d03e5020-af96-11e8-9d72-f10b53f82929>

Content-Type: application/http; msgtype=response
Content-Length: 1694
HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 16:31:19 GMT
Server: Apache/2.4.10
Age: 0
Vary: Accept-Encoding
Content-Type: text/html
Via: 1.1 varnish-v4
cache-control: max-age=86400
X-Varnish: 28318986 28187250
Memento-Datetime: Sun, 23 Dec 2012 12:27:58 GMT
Connection: keep-alive
Accept-Ranges: bytes
Link: <http://collections.internetmemory.org/nli/content/20121223122758/http://www.whitehouse.gov/>; rel="memento"; datetime="Sun, 23 Dec 2012 12:27:58 GMT", <http://collections.internetmemory.org/nli/content/20110223072152/http://www.whitehouse.gov/>; rel="first memento"; datetime="Wed, 23 Feb 2011 07:21:52 GMT", <http://collections.internetmemory.org/nli/content/20180528183514/http://www.whitehouse.gov/>; rel="last memento"; datetime="Mon, 28 May 2018 18:35:14 GMT", <http://collections.internetmemory.org/nli/content/20121221220430/http://www.whitehouse.gov/>; rel="prev memento"; datetime="Fri, 21 Dec 2012 22:04:30 GMT", <http://collections.internetmemory.org/nli/content/20131208014833/http://www.whitehouse.gov/>; rel="next memento"; datetime="Sun, 08 Dec 2013 01:48:33 GMT", <http://collections.internetmemory.org/nli/content/timegate/http://www.whitehouse.gov/>; rel="timegate", <http://www.whitehouse.gov/>; rel="original", <http://collections.internetmemory.org/nli/content/timemap/http://www.whitehouse.gov/>; rel="timemap"; type="application/link-format"
Content-Length: 287

<HTML><head>
<title>[ARCHIVED CONTENT] Access Denied</title>
</head><BODY>
<H1>Access Denied</H1>
Reference&#32;&#35;18&#46;d8407b5c&#46;1356265678&#46;2324d94
</BODY>
</HTML>

You don't have permission to access "http&#58;&#47;&#47;wwws&#46;whitehouse&#46;gov&#47;" on this server.<P>

When requesting the corresponding memento from archive-it.org: 

http://wayback.archive-it.org/10702/20121223122758/http://www.whitehouse.gov/

the request redirected to another URI-M:

http://wayback.archive-it.org/10702/20121221222130/http://www.whitehouse.gov/administration/eop/nec/speeches/gene-sperling-remarks-economic-club-washington

which is "200 OK". Notice that not only the values of the Memento-Datetime are different but also the URI-Rs. The representations of both mementos from the original and new archives are shown below:



Observation 4: Both archives handle the archival 4xx/5xx differently

The replay tool in the original archive (internetmemory.org) is configured so that it returns the status code "200 OK" for archival 4xx/5xx. 

For example, when requesting the memento:

http://collections.internetmemory.org/nli/20121021203647/http://www.amazon.com/

on September 03, 2018, the original archive returned "200 OK" for an archival "503 Service Unavailable" as the WARC record below shows:

WARC/1.0
WARC-Type: response
WARC-Target-URI: http://collections.internetmemory.org/nli/20121021203647/http://www.amazon.com/
WARC-Date: 2018-09-03T16:46:51Z
WARC-Record-ID: <urn:uuid:f4f2e910-af98-11e8-8de6-6f058c4e494a>
Content-Type: application/http; msgtype=response
Content-Length: 271841

HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 16:46:39 GMT
Server: Apache/2.4.10
Age: 0
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Via: 1.1 varnish-v4
cache-control: max-age=86400
Transfer-Encoding: chunked
X-Varnish: 28349831
Memento-Datetime: Sun, 21 Oct 2012 20:36:47 GMT
Connection: keep-alive
Accept-Ranges: bytes
...

Even the HTTP status code of the inner iframe in which the archived content is loaded had "200 OK":

WARC/1.0
WARC-Type: response
WARC-Target-URI: http://collections.internetmemory.org/nli/content/20121021203647/http://www.amazon.com/
WARC-Date: 2018-09-03T16:46:51Z
WARC-Record-ID: <urn:uuid:f500cbc0-af98-11e8-8de6-6f058c4e494a>
Content-Type: application/http; msgtype=response
Content-Length: 2642

HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 16:46:40 GMT
Server: Apache/2.4.10
Age: 0
Vary: Accept-Encoding
Content-Type: text/html
Via: 1.1 varnish-v4
cache-control: max-age=86400
Transfer-Encoding: chunked
X-Varnish: 27227468 27453379
Memento-Datetime: Sun, 21 Oct 2012 20:36:47 GMT
Connection: keep-alive
Accept-Ranges: bytes
Link: <http://collections.internetmemory.org/nli/content/20121021203647/http://www.amazon.com/>; rel="memento"; datetime="Sun, 21 Oct 2012 20:36:47 GMT", <http://collections.internetmemory.org/nli/content/20110221192317/http://www.amazon.com/>; rel="first memento"; datetime="Mon, 21 Feb 2011 19:23:17 GMT", <http://collections.internetmemory.org/nli/content/20180711130159/http://www.amazon.com/>; rel="last memento"; datetime="Wed, 11 Jul 2018 13:01:59 GMT", <http://collections.internetmemory.org/nli/content/20121016174254/http://www.amazon.com/>; rel="prev memento"; datetime="Tue, 16 Oct 2012 17:42:54 GMT", <http://collections.internetmemory.org/nli/content/20121025120853/http://www.amazon.com/>; rel="next memento"; datetime="Thu, 25 Oct 2012 12:08:53 GMT", <http://collections.internetmemory.org/nli/content/timegate/http://www.amazon.com/>; rel="timegate", <http://www.amazon.com/>; rel="original", <http://collections.internetmemory.org/nli/content/timemap/http://www.amazon.com/>; rel="timemap"; type="application/link-format"

<html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=iso-8859-1"/>
<title>[ARCHIVED CONTENT] 500 Service Unavailable Error</title>
</head>
<body style="padding:1% 10%;font-family:Verdana,Arial,Helvetica,sans-serif">
  <a  target="_top" href="http://collections.internetmemory.org/nli/20121021203647/http://www.amazon.com/"><img src="http://collections.internetmemory.org/nli/content/20121021203647/http://ecx.images-amazon.com/images/G/01/img09/x-site/other/a_com_logo_200x56.gif" alt="Amazon.com" width="200" height="56" border="0"/></a>
  <table>
    <tr>
      <td valign="top" style="width:52%;font-size:10pt"><br/><h2 style="color:#E47911">Oops!</h2><p>We're very sorry, but we're having trouble doing what you just asked us to do. Please give us another chance--click the Back button on your browser and try your request again. Or start from the beginning on our <a  target="_top" href="http://collections.internetmemory.org/nli/20121021203647/http://www.amazon.com/">homepage</a>.</p></td>
      <th><img src="http://collections.internetmemory.org/nli/content/20121021203647/http://ecx.images-amazon.com/images/G/01/x-locale/common/errors-alerts/product-fan-500.jpg" alt="product collage"/></th>
    </tr>
  </table>
</body>

</html>

When requesting the corresponding memento:

http://wayback.archive-it.org/10702/20121021203647/http://www.amazon.com/

Archive-It properly returned the status codes 503 for the archival 503:

curl -I http://wayback.archive-it.org/10702/20121021203647/http://www.amazon.com/

HTTP/1.1 503 Service Unavailable
Server: Apache-Coyote/1.1
Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-inline' 'unsafe-eval' *.archive-it.org archive-it.org *.qa-archive-it.org qa-archive-it.org archive.org *.archive.org https://*.archive-it.org https://archive-it.org https://*.qa-archive-it.org https://qa-archive-it.org https://archive.org https://*.archive.org data: blob: ; frame-src 'self' *.archive-it.org archive-it.org *.qa-archive-it.org qa-archive-it.org archive.org *.archive.org https://*.archive-it.org https://archive-it.org https://*.qa-archive-it.org https://qa-archive-it.org https://archive.org https://*.archive.org ; report-uri https://partner.archive-it.org/csp-report
Memento-Datetime: Sun, 21 Oct 2012 20:36:47 GMT
Link: <http://www.amazon.com/>; rel="original", <https://wayback.archive-it.org/10702/timemap/link/http://www.amazon.com/>; rel="timemap"; 
...

Observation 5: The HTTP status code may change in the new archive

The HTTP status codes of URI-Ms in the new archive might not be identical to the HTTP status code of the corresponding URI-Ms in the original archive. For example, the HTTP request of the URI-M:

http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/


to the original archive resulted in "200 OK" as the part of the WARC below shows:

WARC/1.0
WARC-Type: response
WARC-Target-URI: http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/
WARC-Date: 2018-09-03T08:40:43Z
WARC-Record-ID: <urn:uuid:0b947600-af55-11e8-9b13-5bce71cafd38>
Content-Type: application/http; msgtype=response
Content-Length: 28447

HTTP/1.1 200 OK
Date: Mon, 03 Sep 2018 08:40:30 GMT
Server: Apache/2.4.10
Age: 0
Vary: Accept-Encoding
Content-Type: text/html; charset=utf-8
Via: 1.1 varnish-v4
cache-control: max-age=86400
Transfer-Encoding: chunked
X-Varnish: 27888670
Memento-Datetime: Sun, 23 Dec 2012 03:18:37 GMT
Connection: keep-alive
Accept-Ranges: bytes
Link: <http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/>; rel="memento"; datetime="Sun, 23 Dec 2012 03:18:37 GMT", <http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/>; rel="first memento"; datetime="Sun, 23 Dec 2012 03:18:37 GMT", <http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/>; rel="last memento"; datetime="Sun, 23 Dec 2012 03:18:37 GMT", <http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/>; rel="prev memento"; datetime="Sun, 23 Dec 2012 03:18:37 GMT", <http://collections.internetmemory.org/nli/20121223031837/http://www2008.org/>; rel="next memento"; datetime="Sun, 23 Dec 2012 03:18:37 GMT", <http://collections.internetmemory.org/nli/timegate/http://www2008.org/>; rel="timegate", <http://www2008.org/>; rel="original", <http://collections.internetmemory.org/nli/timemap/http://www2008.org/>; rel="timemap"; type="application/link-format"



<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN"
...

The representation of the memento is illustrated below:

In internetmemory.org

The request to the corresponding URI-M:

http://wayback.archive-it.org/10702/20121223031837/http://www2008.org/

from Archive-It results in "404 Not Found" as the cURL session below shows:

$ curl --head --silent http://wayback.archive-it.org/10702/20121223031837/http://www2008.org/

HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Content-Security-Policy-Report-Only: default-src 'self' 'unsafe-inline' 'unsafe-eval' *.archive-it.org archive-it.org *.qa-archive-it.org qa-archive-it.org archive.org *.archive.org https://*.archive-it.org https://archive-it.org https://*.qa-archive-it.org https://qa-archive-it.org https://archive.org https://*.archive.org data: blob: ; frame-src 'self' *.archive-it.org archive-it.org *.qa-archive-it.org qa-archive-it.org archive.org *.archive.org https://*.archive-it.org https://archive-it.org https://*.qa-archive-it.org https://qa-archive-it.org https://archive.org https://*.archive.org ; report-uri https://partner.archive-it.org/csp-report
Content-Type: text/html;charset=utf-8
Content-Length: 4902
Date: Thu, 05 Sep 2019 08:28:27 GMT

The list of all 979 URI-Ms is appended below. The file contains the following information:

• The URI-M from the original archive (original_URI-M).
• The final URI-M after following redirects, if any, of the URI-M from the original archive (final_original_URI-M).
• The HTTP status code of the final URI-M from the original archive (final_original_URI-M_status_code).
• The URI-M from the new archive (new_URI-M).
• The final URI-M after following redirects, if any, of the URI-M from the new archive (final_new_URI-M).
• The HTTP status code of the final URI-M from the new archive (final_new_URI-M_status_code).
• The difference (in seconds) between the Memento-Datetimes of the final URI-Ms (delta).
• Whether the URI-Rs of the final URI-Ms are identical or not (same_final_URI-Rs).
• Whether the status codes of the final URI-Ms are identical or not (same_final_URI-Ms_status_code).

Conclusions

We did not find any changes in the 979 mementos of the National Library of Ireland (NLI) collection when they were moved from europarchive.org to internetmemory.org in May 2018.  Both archives had used the same replay tool and archival banners. The NLI collection then was moved to archive-it.org in September 2018.  We found that 192 out of the 979 mementos resurfaced in archive-it.org with a change in Memento-Datetime, URI-R, or the final HTTP status code. We also found that the functionality that used to be available in the original archival banner has gone from the new archive. We also noticed that both archives internetmemory.org and archive-it.org react differently to requests of archival 4xx/5xx. 

In the upcoming posts we will provide some details about changes in the archives:

• Public Record Office of Northern Ireland (PRONI) (webarchive.proni.gov.uk)
• WebCite (webcitation.org)

--Mohamed Aturban

2019-09-10: Twitter Follower Growth for the 2020 Democratic Candidates

Figure 1: Popularity measure of the candidates labeled on the basis of their Twitter follower growth in absolute number and percentage

There are more than 20 candidates running for the 2020 Democratic party Presidential nomination but everyone knows there will be only one winner. Since only a handful of the candidates have a real shot of receiving the nomination, the question then arises "why are so many candidates running for their party's nomination?" One answer is that running for the nomination increases a candidate's national media coverage and the resulting popularity creates a launchpad for their future endeavors. This is clearly evident in case of candidates like Pete Buttigieg and Andrew Yang, both of whom have enjoyed increased national exposure regardless of the outcome of the primaries. Since "popularity" is hard to define and quantify, we use the Twitter followers for each candidate as a proxy for their popularity. The absolute and relative increase in the Twitter followers since January 1, 2019 can then be an indicator if the candidates' efforts have been worthwhile in increasing the size of their audience.

Previous Works on Study of Twitter Follower Growth 

FiveThirtyEight published their two-article series about the NBC and CNN Democratic debates where they analyzed each candidate based on five criteria. One of the criteria was the Twitter follower growth chart from the night of the debate to the following afternoon.

Figure 2: Twitter follower growth for Democratic candidates after the NBC debate between the night of the debate and the following afternoon. 
Source: https://fivethirtyeight.com/features/the-first-democratic-debate-in-five-charts/

Figure 3: Twitter follower growth for Democratic candidates after the CNN debate between the night of the debate and the following afternoon. 
Source: https://fivethirtyeight.com/features/the-second-democratic-debate-in-5-charts/

Although FiveThirtyEight captures the immediate effect on an event using the Twitter follower growth, there are other major events such as their campaign announcement and appearing on T.V. shows talking about their candidacy which also affect their overall Twitter follower growth. 

For a better understanding of the Twitter follower growth as a measure of a candidate's popularity, we need to include all the events that have happened in a candidate's campaign until today which requires us to study their historical Twitter information. Miranda Smith in her post "Twitter Follower Count History via the Internet Archive" explains the reason for using the web archives over the Twitter API for finding historical information of Twitter follower count. In this post, we will rework the analysis done by Miranda Smith on the Democratic candidates using the web archives to gather their historical information and present a broader view of the Twitter follower growth for each candidate in 2019.

How we built our data set

We used the 2020 Democratic Party presidential primaries Wikipedia webpage on August 24, 2019 to create our baseline of the candidates. On August 24, 2019, we found 21 candidates to be running for the presidential elections and five candidates had already withdrawn. With 24 of the 26 candidates announcing their candidacy in 2019, we limited our study between January 1, 2019 and August 23, 2019. We used the same Twitter handle for all the 21 candidates mentioned in the FiveThirtyEight article and for the rest of the candidates we used the Twitter accounts which mentioned their 2020 Presidential candidacy in their Twitter bio.

We collected mementos from multiple web archives using MemGator for all the Twitter handles between January 1, 2019 and August 23, 2019 and retrieved the follower count from each memento to build our data set of historical follower count information.

Results   

Table 1 is a sortable table which contains the follower count growth in absolute numbers and in percentage, start follower count, end follower count, memento date range, and Twitter handle for all the candidates. Memento date range for a candidate represents the start and the end time of all the mementos collected from the web archives for their Twitter account. Increase represents the increase in absolute number of Twitter followers for a candidate from the first memento to the last memento. Increase% represents the percentage increase in the follower count for a candidate with respect to their first memento follower count.

   

There are two possible reasons why all candidates do not have mementos in the web archives for their Twitter handles on or near 2019-01-01:

• Archiving rate correlates with popularity, so candidates who were less popular are likely to have a lower archival rate. Pete Buttigieg's Twitter account, @petebuttigieg, has 329 mementos in 2019 which is in contrast with the 70 mementos between 2012 and 2018. The high archival rate of his Twitter account can be attributed to the meteoric rise in his Twitter followers by 1.3M. Although Marianne Williamson has 2.6M Twitter followers, she was archived thrice by the Internet Archive between January and March 2019. So, the correlation between archival rate and popularity might not hold in all the situations. 
• The web archives index their mementos by their URLs (URI-R). A change in the Twitter handle creates a new URL for the same web page. In order to fetch all the mementos for a Twitter account, we need to query the web archives with both the URLs which includes the previous and current Twitter handle URLs. Andrew Yang changed his Twitter handle from @andrewyangvfa to @andrewyang which changed the URL to his Twitter account. Therefore, the first memento for @andrewyang in the Internet Archive is from March 21, 2019. John Hickenlooper (@hickforco to @hickenlooper), John Delaney (@JDelaneyforMD to @JohnKDelaney to @johndelaney), and Michael Bennet (@BennetForCO to @MichaelBennet) have also changed their Twitter handles to reflect their shift from state to national focus.   

Table 1: List of all the Democratic Candidates with their follower count increase in 2019.
"1" represents the first Democratic debates held on the NBC. "2" represents the second Democratic debate held on the CNN. "3" represents the third Democratic debate to be held on 2019-09-12 on ABC. "D" represents the candidates who have dropped out as on 2019-08-24.

Name TwitterHandle	Memento Date Range	Start Follower Count	End Follower Count	Increase	Increase%
Michael Bennet 1, 2 @michaelbennet	18 May - 23 Aug	21,210	38,267	17,057	80.42
Joe Biden 1, 2, 3 @joebiden	01 Jan - 23 Aug	3,175,558	3,690,554	514,996	16.22
Cory Booker 1, 2, 3 @corybooker	03 Jan - 23 Aug	4,083,584	4,341,153	257,569	6.31
Pete Buttigieg 1, 2, 3 @petebuttigieg	24 Jan - 23 Aug	94,963	1,383,508	1,288,545	1356.90
Julian Castro 1, 2, 3 @juliancastro	02 Jan - 23 Aug	136,275	373,432	237,157	174.03
Bill de Blasio 1, 2 @billdeblasio	10 Jan - 23 Aug	139,293	168,092	28,799	20.68
John Delaney 1, 2 @johndelaney	16 Apr - 23 Aug	19,201	35,595	16,394	85.38
Steve Bullock 2 @governorbullock	02 Jan - 20 Aug	166,137	184,448	18,311	11.02
Tulsi Gabbard 1, 2 @tulsigabbard	01 Feb - 23 Aug	216,704	537,745	321,041	148.15
Kamala Harris 1, 2, 3 @kamalaharris	01 Jan - 23 Aug	1,990,349	3,071,524	1,081,175	54.32
Amy Klobuchar 1, 2, 3 @amyklobuchar	24 Jan - 23 Aug	566,492	751,017	184,525	32.57
Beto O'Rourke 1, 2, 3 @betoorourke	01 Jan - 23 Aug	1,111,690	1,553,346	441,656	39.73
Tim Ryan 1, 2 @timryan	16 Apr - 20 Aug	18,116	36,434	18,318	101.12
Bernie Sanders 1, 2, 3 @berniesanders	01 Jan - 23 Aug	8,943,122	9,580,209	637,087	7.12
Elizabeth Warren 1, 2, 3 @ewarren	03 Jan - 23 Aug	2,172,769	3,041,438	868,669	39.98
Marianne Williamson 1, 2 @marwilliamson	05 Mar - 20 Aug	2,602,291	2,758,962	156,671	6.02
Andrew Yang 1, 2, 3 @andrewyang	27 Mar - 23 Aug	200,361	713,462	513,101	277.02
Kirsten Gillibrand 1, 2 @sengillibrand	02 Jan - 23 Aug	1,297,306	1,461,659	164,353	12.67
Joe Sestak @joesestak	25 Jun - 20 Aug	10,715	12,346	1,631	15.22
Wayne Messam @waynemessam	16 Apr - 21 Aug	5,834	8,415	2,581	44.24
Eric Swalwell 1, D @ericswalwell	01 Jan - 20 Aug	23,873	107,484	83,611	350.23
John Hickenlooper 1, 2, D @hickenlooper	13 Mar - 23 Aug	135,774	159,947	24,172	17.80
Jay Inslee 1, 2, D @jayinslee	01 Jan - 20 Aug	30,614	105,935	75,321	246.03
Tom Steyer @tomsteyer	01 Jan - 20 Aug	211,202	244,384	33,182	15.71
Mike Gravel @mikegravel	04 Apr - 21 Aug	40,497	131,905	91,408	225.72
Seth Moulton D @sethmoulton	06 Feb - 20 Aug	135,481	147,289	11,808	8.72

Table 2: Table lists all the candidates into each category based on their increase in follower count in numbers and percentage

	Low % Increase in Follower Count	High % Increase in Follower Count
High Increase in Follower Count	Already Popular @marwilliamson 155K, 6% @sengillibrand 165K, 13% @amyklobuchar 185K, 32% @corybooker 260K, 6% @betoorourke 440K, 40% @joebiden 515K, 16% @berniesanders 640K, 7% @ewarren 870K, 40% @kamalaharris 1.1M, 55%	Big Winners @juliancastro 240K, 175% @tulsigabbard 320K, 150% @andrewyang 515K, 275% @petebuttigieg 1.3M, 1350%
Low Increase in Follower Count	Nobody Noticed @joesestak 1.5K, 15% @waynemessam 2.5K, 44% @sethmoulton 11K, 9% @johndelaney 16K, 85% @michaelbennet 17K, 80% @governorbullock 18K, 11% @hickenlooper 24K, 18% @billdeblasio 29K, 21% @tomsteyer 33K, 16%	Beneficial @timryan 18K, 101% @jayinslee 75K, 245% @ericswalwell 84K, 350% @mikegravel 90K, 225%

Figure 4: @billdeblasio added 29K followers with growth rate of 21% and is an example from the "Nobody Noticed" category

Figure 5: @berniesanders added 640K followers with growth rate of 7% and is an example from the "Already Popular" category 

Figure 6: @ericswalwell added 84K followers with growth rate of 350% and is an example from the "Beneficial" category 

Figure 7: @petebuttigieg added 1.3M followers with growth rate of 1350% and is an example from the "Big Winners" category 

Some observations:

• Joe Biden and Bernie Sanders already have a large number of Twitter followers, so even though the absolute size of their increase is large, their relative increase is small. This matches our intuition of them both being nationally recognized names for whom their candidacies are not about setting up their "next move".
• Since January 2019, Pete Buttigieg's Twitter account has witnessed an increase in his follower count by 1.3M, similar to Andrew Yang who witnessed a Twitter follower rise of 515K. The meteoric rise in their Twitter followers is in congruence with both the candidates becoming nationally recognized names within a span of months.    
• The top three candidates with the lowest Twitter follower growth (@JoeSestak, @WayneMessam, and @SethMoulton) have not appeared in any debate.
• Except for Senator Tulsi Gabbard, all the other ten candidates from the top 11 candidates who have the highest growth in their Twitter follower count will be appearing in the third Democratic debate on September 12, 2019. This matches our intuition of using Twitter followers as a proxy for measuring the popularity of candidates.

We analyzed the follower counts for 26 Democratic candidates and used their Twitter followers growth as a proxy to measure their popularity. We categorized four Twitter handles to be the big winners, four in the beneficial, nine in the already popular, and nine in the nobody noticed category based on the threshold value of 100K and 100% Twitter follower growth in absolute number and percentage growth. The ten candidates who will appearing in the third debate make up the top 11 highest gainers of Twitter followers matching our intuition of using Twitter followers as a proxy to measure the popularity of candidates.

--------
Mohammed Nauman Siddique
(@m_nsiddique)