2020-12-08: Twitter Added Labels On Its Old User Interface
Figure 1: A screen capture where all the tweets in the view-port are filled with “Fact-check label” (left) and “Violated Twitter Rules” label (right).
To curb the spread of misinformation on their platform, Twitter warns users by labeling the tweets with misleading content. Twitter labels first came to light on May 26, 2020 when Twitter decided to assign one of their fact-check labels to a tweet from U.S. President Donald Trump. A few days later, on May 29, 2020 Twitter labeled another tweet of Trump for violating Twitter rules about glorifying violence. In addition to labeling the tweet, Twitter also disabled the engagements with this post. In Figure 1, you can see how all of his tweets in the view-port are labeled. We have used the account @realDonaldTrump for this study as this account is frequently archived. As of November 25, 2020, there are 160 tweets authored by @realDonaldTrump that have been labeled by Twitter. For this study, we have broadly categorized the two types of labels as “Fact-check” label and “Violated Twitter Rules” label. Out of all the labeled tweets, 135 of them are fact-check labels and 25 of them are “Violated Twitter Rules” labels. Initially, when we replayed these tweets from web archives in May 2020, we noticed that none of the mementos had the labels. This has been covered in our blog post “Twitter Was Already Difficult To Archive, Now It's Worse!”, which includes how different archiving platforms failed to replay Twitter's label. This is because Twitter did not add the label to the old UI at that time. However, as shown in Figure 2, we can see that Twitter has added the “Violated Twitter Rules” label to its old UI. However, the fact-check label still does not exist in the old UI.
Old UI vs. new UI
On June 1, 2020, Twitter shut down access to the old Twitter UI, requiring its users to use the new Twitter UI. However, as we have discussed in our previous blog post, we can still access the old UI by masquerading as Googlebot in the browser’s incognito mode. We also mentioned that labels are likely a feature of the new UI. However, we uncovered that this feature has since been added to Twitter’s old UI at least by August 26, 2020. Figure 3 shows how the “Violated Twitter Rules” label is showing up on the live web when the old UI is accessed by setting the UA as “Googlebot” and accessing the tweet in incognito mode (to prevent sending identifying information via cookies).
Figure 3: “Violated Twitter Rules” label appears in the old Twitter UI on the live web (accessed through Googlebot in incognito mode)
Labels in the new UI
Twitter misinformation labels have always been present in its new UI, but as we have mentioned before, archiving these new UI mementos is difficult due to the rate-limiting nature of the Twitter API. The new UI mementos currently present in IA are also vulnerable to temporal violations, which we covered in “New Twitter UI: Replaying Archived Twitter Pages That Never Existed”. As shown in Figure 4, the memento (2020-08-23T11:26:07Z) suggests that the tweet created on 2020-08-23T11:25:59Z was labeled within 68 seconds of tweeting. However, this particular memento is temporarily violative; the tweet content is populated from a JSON response archived 9 hours in the future relative to the root HTML (2020-08-23T11:26:07Z). Since tweets cannot be edited, the 9-hour difference does not change the content of the tweet, but it will give the illusion that Twitter labeled the tweet almost instantaneously.
Figure 4: Temporal violation in new UI memento captured on 2020-08-23T11:26:07Z. The memento displayed tweet content from 9 hours (2020-08-23T20:04:22) in the future giving the illusion that Twitter labeled the tweet almost instantaneously.
What are the two types of labels?
Since archiving the new UI is problematic, popular web archiving services like Internet Archive’s Save Page Now (SPN) and archive.is are archiving Twitter’s old UI. We made use of the old UI mementos in the IA and archive.is to identify the timeframe when Twitter added labels to its old UI by locating the earliest memento of Trump’s tweets with either of the two labels. We have categorized these label types based on whether the engagements are allowed with the tweet once Twitter assigned the label to it. If some or all of the content in the tweet violates the Twitter rules and Twitter has limited the engagement to quote tweeting, then we have classified it as a “Violated Twitter Rules” label. We have compiled a dataset for labeled tweets and retweets by @realDonaldTrump from May 25, 2020 through Nov 25, 2020 (six months) which were labeled by Twitter. When categorizing the tweets as “Fact-check” and “Violated Twitter Rules”, we have disregarded certain retweets/quote tweets as described below (Figure 5):
Retweets/quote tweets of others where only the original tweet is labeled. This is because we are only focused on the labeled tweets authored by Donald Trump himself (Figure 5 - left).
Retweets/quote tweets of his own tweets where only the original tweet is labeled since we do not need to calculate the same labeled content twice (Figure 5 - right).
Figure 5: This figure shows an instance where @realDonaldTrump has quote tweeted a tweet by @RandyRRQuaid (left) and of himself (right). In both of these cases, only the original tweet is labeled by Twitter.
Regardless of the above two points, if Donald Trump has quote tweeted someone else’s tweet or his own tweet and his quote tweet is labeled, we have counted it towards our categorization (Figure 6).
Figure 6: This figure shows an instance where @realDonaldTrump has quote tweeted a tweet by @RyanAFournier where the original tweet and the quote tweet are both labeled by Twitter.
Fact-check Label
While trying to locate the earliest memento with a fact-check label, we noticed that the fact-check label still is not a part of the old twitter UI. We verified this by checking it on the live web by accessing the old UI for a fact-check labeled tweet. Figure 7 shows how the old UI is not showing the labels of this kind, while the new UI does. It also reasserts the fact that labels were initially features of the new UI.
Figure 7: Tweet with fact-check label in old UI (left) and new UI (right) in the live web. Fact-check labels are currently not displaying in Twitter’s old UI.
We looked into how Twitter applies the fact-check label in the new UI. Shown below (Figure 8) is a snippet of the JSON response for a tweet with a fact-checked label. In the tweet JSON, the content nested deeply under the timeline object contains the entry for the main tweet. The entry for the main tweet under the timeline object contains the “forwardPivot” key which holds the following information regarding the label:
“DisplayType” key and “SoftIntervention” value: We believe that this defines the nature of the action Twitter is taking by assigning this label.
“Text” key and “This claim about election fraud is disputed” value: The text that is displayed in the label (as shown in Figure 8).
It may also provide a link to an external URL or Twitter Moment which will provide further information.
Exclamation mark icon image
$ jq .timeline.instructions[][][][].content[][] 1325099845045071873F.json | head -24
{
"tweet": {
"id": "1325099845045071873",
"displayType": "Tweet",
"hasModeratedReplies": false,
"forwardPivot": {
"text": {
"text": "Official sources may not have called the race when this was Tweeted",
"entities": []
},
"landingUrl": {
"urlType": "ExternalUrl",
"url": "https://twitter.com/i/events/1318573265820921857"
},
"displayType": "SoftIntervention",
"iconImageVariant": {
"url": "https://pbs.twimg.com/semantic_core_img/1222518808658636800/8isT_btV?format=png&name=orig",
"width": 48,
"height": 48
},
"engagementNudge": true
}
}
}Figure 8: A snippet of the JSON response for a tweet with a fact-checked label in the new Twitter UI. The jq command used for exploring this JSON is also listed above.
A tweet with a fact-check label has a similar structure to a usual tweet, but with the forwardPivot element in addition. Unlike violated rules labels, the fact check label does not limit the interaction with the tweet. However, the fact-check label does not get displayed in old UI mementos since Twitter has not added this particular label to its old UI.
Violated Twitter Rules Label
We noticed that there were mementos present in the Internet Archive and archive.is with the “Violated Twitter Rules” label. We verified this change by looking into it on the live web. If you take a look at Figure 9, you would notice how the old UI and the new UI are both displaying the “Violated Twitter Rules” label.
Figure 9: “Violated Twitter Rules” label is displayed in both old Twitter UI (left) and new UI (right) on the live web
We noticed that this label in the old UI was embedded in the HTML itself with the class attribute as “tombstone-label” (Figure 10). However, in the new UI, the label is part of the tweet JSON.
Figure 10: “Violated Twitter Rules” label is embedded in the root HTML for Twitter’s old UI
We looked into how Twitter applies the “Violated Twitter Rules” label in the new UI. In the tweet JSON of the new Twitter UI (Figure 11), the content under the timeline object includes the tombstone object which holds the following information:
“DisplayType” key and “NonCompliant” value: Defining that this particular tweet is not compliant with the Twitter policies.
“tombstoneInfo” object: Includes the richText which also may contain an external URL further down the hierarchy.
The rich text contains the text displayed in the tombstone above the tweet.
External URL provides additional information on the matter from a trusted source.
Tweet object: Includes the tweetId and the “displayType” as “Tweet”.
$ jq .timeline.instructions[][][][].content[][] 1313449844413992961V.json
{
"tombstone": {
"displayType": "NonCompliant",
"tombstoneInfo": {
"text": "",
"richText": {
"text": "This Tweet violated the Twitter Rules about spreading misleading and potentially harmful information related to COVID-19. However, Twitter has determined that it may be in the public’s interest for the Tweet to remain accessible. Learn more",
"entities": [
{
"fromIndex": 230,
"toIndex": 240,
"ref": {
"url": {
"urlType": "ExternalUrl",
"url": "https://covid19.twitter.com/"
}
}
}
],
"rtl": false
}
},
"tweet": {
"id": "1313449844413992961",
"displayType": "Tweet"
}
}
}Figure 11: The JSON response for a tweet with a “Violated Twitter Rules” label in the new Twitter UI. The jq command used for exploring this JSON is also listed above.
Fact-check label embedded in the “Violated Twitter rule” label
We noticed a variation in the “Violated Twitter Rule” label, where the fact-check label was also assigned to the tweet by Twitter (Figure 12). We consider this as a “Violated Twitter Rule” label since the engagements are disallowed in these types of tweets due to violating twitter policies. Twitter has taken one more step to minimize the interaction of these tweets and to provide more information by adding a fact-check label. The JSON response for a tweet with this type of label in the new Twitter UI is also shown below (Figure 13).
Figure 12: A tweet with fact-check label embedded in “Violated Twitter Rule” label in the new Twitter UI
$ jq .timeline.instructions[][][][].content[][] 1320873664296804354FV.json
{
"tombstone": {
"displayType": "NonCompliant",
"tombstoneInfo": {
"text": "",
"richText": {
"text": "Some or all of the content shared in this Tweet is disputed and might be misleading about an election or other civic process. Learn more",
"entities": [
{
"fromIndex": 126,
"toIndex": 136,
"ref": {
"url": {
"urlType": "ExternalUrl",
"url": "https://help.twitter.com/rules-and-policies/election-integrity-policy"
}
}
}
],
"rtl": false
}
},
"tweet": {
"id": "1320873664296804354",
"displayType": "Tweet",
"forwardPivot": {
"text": {
"text": "Learn how voting by mail is safe and secure",
"entities": []
},
"landingUrl": {
"urlType": "ExternalUrl",
"url": "https://twitter.com/i/events/1306326582017945601"
},
"displayType": "SoftIntervention",
"iconImageVariant": {
"url": "https://pbs.twimg.com/semantic_core_img/1222518808658636800/8isT_btV?format=png&name=orig",
"width": 48,
"height": 48
},
"engagementNudge": true
}
}
}
}Figure 13: The JSON response for a tweet with a fact-check label embedded in the “Violated Twitter Rule” label in the new Twitter UI. The jq command used for exploring this JSON is also listed above.
The JSON for the tweet in violation follows a different hierarchical structure than the tweet with a fact-check label. This is mainly because of the fact that any engagement with these tweets has been disabled (thus “tombstone”) with only quote tweets allowed. These labels are meant to warn users about the misleading content in the tweets and to discourage them from interacting with the tweet. However, these labels were not reflected on the mementos until 2020-08-26 as they were not added to the old UI before.
When did twitter add the “Violated Twitter Rules” label to the old UI?
The earliest memento for Trump’s tweet with a label that we could find was on August 26, 2020. We took the first seven tweets of Trump labeled by Twitter for violating Twitter rules to verify when this label was added to the old UI. We collected the old UI mementos for these seven labeled tweets from IA’s Wayback machine using MemGator and Wayback CDX Server API. Figure 14 shows that the label did not appear until August 26, 2020 in the old UI memento even for the Tweet that was created on May 29, 2020. However, after September 9, 2020, labels can be seen in old UI mementos for all seven tweets. For the time period between August 26, 2020 and September 9, 2020, we can see mementos with and without labels. This shows that the change was not instantaneous and happened over time.
For the seven tweets visualized above, Table 1 shows when the tweet was created, labeled, and the earliest memento we could find with the label. To find when the tweet was labeled we relied on the earliest evidence of the label that we could find. We observed that Twitter announced on its @TwitterSafety handle that it had labeled Trump's tweet. We could not locate such an announcement tweet from Twitter for the seventh tweet, so we used the earliest memento available with the label as the evidence.
Table 1: Tweet ID, creation date, labeling date, and the earliest memento present in the archives (IA or archive.is) for each of the seven tweets.
Conclusion
In summary:
We have classified the types of labeled tweets broadly as “Fact-check” and “Violated Twitter Rules” labels. To date, the “Fact-check” label has not been added to the old Twitter UI, whereas the “Violated Twitter Rules” label was added to the old UI after August 26, 2020.
Since the “Fact-check” label is still not added to the old UI, the label is missing on the mementos of those labeled tweets in both the Internet Archive and archive.is.
Based on our dataset, we can say that at least by August 26, 2020, Twitter has started adding the “Violated Twitter Rules” label to their old UI. The mementos of labeled tweets (“Violated Twitter Rules” label) in the Internet Archive between August 26, 2020 - September 9, 2020 may or may not contain the label. However, our data confirm that after September 9, 2020, the “Violated Twitter Rules” label can be seen in all the mementos for the labeled tweets.
Hence, there is a span of time between May 26, 2020 and August 26, 2020, where these labels did appear on the live web but not in archived copies with the old Twitter UI. There are real implications to this issue, since historians and researchers may draw inaccurate conclusions about what appeared on the live web.
Acknowledgments
We would like to acknowledge Dr. Michael Nelson, Dr. Michele Weigle, and Sawood Alam for their guidance in compiling this blog post.
--
Kritika Garg (@kritika_garg) & Himarsha Jayanetti (@HimarshaJ)
** Updated the legend of Figure 14 on 2021-03-17; fixed the interchange of colors representing the mementos with and without labels.
Comments
Post a Comment