2019-05-06: Twitter broke my scrapers

Fig. 1: The old tweet DIV showing four ( data-tweet-id , data-conversation-id , data-screen-name , and tweet-text ) attributes with meaningful names. These attributes are absent in the new tweet DIV (Fig. 2). On April 23, 2019, my Twitter desktop layout changed. I initially thought a glitch caused me to see the mobile layout on my desktop instead of the standard desktop layout, but I soon learned this was no accident . I was part of a subset of Twitter users who did not have the option to opt-in to try the new layout. New desktop look While others might have focused on the cosmetic or functional changes, my immediate concern was to understand the extent of the structural changes to the Twitter DOM . So I immediately opened my Google Chrome Developer Tools to inspect the Twitter DOM, and I was displeased to learn that the changes to the layout seeped beyond the cosmetic new looks of the icons into the DOM. This meant that I would have to rewrite all my...