Web Science and Digital Libraries Research Group

At iPres 2018 , I will be presenting "The Many Shapes of Archive-It", a paper that focuses on some structural features inherent in Archive-It collections. The paper is now available as a preprint on arXiv . As part of the data gathering for " The Many Shapes of Archive-It ", and also as part of the development the Off-Topic Memento Toolkit , I had to write code that extracts metadata and seeds from public Archive-It collections. This capability will be useful to several aspects of our storytelling and summarization work , so I used the knowledge gained from those projects and produced a standalone Python library named Archive-It Utilities (AIU) . This library is currently in alpha status, but is already being used with upcoming projects. The metadata available from an Archive-It collection Archive-It curators can use the predefined metadata fields of Dublin core. They can also supply their own custom metadata fields. An screenshot of Archive-It collecti...

A Portion of a Tweet Conversation about the Ebola Virus We conducted another study in the same spirit as the first , as part of our research ( funded by IMLS ) to build collections for stories or events. This time we sought to understand how to extract not just a single tweet, but the conversation of which the tweet belongs to. We explored how the visibility of tweets in a conversation graph changes based on the tweet selected. A need for archiving tweet conversations Archiving tweets usually involves collecting tweets associated with a given hashtag. Even though this provides a "clean" way of collecting tweets about the event associated with the hashtag, something important is often missed - conversations. Not all tweets about a particular topic will have the given hashtag,  including portions of a threaded conversation, even if the initial tweet contained the hashtag. This is unfortunate because conversations may provide contextual information about t...

I recently conducted an experiment on tags provided from the bookmarking site delicious.com . The goal was to obtain a decent sized sample set of URIs and tags that users have used to annotate the URIs. The website provides a recent tool that automatically redirects to a somewhat random URI that was recently annotated by some Delicious user. By parsing the HTTP headers I was able to grab the redirect URI and therefore build a corpus of 5000 unique URIs. The URI for the tool is http://www.delicious.com/recent/?random=1 . As the second step I needed to obtain the corresponding tags for each URI. I tried to be a good programmer and used the Delicious API to query for the tags instead of parsing the web interface. In order to use the API (v1) you need an account with Delicious/Yahoo. The request for https://username:pwd@api.del.icio.us/v1/posts/suggest?url=http://www.google.com/ for example returns an XML formated response with the top five popular tags: search google search eng...