Web Science and Digital Libraries Research Group

As researchers, we often create artifacts (i.e., data and code) that others (including ourselves) might reuse in the future. Especially during early stages of research (i.e., the exploration phase), we hack bits and pieces of code together to test out different hypotheses. Upon discovering few hypotheses that work, our focus shifts towards rigorous testing, academic writing, and publication. At which stage should we ensure the reusability of those artifacts? Ideally, as early as possible; yet this is easier said than done. For instance, at exploration stage, it's often impractical to allocate time for data/code reusability. After all, at this point, it's quite unclear whether the hypotheses would even pan out. Likewise, at every stage that follows, it's quite easy to get consumed with testing, academic writing, and publication. Such circumstances have, and continue to, push reusability down to an afterthought. On the flip side, having to work with "difficult...

Streaming-hub [ Link ] In my  previous post , I elaborated on how dataset metadata could be standardized in a manner that enables researchers to efficiently discover and reuse data already collected for past studies. Adopting such a standard brings a host of benefits to research communities – such as simplified data sharing, massively collaborative research, and automated data pre-processing. However, formulating and adapting such a standard would take years, if not decades, unless 1) the public realizes its practical benefits over the initial hassle of transition, and 2) tools and libraries are built that would ease workflows after transition. My previous post tries to addresses the first concern by introducing DFS and DDU. In this post, I describe our work towards addressing the second concern.