2026-07-18: ICSSI 2026 - Trip Report from Boulder, Colorado
From June 29 to July 1, 2026, I attended the 5th International Conference on the Science of Science and Innovation (ICSSI 2026) in Boulder, Colorado. The conference was hosted jointly at the Limelight Hotel and the University of Colorado Boulder. ICSSI brings together researchers who study science itself, including how discoveries happen, how careers are built, how funding shapes research, and how AI is changing all of these. The conference was supported by the US National Science Foundation, the Alfred P. Sloan Foundation, Digital Science, Cevian Labs, and the BioFrontiers Institute.
If you are new to the field, “science of science” (or metascience) is the study of science itself, using data, network analysis, and computational methods to understand how research careers, collaborations, and ideas develop over time. Dashun Wang's The Science of Science is a good introduction if you want an overview before reading the rest of this post.
This year's edition felt especially timely. Shifting federal funding priorities, the growing use of large language models in research, and ongoing debates about peer review and scientific credit shaped many of the discussions. Across keynotes, panels, and lightning talks, speakers kept returning to the same question: what does it mean to do trustworthy, resilient science today? Below is a recap of each day, along with my reflections, followed by a brief overview of the poster I presented.
Day 1 (June 29, 2026)
The morning opened with welcome remarks from the conference co-chairs, Dan Larremore, Erin Leahey, and Bhaven Sampat. It was great to see many familiar faces from the Ai4SciSci workshop community in person. During breakfast and the coffee breaks, I had the opportunity to talk with Jeff Tsao, Alexander Petersen, Daniel Acuña, and Dashun Wang.
The first two invited talks explored how AI is reshaping scientific research from complementary perspectives. Ryan Hill used AlphaFold as an experiment, showing that while it has expanded research on previously unsolved proteins and made experimental work more efficient, it has not displaced experimental structure determination. Instead, researchers are increasingly focusing on problems where AlphaFold remains weakest, highlighting how AI complements rather than replaces scientific work. Chaoqun Ni broadened the discussion by examining how generative AI is changing the research ecosystem across six core tasks, from exploration to evaluation. Her findings suggest that higher AI use is associated with a reorganization of collaboration, with contributor roles becoming increasingly modular and differentiated. This points to changes in how research teams are structured rather than simply making individual scientists more productive.
The discussion then shifted from AI to science policy with a panel on the “past, present, and future of the social contract for science”, featuring Lisa Margonelli, Tony Mills, and Heather Douglas, and moderated by Bhaven Sampat. Heather Douglas argued that she does not subscribe to Vannevar Bush's post World War II linear model, noting that it is difficult to judge the value of research before it is done, making early evaluation an unreliable basis for funding decisions. Tony Mills emphasized that while publicly funded researchers have obligations to society, congressional oversight of how research funding is spent has weakened over time. Lisa Margonelli broadened the discussion by reflecting on the implicit social contract between science and society, how it is sustained through the stories we tell about science, and how those stories shape public expectations.
Following the panel discussion, Kevin Gross presented “Risk, Reward, and the Choices We Face as Scholars”, exploring why scientists often avoid risky research despite the potential for larger breakthroughs. Using mathematical models, he discussed how incentives around grants, publications, and career advancement shape research choices. One line that stayed with me was: “Risk aversion in science is not necessarily evidence of poor science”. He also argued that tenure serves as a form of risk insurance, allowing researchers to pursue ambitious ideas while still maintaining expectations for meaningful contributions. He made the point humorously that tenure is not a license for professors to go hiking instead of showing up to the lab.
I spent the afternoon in parallel sessions covering Career Trajectories and Data, Software, and Research Infrastructure. Ben Aoki-Sherwood's study of interdisciplinary faculty hiring found that 17% of faculty hires move into a field completely different from their PhD field. Xiang Zheng showed that interdisciplinary PhDs still face placement barriers at top universities even after entering the job market. On the infrastructure side, Eva Brown and Nicholas Weber's analysis of software credit was one that stood out: 29% of GitHub code contributors never receive authorship credit on related papers, and only 9% of imported software dependencies are mentioned on average. The most-used libraries are often among the least credited. Yulin Yu's “data hedgehog vs. data fox” framing, which argues that using diverse datasets can improve both research impact and career retention, was a good way to summarize an empirical finding. Finally, Seorin Kim’s audit on OpenAlex abstracts found that roughly one in eight contains an integrity issue, often because only part of a structured PubMed abstract is stored, creating problems for downstream analyses and LLM-based tools.
The day concluded with the poster session, which featured about 50 posters. I presented our ongoing project, “Toward a Cross-Domain AI-Ready Database for Reproducibility and Replicability Studies”, with Dr. Sarah Rajtmajer and Dr. Jian Wu. The project aims to build infrastructure that allows researchers and AI systems to systematically query which findings have been reproduced, replicated, or contested across fields, rather than reconstructing that information paper by paper. I had several engaging conversations at the poster, including with staff members from NIH who were interested in Reproducibility and Replicability Studies from the perspective of research funding and evaluation.
Our Poster: Toward a Cross-Domain AI-Ready Database for Reproducibility and Replicability Studies
Day 2 (June 30, 2026)
One of the highlights of the second day was the panel, “The Future Infrastructure of the Scientific Ecosystem”, featuring Jessica Hullman, Daniel Acuña, and Aaron Clauset, moderated by Misha Teplitskiy. Daniel Acuña argued that AI is already reshaping the research ecosystem, from peer review to scientific communication, but emphasized that while AI can quickly evaluate ideas, it still cannot decide which questions are worth pursuing. Jessica Hullman focused on the limits of AI for scientific evaluation, noting that automated reproducibility checks can be easy to cherry-pick and often fail to capture the broader validity of empirical claims. She argued that AI should support, rather than replace, human judgment and highlighted the need for systems that preserve diversity of thought instead of reinforcing the most popular ideas. Aaron Clauset closed by reminding the audience that scientific knowledge ultimately resides in people, not papers, and that strong human communities remain the foundation of trustworthy science.
In his invited talk, “On Scientific Memory and Innovation”, Lingfei Wu explored the relationship between scientific memory and innovation through the disruption index, which measures how much a paper displaces the work that came before it. He raised an intriguing question about AI: while today's models excel at remembering, reasoning, and recombining knowledge, can they also learn to “forget” dominant ideas in ways that foster innovation? As he put it, perhaps hallucinations are not always a bug but sometimes a feature.
The next two invited talks shifted the focus from ideas to the people and institutions that shape science. In “Canary in the Coal Mine? Prospects for Early Career Scientists”, Donna Ginther discussed the challenges facing early career researchers, from funding pressures to recruiting international students, while emphasizing that AI is increasing the demand for human judgment and evaluation. Her practical advice was simple: choose your advisor carefully and be persistent when facing rejection. Charles Gomez concluded with “Elite Nations Drive the Convergence of Global Research Agendas”, presenting evidence that a small group of research-intensive countries increasingly shapes the language and direction of global science, raising important questions about whose ideas gain visibility and influence.
The parallel sessions I attended on the second day covered knowledge flow and science communication. Several presentations challenged how we think about the spread of ideas. Zheng Fu argued that citations often reflect scholarly obligation rather than true intellectual influence, meaning ideas can spread widely without their original sources receiving credit. Kyle Siler showed how a small group of countries continues to capture a disproportionate share of global scientific attention, regardless of where new ideas originate. On the science communication side, Hong Chen presented a framework for tracking research mentioned in podcasts. One finding that stood out was that most podcast discussions include no bibliographic information, allowing retracted findings to continue circulating without being flagged.
Day 3 (July 1, 2026)
The final day opened with lightning talks on the future of science policy. Several presentations examined how science interacts with policy and funding, but two stood out to me. Junsol Kim showed how partisan think tanks can reinterpret scientific evidence as it moves into policy documents, often removing important context. Elena Parkerson then quantified the economic costs of multi-year disruptions in NIH funding, a timely reminder of how funding instability can have long-lasting consequences for scientific research.
The discussion continued with the panel “The Future of Science Policy: Research Needs and Opportunities”, featuring Kaye Husbands Fealing, Andrew Gerard, and Matt Hourihan, moderated by Cassidy Sugimoto. The panel offered several practical takeaways for researchers. Kaye Husbands Fealing encouraged researchers to be “pivot-ready”, noting that funding priorities change across administrations and that science of science researchers should develop enough domain expertise to explain the broader context of their findings. Andrew Gerard emphasized the importance of communicating research clearly and making its implications accessible to a broad audience. Matt Hourihan highlighted the persistent tension between the value of research and the funding available to support it, reminding researchers to understand their audience and remain confident in the value of their work.
The conference concluded with Melinda Baldwin's invited talk, “In Referees We Trust? The Rise of Peer Review”, which examined the history of peer review and its role in shaping how scientific work is evaluated and trusted, including its use in research grant decisions at institutions such as the NSF. The conference then closed with awards and closing remarks from co-chairs and James A. Evans, along with the announcement that the 6th ICSSI will be hosted in Rome, Italy.
Closing Thoughts
Three days in Boulder gave me a clearer picture of where the science of science community is heading. The field is no longer focused solely on understanding how science works. Increasingly, it is also asking how AI is reshaping research, scientific collaboration, incentives, and trust. Throughout the conference, discussions repeatedly returned to the same theme: how can we build a scientific ecosystem that remains trustworthy, resilient, and effective as AI becomes an integral part of the research process? While there were no definitive answers, the conversations made it clear that these questions will shape the field for years to come.
I am grateful to my advisor, Dr. Jian Wu, for supporting my travel to ICSSI 2026.
Comments
Post a Comment