2024-06-06: 30 Years of the NASA Technical Reports Server (NTRS): 1994--2024

 

The announcement of NTRS in Usenet's sci.aeronautics newsgroup on June 6, 1994.  Note the quaint requirement of "Forms" support (which was not yet available in all web browsers in 1994).

Today marks the 30th anniversary of the NASA Technical Reports Server (NTRS) being advertised to the public on the Usenet newsgroup sci.aeronautics.  I've had many noteworthy accomplishments in my career, but I'm especially proud that NTRS continues in name and purpose today.  Naturally, the implementation and architecture have changed several times, but in my field there are not many opportunities to have a project that last 30 years (and counting).  

I've talked about the evolution of the NTRS architecture and how it reflected contemporary thinking about DL design before, but I'll provide a summary here.  In 1993, I set up an anonymous FTP server for technical memorandums and technical papers published by Langley Research Center.  I called it the Langley Technical Report Server (LTRS), and it effectively brought the computer science "technical reports via FTP" culture to NASA (see NASA TM-4567).  Later in 1993, I added a web interface to LTRS (see NASA TM-109162), which provided a much needed boost in usability, not just in browsing but also via searching by indexing the abstracts with WAIS, which was pretty much the only free search software at the time (for example, MySQL did not come out until 1995).  

LTRS ca. 1993.

LTRS ca. 1998 (http://techreports.larc.nasa.gov/).

There was a great deal of interest in LTRS, and I was contacted by other centers and projects who wanted to set up their own "report server" (the software to do this was later codified as "TRSkit").  At the same time, the Center for AeroSpace Information (CASI) had their own abstracts-only WAIS server of all publicly available, NASA authored reports and articles.  A web site (the term "digital library" had yet to be fully embraced at that time) that allowed simultaneous search of all the NASA and NASA-affiliated report servers with a WAIS server was clearly needed.  A little bit of Perl programming later, and NTRS was born.  I'd have to dig more thoroughly through my notes to find when it was first available, but Usenet was the social media of the time, and web sites then were bootleg projects that did not receive approval or review, so it's likely that the announcement on June 6, 1994 followed soon after it was working.  Of course, I did not think to take a screen shot of it then, but the image below is from 1995 and surely sufficiently similar to how it looked in 1994.  

NTRS, ca. 1995. Since HTML tables were not yet supported, the "databases" are in a single column. 

NTRS, ca. 1999.  The first memento of NTRS's first URL, http://techreports.larc.nasa.gov/cgi-bin/NTRS. Yes, tables are used for the double column layout; CSS existed but it took a long time to supplant tables for basic formatting.

This version of NTRS was detailed in few publications (AIAA-95-0964, IR 5(2), IR 6(1)), and as I wrote in 2016, the DL community was committed to making distributed search work.  For a variety of reasons, some theoretical but mostly engineering, distributed search did scale.  As Herbert and I wrote in 2015, the community's collective frustration with distributed searching informed the transition from distributed searching to metadata harvesting, specifically OAI-PMH.  

After completing my PhD in 2000, spending a post-doc year at UNC SILS, and then returning, I left NASA LaRC and joined ODU as an assistant professor in the computer science department in July 2002.  I continued to work on NTRS as a contractor, and July 2002 is also when NTRS transitioned from its previous WAIS distributed searching architecture to using OAI-PMH to harvest Dublin Core metadata records, index them in a MySQL database, and then managed them via a modified "bucket" (this architecture is documented in LHT 2003, JCDL 2004, CR 2005).  This version of NTRS was still a hacked together, home-brew implementation, but it was certainly more principled and standards-based than the previous version, and was at least nominally maintainable by someone other than myself.  It also provided the platform for additional DL research, such as deploying and evaluating a recommendation service (WIDM 2004).  It was around 2004 or 2005 when I ceased being involved in the operation of NTRS, and I lost track of the various architecture and implementations after that.  Eventually, the semi-distributed architecture of OAI-PMH was dropped and everything was centralized.  David SH Rosenthal has written extensively and persuasively about the tendency to centralization ("It Isn't About The Technology" from 2018 is a good place to start), and while NTRS is not FAANG, I suppose that, like death, centralization comes for us all.  

NTRS Simple Search, ca. 2003. 

NTRS Advanced Search, ca. 2003.

NTRS had > 500k records ca. 2003, but about 1/2 of that was from harvesting arXiv.org ("If You Harvest arXiv.org, Will They Come?") 

NTRS, ca. late 2005. The UI has been updated, but the code base is (likely) the same.

NTRS, ca. 2009. The UI is similar, but the code has transitioned from Perl to JSP by this time.

Another aspect of working on NTRS (and LTRS) that have influenced my career and architectural perspectives is that if information is valuable, it must have a path out of its hosting organization. Or, as I would say more succinctly in candid moments: "NASA reports are too important to be left to NASA".  Shutdowns and server unavailability were simply a fact of life: from hurricanes, to 9/11, to alleged espionage.  The latter event produced my 2013 posts regarding NTRS, Handles, and web archives, where I talked about all the different non-NTRS locations where you could get NASA reports ("NTRS, Web Archives, and Why We Should Build Collections", and "NTRS, Memento, and Handles").  I'm happy to say that NTRS still supports an API as well as a bulk download option so this reposting of NASA reports can still continue.  

NTRS taken down, ca. early 2013. 

Blocked (HTTP 403), ca. 2017.  I don't recall the reason: was this detecting a "bot" or was the service down? 

In many ways, the evolution of NTRS tracks the evolution of digital libraries, as well as the arc of my own career.  I was fortunate to have the chance to create this service for the agency, as well as use this as a starting point to interact with many excellent engineers and scientists, both within NASA and beyond.  I maintained it for about the first 10 years, and I'm happy to know that it has persisted for 20+ years after my involvement ceased -- a rarity for most software projects.  Of course, NTRS was made possible through the help and coordination of many people, too many to list and some probably lost to the haze of time.  I would like to specifically thank Calvin Mackey and JoAnne R. Calhoun, who both handled my transition away from NTRS, Gretchen Gottlich and David Bianco, who were both integral early partners in crime, and to Mary McCaskill and Frank Thames, who were the first managers to "get it" and provided the necessary protection for this bootleg project.  Thanks to all, and here's to the next 30 years of NTRS.

--Michael   

https://ntrs.nasa.gov/ as of 2024-06-06. The name "NASA STI Repository" now seems to be an alternate name for the service,  but I'm hoping "NTRS" is never retired.