2026-10-19: FidEx and Wayback QA, Two Tools That Assist With Identifying Why A Resource Failed To Load During Replay

 

Figure 1: Web archiving process augmented with FidEx. Figure from https://www.usenix.org/system/files/nsdi26-zhu-jingyuan.pdf.

To improve the annotation feature for our web archiving livestreams, I have been looking at different Quality Assurance tools that detect replay problems and explain why specific resources fail to load. Most of the Quality Assurance tools that determine the quality of the archived web page do not specify the reason why each web resource failed to load. Zhu et al.’s FidEx tool and Archive-It's Wayback QA tool try to assist with this problem.

Zhu, Sun, and Madhyastha’s paper "Detecting and Diagnosing Errors in Serving Archived Web Pages" describes their approach (Figure 1) for detecting problems with replay quality that are caused by rewriting errors made by the replay system. Their approach involves comparing the layout trees (HTML DOM [Document Object Model] and CSSOM [CSS Object Model]) and the JavaScript writes for the live and archived web page. This can be an improvement over comparing screenshots of the web page, since taking a screenshot of a dynamic web page can result in significant differences in the pixel values when videos, images, and text are frequently being changed. In this case, the layout tree and the JavaScript writes should still be the same on the live and archived web page.

Video 1: Presentation video about the FidEx tool.

Zhu et al.’s presentation slides about FidEx.

Zhu et al.’s FidEx tool can detect replay problems and can output a JSON file that explains why an error occurred. Figure 2 shows an example output file which contains a list of detected errors for each archived web page. Each error specifies the exact line of JavaScript code (including replay system scripts like wombat.js) and the associated error message.

Figure 2: FidEx's pinpoint output file shows errors that occurred during replay.

Archive-It's Wayback QA is another tool that helps with identifying the reason why a resource did not load during replay. According to Yale’s Web Archiving Working Group Quality Assurance Report Archive-It's Wayback QA tool has an option for exporting a CSV file with all of the missing content. This file shows all of the data that would be shown on the Wayback QA tab (Figure 3), including the missing resource's URI-R, the URI-R for the web page that loaded the resource, the reason for the missing content, file type, size, and crawl date. The reason for the missing content is categorized as Excluded, Invalid_Access_Denied, Invalid_Unknown, Invalid_Not_Found, or Not Crawled. The Excluded category is for resources that were intentionally excluded in the missing documents list, like ads and analytics URLs.

Figure 3: Wayback QA's “Missing Documents” page. This screenshot is from a video that is embedded in Archive-It’s blog post “Archive-It Workshop Series Part 2: Reviewing & Quality Assurance”.

FidEx's pinpoint output file provides a more detailed description than Wayback QA for the errors that occurred when replaying the archived web page, but it does not specify which errors caused a specific resource from failing to load. Wayback QA’s “Possible Reason” column,  provides a high-level description of what went wrong for each web resource that failed to load. FidEx’s approach for displaying the errors that occurred during replay is useful for technical users who are familiar with inspecting the JavaScript and HTML files for a web page to see which resources failed to load because of a replay error. In contrast, Wayback QA’s high-level description of the possible error is better suited for archivists looking to quickly identify and patch missing resources.

Another difference between these tools is that FidEx helps identify problems that impact interactional and visual correspondence and Wayback QA displays errors that are associated with the completeness of the archived web page. Reyes Ayala’s grounded theory (Paper: “Correspondence as the primary measure of information quality for web archives: A human-centered grounded theory study”) defined interactional correspondence, visual correspondence, and completeness. Interactional correspondence is associated with being able to perform the same user interactions (e.g., scrolling down a web page to load more web resources or clicking the play button for a video) on the live and archived versions of the web page. Visual correspondence is the visual similarity between the live web page and the archived web page. Completeness for an archived web page is associated with the number of sub-resources that were originally on the live web page that have been archived. In a previous blog post, I surveyed Quality Assurance tools that can help with measuring visual correspondence, completeness, and interactional correspondence.

Summary

FidEx and Wayback QA can help with identifying why a sub-resource failed to load during replay of an archived web page. FidEx focuses on identifying problems caused because of the rewriting that is done by the replay system. Fixing the problems identified by FidEx can help improve the interactional and visual correspondence for an archived web page. Wayback QA assists with problems associated with the completeness of an archived web page, since it specifies possible reasons why each missing resource was not archived. 

Learning about tools like FidEx and Wayback QA helps with identifying what information would be useful to display during a web archiving livestream when an error occurs during replay. Our goal for the annotation feature is to create automated annotations that provide context information for missing sub-resources.  Currently our annotations specify when a sub-resource was not archived. In the future, we want to display potential reasons why the sub-resource failed to replay, such as specifying when a URL matching error or other replay rewriting error occurred. The approaches used by tools like FidEx and Jawa (paper: "Jawa: Web Archival in the Era of JavaScript") could assist with determining which type of rewriting error caused a sub-resource to fail to replay.

--Travis Reid (@treid803.bsky.social)