2020-04-25: Effect of Reading Patterns of Novice Researchers using Eye Tracking

Figure 1: A participant reading the research paper wearing the PupilLabs Core eye tracker. 

Scientific literature gives novel research ideas as well as solutions to various problems. When it comes to scientific literature, reading pattern vary from one person to another. Common reading patterns may exist among researchers having similar expertise in a particular area, novice researchers may have different reading patterns compared to more experienced researchers. We can expect a difference in reading patterns in terms of scan paths and pupillary activity.

The ability to seek information from different sections of research papers determines the reading process of a researchers. Some researchers read the research papers starting from the beginning of the research paper till the end, whereas others read them in a different order than presented. One way to read a research paper is the three-pass approach. Researchers also tend to change their reading patterns over time as they familiarize with the content and structure of research papers.

To explore the eye movements of novice researchers using eye-tracking measures during a research papers reading task, Dr. Sampath Jayarathna, Dr. Jian Wu, and I (Gavindya Jayawardena) conducted a study with three Ph.D. students who were in their first or second year in the doctoral program. They were all familiar with reading and reviewing research papers in various conference venues. Our goal was to explore three aspects of the reading task:

1. Order of the sections read 
2. Fixation count made on each section
3. Cognitive load when reading each section

As the reading material, we selected a 2-page poster publication from the 2018 Joint Conference on Digital Libraries (JCDL). We asked participants to read the research paper while wearing the PupilLabs Core eye tracker (see Figure 1). Before each participant read the given paper, we calibrated their eye movements using the screen marker calibration in Pupil Capture Software.

After all three participants complete reading task, we used Pupil Player Software to extract recorded eye-movements. Example recording of eye movements is shown below.

Video 1: Sample recording of eye movements of a participant while reading the research paper.

We defined five areas of interest (AOIs), corresponding to sections of the research paper: (1) title, (2) abstract, (3) motivation, (4) methodology, and (5) conclusion to analyze the eye-movements of participants.

To observe how the eye-tracking measurements change over the course of the reading task, we used the Real-Time Advanced Eye Movements Analysis Pipeline (RAEMAP), which is a modified version of the gaze analytics pipeline. We applied the gaze analytics pipeline to analyze the performance of adolescents with ADHD Using eye-tracking measurements recorded during an audiovisual Speech-In-Noise (SIN) task. This work is published at:

*Gavindya Jayawardena, Anne Michalek, Andrew Duchowski, and Sampath Jayarathna. "Pilot Study of Audiovisual Speech-In-Noise~(SIN) Performance of Young Adults with ADHD". Proceedings of the ACM Symposium on Eye Tracking Research and Applications (ETRA), June 2-5, 2020, Stuttgart, Germany. 

RAEMAP facilitates computation of fixation counts, fixation duration, and various other eye movement measurements such as pupillometry measurements which indicates the cognitive load. RAEMAP also has the capability of generating visualizations of gaze points, AOIs, scan paths, and fixations on AOIs. The architecture of RAEMAP is shown in Figure 2.

Figure 2: The architecture of RAEMAP which process eye-tracking data as being streamed by an eye-tracker. Process step calculate fixations, fixations on AOIs, saccade amplitudes, saccade duration, and IPA, whereas graph step generate visualizations. Collate aggregates calculated eye gaze metrics.

In RAEMAP, the calculations of eye gaze metrics of participants are done in separate processes utilizing distributed computing resources as illustrated in Figure 2. It is because eye movements of participants are independent of one another. The current implementation of RAEMAP have the stream processing capability to calculate eye gaze metrics as data is being streamed by an eye tracker.  Read more on RAEMAP architecture in our paper:

*Gavindya Jayawardena. “RAEMAP: Real-Time Advanced Eye Movements Analysis Pipeline”. In Symposium on Eye Tracking Research and Applications (ETRA ’20 Adjunct), June 2–5, 2020, Stuttgart, Germany.

We applied RAEMAP to calculate the fixation counts, fixation duration, and IPA counts for each participant during the reading task. When analyzing the eye movement recording to find out the order of the sections read, we observed that all researchers started reading the research paper from the title section. Following this, two out of three participants read the abstract and scanned the images of the paper, whereas one participant scanned the images of the paper and then read the abstract. Afterwards, two out of three participants read the motivation of the paper, whereas one participant read the conclusions of the paper. Among the two participants who read the motivation, one proceeded with reading the methodology and conclusions, whereas the other proceeded with reading the conclusions and methodology. dons instead of the motivation proceeded with reading the motivation and methodology. Overall, we observed different scan paths among the participants indicating that novice researchers who participated in our study had different reading patterns from one another.

As the next step, we analyzed the fixation count and fixation duration on each AOI. Fixation count indicates the number of times that the eyes fixated on an AOI, whereas fixation duration indicates the total time of the fixations. We calculated fixations counts and fixation duration of the participants using RAEMAP. The average fixation counts and fixation duration of participants on AOIs suggested that participants strongly preferred to fixate more on the methodology section and spent more time reading it compared to the others.

Figure 3: Average Fixation Count and Fixation Duration on each AOI

Finally, we calculated the Index of Pupillary Activity (IPA) count. It is a measurement of cognitive load calculated using a wavelet decomposition of the pupil diameter signal. Low IPA counts indicate low cognitive load, whereas high IPA counts indicate higher cognitive load. 

Figure 4: Average IPA counts on each AOI

Since we observed the highest number of fixations on the methodology section, we expected cognitive load of participants to be higher when reading the methodology section. In contrast, we observed a higher cognitive demand on participants when reading the title section, indicating a higher cognitive demand prior to exploring the research idea presented in the paper. 

As we move forward, we plan to conduct a comparison study to explore scan paths of both novice and experienced researchers using Pupil Invisible eye-tracker.

-- Gavindya Jayawardena (@Gavindya2)