2024-08-06: Disinformation Detection and Analytics REU Program 2024 - Final Presentations

By -

Students presenting at ODU CS REU Program 2024 - Final Presentations

This year marks the third year of the National Science Foundation (NSF) funded program of Research Experiences for Undergraduates (REU) on Disinformation Detection and Analysis at Old Dominion University. Each year, eight undergraduate students are selected nationwide and conduct research projects with their mentors. The students presented their projects on July 23, 2024, concluding this year's REU program. The program was hosted by the Web Science and Digital Libraries (WS-DL) Research Group within the Department of Computer Science in collaboration with VMASC. More than 20 people, including mentors, REU students, graduate students, and other faculty, participated in the final presentation either online or in person. More detailed information on this year’s REU program can be found in the introductory blog post by Lawrence Obiuwevwi and the midsummer presentation blog post by Kumushini Thennakoon.

The final presentation session of the REU program 2024, started with a brief introduction from Dr. Jian Wu.

Matthew Maisonave (@Matt_Maisonave) advised by Dr. Jian Wu (@fanchyna)

As the first presenter, Matthew showcased his project “Scientific News Verification with Chain-of-Thought”. The research project focused on improving the accuracy and reliability of scientific news verification using few-shot learning and chain-of-thought (CoT) on large language models. He discussed how he first built MSVEC v2, a revised dataset dataset consisting of 138 scientific claims, each accompanied by a research paper that supports or refutes the claim. He compared the performance of zero-shot, few-shot, and CoT prompting methods on stance labeling and rationale annotation tasks. The results showed that the revisions in MSVEC v2 significantly affected the stance labeling performance compared with the previous version. The proof-of-concept experiment on CoT prompting demonstrated its potential to correctly re-evaluate scientific claims that were incorrectly labeled by the zero-shot method. The research results indicate that while CoT outperforms zero-shot and few-shot on the rationale annotation task, it underperforms zero-shot and few-shot on the stance labeling task.

Kevin Cauchi (@CauchiKevin) advised by  Dr. Sampath Jayarathna (@OpenMaze)

Second, Kevin presented his project “Misinformation in Virtual Reality”, exploring the impact of virtual reality (VR) on the perception of misinformation. Kevin’s survey found that videos with a human element, such as a person reacting to the information, are more credible than “AI-generated” ones. According to the study, VR experiences enhanced immersion and focus on the content, potentially making misinformation more believable. He concluded that integrating eye-tracking technology and AI within VR environments leads to promising directions for understanding and combating the spread of misinformation. His research highlighted the impact of VR on information consumption and the need for innovative approaches to address the challenges posed by misinformation in immersive environments.

Stefania Dzhaman (@sdzhaman) advised by Dr. Vikas G. Ashok (@vikas_daveb)

As the third presenter, Stefania presented “Dark Pattern Detection", focusing on detecting dark patterns in web user interfaces, which are designed to mislead users into unintended actions such as purchases or subscriptions. Using datasets from 2019 and new collections, her study involved classifying dark patterns through zero-shot and few-shot prompting methods using ChatGPT. Few-shot prompting showed superior results with 100% accuracy compared to 66% from zero-shot. She concluded with recommendations for future work including dataset expansion, model fine tuning, advanced prompting techniques, and implementing a browser extension to mitigate dark patterns.

Emily Ebalo (@em_ebalo) advised by Dr. Faryaneh Poursardar (@Faryane)

Next, "Early Misinformation Detection in Russian-Ukraine News Articles" by Emily Ebalo outlined the research conducted on detecting misinformation within news articles related to the Russian-Ukraine conflict. The study incorporated a variety of machine learning models, including traditional models like SVM and Random Forest, as well as neural network models and transformers like RoBERTa and XLNet. She had collected data from a balanced set of news articles, which were preprocessed and analyzed using features such as lexical properties, sentiment, and readability. The results showed that fine-tuned RoBERTa models significantly improved detection performance at the sentence level, while traditional models like the Random Forest Classifier performed better at the document level. Her conclusions highlighted the potential for using advanced machine learning techniques to enhance topic-specific misinformation detection.

Jessica Melton (@jessicasmelton) advised by Dr. Erika Frydenlund (@ErikaFrydenlund)

As the fifth presenter, Jessica presented “Impact of Venezuela-Guyana Border Dispute Misinformation on Sentiments Toward Venezuelan Migrants in Guyana”. Her study analyzed over 20,000 YouTube comments using sentiment analysis, focusing on comments related to the border dispute and Venezuelan migrants. The results showed an overall positive sentiment, with a correlation between comments on the border dispute and Venezuelan migrants. During the study, she faced challenges in developing a reliable misinformation detection model, which limited the ability to fully assess the impact of misinformation. Jessica recommended enhanced data collection, improved data cleaning, and the creation of a custom-trained misinformation detection model to further explore this issue.

Ashlyn Farris (@AFarris04) advised by Dr. Michael L. Nelson (@phonedude_mln)

Next, Ashlyn presented “Categorizing Social Media Screenshots for Identifying Author Misattribution”. She examined how author misattribution occurs on social media platforms in the context of spreading disinformation, using screenshots from Twitter, Truth Social, Instagram, and Facebook. Her work involved analyzing various posts, categorizing them based on internal structure, extracting metadata from screenshots, and assessing the effectiveness of different methods for identifying authorship and timestamps to determine the origin of the post. She used metrics such as precision, recall, and F1 scores to evaluate the categorization of posts. In addition to that, she created a dataset containing 16,620 images collected from screenshots of posts on the previously mentioned four social media platforms.

Leah Prince (@leah_princ635) advised by Dr. Michele C. Weigle (@weiglemc)

Then, Leah presented “Examining Methods Engagement on Instagram” focusing on analyzing user engagement on Instagram through web archiving techniques. Her study demonstrated that Instagram mementos from archive.today were scrapable, and emphasized the significance of examining inactive and active accounts to understand social networks better. Her study’s key methods included classifying hashtags and captions, clustering users based on hashtag frequency, and detecting backup accounts. Her findings suggested that web archives could be used to analyze banned social media accounts. Additionally, she indicated that clustering could reveal inter-group overlaps, providing insights into the dynamics of social media engagement.

Caoilainn Christensen (@caoilainnsc) advised by Dr. Anne Perrotti (@slpmichalek)

As the last presenter for the day, Caoilainn presented “Neurodivergence in the Criminal Justice System”. Her study aimed to investigate how experiences differ between neurodivergent (ND) and neurotypical (NT) individuals within the American criminal justice system (CJS). The study involved a comprehensive literature review using Google Scholar and other sources, identifying key themes through text clustering and BERTopic analysis. Her findings highlighted significant gaps in US-based research and illustrated the need for better understanding and accommodation of ND individuals in the CJS. Key themes included the influence of education and parenting, bias and lack of training among CJS personnel, and the importance of understanding culpability. Despite challenges with data noise and numerous clusters, her study indicated the necessity for improved analytical methods and further research to address the disadvantages faced by ND individuals in the CJS.

At the end of the presentations, Dr. Sampath Jayarathna addressed the audience with concluding remarks. Then the students received participation certificates and gift packs from their mentors as a token of appreciation for their hard work and dedication throughout the program.

You can find detailed information on the Disinformation Detection and Analytics REU Programs hosted by WS-DL below.

Third Year (2024)
Second Year (2023)
Inaugural Year (2022)


– Rochana R. Obadage

Comments

Post a Comment