Recap: International Conference on Computational Social Science2024-09-23

Note: This post originally ran on the IDPI blog.

A few weeks ago, several hundred academics, students, and professionals gathered at the University of Pennsylvania for the 10th International Conference on Computational Social Science: two-and-a-half days packed with presentations and posters. This post attempts to highlight some of the work I found most exciting and briefly summarizes the presentations Kevin Zheng and I gave about some of iDPI’s YouTube work.

Two iDPI-related presentations

There’s a lot going on with YouTube at iDPI. We published a paper called Dialing for Videos back in December which detailed our approach to creating random samples of YouTube videos, and we’re in the process of building on that work (including some projects that will have to wait for a different blog post).

By the time Dialing for Videos was published, the dataset it described was more than a year old. Academics, journalists, policymakers, and other stakeholders need better, faster access to basic statistics about YouTube. TubeStats.org is our answer to that—a website which provides fundamental information about YouTube based on periodic samples. Kevin’s presentation showcased this tool, its development, use cases, and the methodologies it builds on.

A pipeline to randomly sample YouTube means we can start studying broad swaths of YouTube. For example, how does YouTube vary by language? I presented about our research project to compare language-specific random samples, detailing the methodology and preliminary results. It’s work we’re going to publish soon, but here’s a spoiler: Hindi YouTube is very different from English, Spanish, or Russian YouTube when you’re just looking at metadata.

Highlights

There was a panel on the big 2020 US Election study on Facebook and Instagram, including Talia Stroud, Chad Kiewiet of Meta, and moderated by Kevin Munger. The project was a vast, sprawling undertaking involving dozens of researchers and Meta employees to better understand the use of Meta’s platforms during the 2020 elections. It involved a wide range of data from millions of users, and has already resulted in five papers (with more on the way). The most interesting part to me wasn’t the [many] results but the discussion about the logistics, concerns, principles, and potentially conflicting values which arise when independent researchers and for-profit companies collaborate to study the latter (frankly I wish the panel focused on those thorny issues).

YouTube Shorts remain understudied, but I was glad to see one session focused on politicization on the platform. Inspired by an Atlantic article about Andrew Tate, Ashton Anderson and George Eilender sought to understand differences in left-wing/right-wing content engagement. The bottom line is right-wing content in their sample received a lot more attention. The implications of that are unclear. Some speculation during Q&A about the relationship with TikTok, which some perceive [without particularly compelling evidence, I’d add] as leaning to the left. Some tangential, but useful talk about the role of the YouTube algorithm being potentially different on Shorts because it’s a steady stream of videos rather than a set of recommended videos for the viewer to choose from.

A study by Khushang Zaveri, Lyle Ungar, Sharath Chandra Guntuku, Shreya Havaldar, Soumna Nema, and Sunny Rai examined how shame is expressed in Hollywood vs. Bollywood movies. They found shame was expressed more frequently in Bollywood. The authors saw more “internal shame” (remorse, past-focused) in Hollywood and “external shame” (anger, present-focused) in Bollywood. Shame about money, cowardice, lying, etiquette, incompetence in Hollywood; shame about gender and academic performance more common in Bollywood. A missing piece of the puzzle is probably to look at the different ways shame is expressed rather than using variations of the word “shame,” but it’s an interesting keyword-based cross-cultural project.

Apparently people do not know that local journalism is in bad shape in the US. A paper by Chris Callison-Burch, David Rothschild, Duncan J. Watts, and Samar Haider began with the observation that many local newsrooms owned by a large parent company will reuse content. They used cosine similarity to find duplicates or near-duplicates, then looked at the newspapers to see how many articles they share with each other. Finally, they found that among newspapers that share a considerable amount of content, the overlap often extends to URLs. 8.6% of local newsroom URLs were duplicates.

One of my favorite presentations was technically called “When Curiosity Gaps Backfire: Effects of Headline Concreteness on Information Selection Decisions” from J. Nathan Matias and Marianne Aubin Le Quéré, but I prefer the unofficial title, “The Anatomy of a Clickbait Headline.” Working with Upworthy, practically synonymous with clickbait engineering, they propose a “concreteness metric,” measuring the degree to which a concept denoted by a word refers to a perceptible entity. Using a suite of concreteness dictionaries, headlines can be processed in large quantities to evaluate the extent to which they are purely informative or clickbait.

Perhaps the most useful presentation for our purposes was the exquisitely titled Careless Whisper from Allison Koenecke, Anna Seo Gyeong Choi, Hilke Schellman, Katelyn X. Mei, and Mona Sloane. The researchers tested OpenAI’s Whisper tool on a dataset of audio recordings of people with forms of aphasia. The goal was to test the extent to which Whisper hallucinated text when transcribing a video, observing that it seemed to hallucinate more during pauses. These hallucinations sometimes included bizarre violence, sometimes included “YouTuber speak” about liking and subscribing, among other errors. 1% of transcripts had such hallucinations, though the authors found that running the same audio through multiple times and removing the areas of difference caught most of them. Thankfully, Koenecke said afterwards that the most recent version of Whisper only hallucinated in 0.1% of transcripts. Still something we’ll have to be mindful of as we integrate Whisper into our YouTube pipeline.

Quick takeaways and observations

  • There’s an awful lot of work studying ChatGPT (or studying with ChatGPT, which is sometimes concerning).
  • There was a fascinating poster about a project that showed the predictive potential of Wikipedia pageviews to detect migration events (Carolina Coimbra Vieira, Ebru Sanliturk, & Emilio Zagheni).
  • Best Title Award: Careless Whisper (see above – *chef’s kiss*)
  • In Chinese social media discussions of the Russo-Ukranian war, posts were generally neutral, but Russian websites had the greatest influence. (Hans William Alexander Hanley, Jennifer Pan, & Yingdan Lu)
  • Someone – I think it was Kevin Munger – used the phrase “aesthetically beautiful science” to describe the Meta elections project. #researchaspirations
  • People in Korea talked more negatively about refugees after a confluence of events led to Yemeni refugees arriving at Jeju Island in 2018. The attitude shift in discourse, which was measured on Twitter and in YouTube comments, was consistent across the political spectrum. (Chloe Ahn & Junghyun Lim)
  • Elizabeth Bruch’s keynote on internet dating research covered a lot of [often entertaining] ground. Think your city is particularly bad for dating? Turns out cities are mostly the same in terms of number of dates people go on – they differ in the number of messages being sent.
  • Google Scholar citations can be gamed. Talal Rahwan’s team created a profile for Hubert J. Farnsworth from Futurama full of conspicuously bogus papers that cite each other repeatedly and watched him climb the citation ranks.