We value: Our readers.
Our independence. Our region.
The power of real journalism.
We're reader supported.
Get our newsletter free.
Help pay for our reporting.
Culture
  |  
Media

Hey Tyee Commenters! Scholars Studied You. Here’s What They Found

Researchers sifted 1.5 million comments on three sites to gauge their value.

By Vasundhara Gautam and Maite Taboada 6 Nov 2019 | TheTyee.ca

Vasundhara Gautam works as a computational linguist at Dialpad. The work reported here was carried out while she was a member of the Discourse Processing Lab at Simon Fraser University.

Maite Taboada is professor of linguistics and director of the Discourse Processing Lab at Simon Fraser University.

Many of us think that thought-provoking and constructive online news comments are rare and that comments do more harm than good. Uncivil online comments, especially those responding to articles about sensitive issues such as climate change, immigration or Indigenous people, erode the public’s trust in media and can lead to harm in the real world.

And yet, news organizations, journalists and media analysts believe in the power of comments to engage communities, create meaningful discussions and to include diverse voices in the public discourse.

Our research group at Simon Fraser University is studying online comments to distinguish the constructive from the toxic. We were pleasantly surprised to see people expressing different viewpoints, providing evidence to support their opinion, sharing personal stories and experience, attempting to inform, convince or better understand the other side, and overall engaging in a meaningful conversation. We have also, sadly, read many vile and insulting comments.

Our in-depth analysis of more than 1.5 million comments gives us hope, while at the same time providing insights for how to make comment sections better.

Constructive or toxic?

We define constructive comments as those that are relevant to the article, make well-reasoned arguments and add meaningful discussion. On the opposite end of the spectrum, toxic comments contain insults, profanity or attacks on the authors or people mentioned in an article.

Using automatic text analysis methods, we have classified online news comments from three Canadian publications: the Globe and Mail, The Tyee and the Conversation Canada. We found that the majority of comments are non-constructive, but also not toxic. They tend to be short and do not contribute much to the conversation. Versions of "I agree" or "I disagree" make up the vast majority of these comments.

Interestingly, we also found that a large number of comments are constructive and toxic at the same time. They contain well-reasoned arguments as well as hateful language. These comments sound condescending and often open with an insult ("You are all idiots" is a mild example), followed with reasoning.

Readers of The Tyee will be pleased to know that this publication has many more constructive comments than the average of the three. There are consistently more constructive than non-constructive comments (except for many of the "I agree" variety, which are neither constructive nor toxic). Few comments on Tyee articles have 80 per cent toxicity or above. There seems to be a good community of Tyee commenters participating in robust debate. Or perhaps there are good moderators at The Tyee. Or perhaps both.

Topics matter

We used topic modelling, a technique to discover the themes or topics in large bodies of data. We then checked which topics receive the most comments, across all three publications. Unsurprisingly, we found that the three top topics with the most comments are about politics at different levels (local, national, international). What was interesting, though, was that those topics tended to include mentions of individuals and abstract words, such as "talk," "problem," "life," "family" and "community." It seems that people comment more about politics than other topics, but they bring in personal experience and evidence when they do so.

Our next question was whether the topics that emerged from our data showed different levels of constructiveness and toxicity. The answer is "yes" for constructiveness. Some topics lead to more or less constructive debate, with general politics or international politics having more constructive comments, as opposed to local politics or policy. It’s possibly easier to be constructive when topics do not affect the commenter so directly.

For toxicity, what we found is that toxicity is an equal-opportunity offender. Unlike constructiveness, toxicity seems to pattern in roughly the same proportion across topics. On average, the toxicity across all topics and across all three publications appears to be about 23 per cent. In other words, the average comment is one fourth toxic in its content. We expected some topics to be more polarizing than others, but instead we found not much fluctuation between topics, suggesting that toxicity in comments is likely just a feature of online language.

In conclusion, we can say that a large proportion of comments do contribute to the conversation, participating in healthy debate. It is also somewhat encouraging to see that there are small numbers of toxic comments (with the caveat that all three publications moderate their comments).

What can news organizations learn from this? We believe it is useful to find that many commenters are contributing thoughtful discussion. Many comments seem to be well reasoned, but open with an insult, which probably deters engagement with the reasoning and only leads to engagement with the insult. Nudging commenters to think of constructiveness as a feature of commenting may lead to more productive discussion.

For further detail on our findings, please see the full report.  [Tyee]

Read more: Media

Share this article

The Tyee is supported by readers like you

Join us and grow independent media in Canada

Get The Tyee in your inbox

Tyee Commenting Guidelines

Do not:

  •  Use sexist, classist, racist or homophobic language
  • Libel or defame
  • Bully, threaten, name-call or troll
  • Troll patrol. Instead, downvote, or flag suspect activity
  • Attempt to guess other commenters’ real-life identities

Do:

  • Verify facts, debunk rumours
  • Add context and background
  • Spot typos and logical fallacies
  • Highlight reporting blind spots
  • Ignore trolls and flag violations
  • Treat all with respect and curiosity
  • Stay on topic
  • Connect with each other

LATEST STORIES

The Barometer

What do you think of a universal basic income?

Take this week's poll