How to fix Twitter’s trending topics? Curation

So the web was abuzz yesterday over some articles focused on Twitter’s “trending topics” feature, in which the most popular words, phrases, hashtags and such are automatically listed in your sidebar, showing you what’s getting the most chatter on the service in near-real-time. In this case, the “trending blog post” was this one: Why Twitter Needs to Do More to Save Trending Topics.

The article goes into what trending topics are and how they’ve been heavily gamed in the past (and present) to get certain topics bubbled up to the top of the list. The writer, Adam Ostrow, also suggests some options of what Twitter could do to fix the situation, namely:

• Make the topics less obvious throughout Twitter, so you don’t see the silliness
• Automatically hide hashtags (the stuff with the # symbol at the start of a unique word or phrase)
• Increase the size of the topics list to perhaps 50 or 100 items to expose more intelligent stuff

These are not solutions to the core problem.

The core problem is what Ostrow points to at the end of the article:

…as a company that’s stated goal is to “[ask] what’s happening and [make] the answer spread across the globe to millions” and has the power to shape public discourse, Twitter needs to do more to balance what its diverse set of users want with what it should see as a responsibility to be a comprehensive guide to what the world’s talking about.

Ostrow, however, points out in the paragraph before this one that editors should not have a hand in shaping the trending list — it should be an expression solely of the users, as calculated by algorithm, and that’s what makes Twitter (and other services) unique.

You sure about that?

Because here’s the list from late Monday night, with my own explanations…

• Jick Nonas — a play on Nick Jonas, the teen singer
• #thatswhyyoursingle — a viral joke-telling thing in which that’s the punchline
• #pleaseexcusemy — another viral humor thing
• #stoptalkingabout — yet another viral item, not so funny
• Craig Sager — sportscaster at the Lakers / Suns game tonight
• Mbenga — basketball player in the Lakers / Suns game tonight
• Lakers won — outcome of the Lakers / Suns game tonight
• Cerati — Argentinian singer that had a stroke
• Game One — tonight’s big Lakers / Suns basketball game was the first in a series
• Grant Hill — basketball player injured by Kobe Bryant tonight

Of these 10 items…

• 40% are nonsensical wordplay games that may be fun, but aren’t particularly insightful as to what is being discussed on Twitter
• 10% are about an injured popular singer
• 50% are about the Lakers / Suns basketball game (and wouldn’t it be better to have 1 entry for everything related to the game? A sort of trending topic cluster?

A modest proposal: Editors

Trending topics lists today are sometimes interesting, but mostly useless because they get clogged with these nonsense games and deliberate attempts to reshape the topics or traffic around them. There’s limited newsworthiness or discussion dynamics. We need an edited list, crafted through a combination of algorithm and human review.

Twitter can continue to share a raw trending topics algorithmic listing for the public. That’s fine. It’s sort of mathematically interesting. But we also need an edited list, in which the trending topics raw numbers are passed through a human filter and lightly organized.

Using the list above, you’d get a cluster of Lakers / Suns basketball and the Argentinian singer, each with a corresponding list of tags or words that seem to be closely related. But then the editor would delve deeper into the data, exposing more human conversation and topical interest amidst the sea of “gamed” tweets.

The result? A rapidly-updated (though not real-time) listing of what people are discussing online, presented in a way that makes sense and shows the meaning behind the tweets (where possible).

How to do it

• Hire editors/curators to maintain the service 24×7 in English (maybe other languages later)
• Editors update the trending topics using the top 250 items out of the algorithm
• Those topics are narrowed to perhaps 25 items, which includes very popular singular terms or clusters of related terms taken from the prior 15-minute interval
• Each topic listed is a clickable link
• A clicked topic link goes to a short-lived topic page in which the highest topic name and all related topics are listed and the most recent tweets are also shown
• This topic page can include advertising that’s relevant to the keywords or topic involved
• Topic listings are repeatedly updated every 15 minutes

You could also get fancier, with more sophisticated algorithms, a broad crowdsourcing process for topic analysis and aggregation and so forth. The advertising could pay for the editor costs and then some.

//

I know engineers want to code artificial intelligence into all the systems out there to make them scalable, but sometimes that just doesn’t get the job done.

Today, I ignore the trending topics because they’re useless and sometimes even offensive. But a well-curated / well-edited trending topics list would be fascinating. Twitter could make money from the listings, of course, and the service would be more relevant to more people more of the time.

Twitter: you can keep the raw numbers, but we need a useful listing of what everyone’s talking about.