Press Pass: Building a Better Aggregator
Review aggregators like Metacritic and GameRankings have a bit of a mixed reputation in and around the game industry. Millions of gamers love aggregators for distilling dozens of reviews into a single number that can aid in their purchasing decisions (and their message-board arguments). Publishers love aggregators for providing a concrete metric by which to rate the quality of their developers and PR units. Those same developers and PR units, of course, often hate aggregators for passing judgment on months of hard work with no real context. And the critics often hate aggregators for reducing hundreds of words full of personality into just another statistic to be lost in the crowd.
A new Web site is looking to solve that last problem, at least, by helping people get to know the critics behind the numbers a little better.
CriticDNA, set to launch later this month, sprang directly from developer complaints about existing aggregators, according to founder Jack Bogdan. "I've been networking with a lot of game developers to build relationships for years, and about a year back, I first got in touch with Q-Games' Dylan Cuthbert in Kyoto," said Bogdan, a 19-year-old student at the Gnomon School of Visual Effects in Los Angeles. "As I started to get to know him, he brought up how fed up he was with Metacritic and how he had an idea that might take them down. It started really small and simple, and has since completely evolved into something else. But that's where things started."
Cuthbert's original idea, Bogdan said, was to let users evaluate each review, promoting the good ones while limiting the effects of the hatchet jobs. "His initial concept was giving more power back to the users in the process, which is something we have built into CriticDNA," Bogdan said. Site users will be able to tag each review on CriticDNA with various positive or negative tags provided by the site, somewhat like the player-review system in LittleBigPlanet. These tags will appear on a critic's profile page, building a reputation system for the best critics right into the site.
"The key is to find things people want to critique fairly," Bogdan said, "Rather than getting the spammers who say, 'Hey, you hated Zelda; now I hate you,' and then they review the critic badly. We're trying to avoid that situation."
In the year since Cuthbert's initial idea, Bogdan said he's built CriticDNA into something more robust than a simple critic-ranking system. Yes, at its core, the site will still serve primarily as a way to get a snapshot of the critical landscape regarding a game. "We're doing a Rotten Tomatoes-style consensus [for each game], because we know some people just want an answer," Bogdan said. "So on the game page ... the consensus bar is right there -- Buy, Rent, Skip -- [along with] one line about why." CriticDNA will also offer a visual summary of the critical responses to a game, arranging reviews as dots on a color-gradient bar representing the range of scores, letter grades or general positivity/negativity of unscored reviews.
It's when you slide your mouse across those review dots that CriticDNA's most interesting and unique features will become apparent, Bogdan says. For each review dot, a small pop-up shows tags that reveal important information about the critic behind the game. These tags include basic demographic information, like age and gender, as well as the aforementioned feedback provided by site users.
More than that, though, there will also be automatically generated tags, like "RPG Hater" and "FPS Fan," that let users know about a critic's gaming preferences. CriticDNA will generate these tags, Bogdan says, based on an analysis of the critic's previous reviews (to seed the system, Bogdan says he and his partner are busy building an archival database of reviews from critics representing the 100 most popular review outlets, covering games from the PlayStation 2 era on). Clicking on a critic's name will bring up a more detailed breakdown of their review history, Bogdan said, along with a graph of their average review scores in each genre and optional, critic-addable RSS feeds from their blogs and Twitter feeds. Suddenly, these aggregated reviews aren't just numbers on a chart, but the products of real people, with real preferences, real reputations and real opinions.
With such detailed data on each critic, Bogdan hopes users will be able to better find and follow critics whose tastes and outlook match theirs. "It eases [readers] into the discovery process," Bogdan says. "You can get an idea of what [critics] like and dislike from the tags, see if they are your age from the demographic info, find out how they write based on user critiques, and better get to know the person through their presence on the rest of the Web, all in one place. It's the little pieces that make up the big picture."
And it's this big picture that Bogdan says he hopes will elevate users' appreciation of the individual personalities that make up those aggregate scores. "I don't think people are given a chance, with all the aggregate systems, to actually get to know anyone," Bogdan said. "Doing the research takes so much time, so we're doing it for you. If you're interested, you can go back for more. I would say there's a reason people go back to sites, and with the ones doing it right, it's personalities. If people really didn't care about the people behind the byline, we wouldn't have seen that huge community uproar surrounding the massive layoffs at 1UP earlier this year."
While Bogdan admits there is still a lot of work to be done before CriticDNA is up and running, he's optimistic that the site will provide an important alternative to the industry-skewing tyranny of the all-important Metacritic score. "That was one of the major reasons we set out to make the site," he said. "It's horrible people's jobs are at risk because of one site. I just hope we aren't used in the same way. At least now people will understand where others are coming from on a basic level."
Check out more Press Pass: