The dangers of in-game data collection

Nick Yee sells secrets.

He knows what you like, what you don’t, and more importantly he understands why. He knows what motivates you, what stops you from achieving your best, what turns you off, and everything in-between. At least, when it comes to games. He’s spent years surveying more than 200,000 gamers through voluntary questionnaires, gathering information about their ages, their likes, their dislikes, and their preferred genres.

And he sells that knowledge to developers. Some change their games based on this knowledge so you’ll buy them, play them, and tell others to do the same.

Yee started a business in 2015, Quantic Foundry, that sells his data to companies including League of Legends owner Tencent, Plants vs Zombies studio PopCap and Magic: The Gathering publisher Wizards of the Coast.

“Historically, game development was disentangled from actual data,” says Yee. “When people were still playing on non-internet connected consoles, the developers didn’t get that data back. You didn’t really have a clear view of how people were playing their games.”

Now, developers say, they have plenty of data, both from product telemetry (players’ in-game behavior) and external sources (like Yee’s surveys). And some are getting concerned they may have too much.

The plethora of password leaks from social networks, businesses, and other sources makes privacy in games more important than ever, many developers say. Additionally, the rise of interconnected government systems, such as China’s social credit system, raises questions: can your behavior in-game affect how you’re treated in the real world?

The first type of data is the information Yee collects about gamers. It’s broad, anonymous, and helps developers pinpoint commonalities and personality types. For instance, Quantic Foundry’s software lets developers pick a game like Civilization, then see a chart that outlines how that game’s players are motivated.

The second is much more specific. How do players interact within a game? What choices do they make?

That information can be used to make better games. And it can also be combined with other types of information to build robust personal profiles. Those personal profiles are typically used to target advertising, but privacy experts warn that in the future, that information may be used in sinister ways we can’t expect.

“Sometimes infrastructure of data collection is built up for one purpose … then people start to think of other uses for it,” says Jay Stanley, a senior policy analyst at the American Civil Liberties Union.

And if that ends up being the case, they say, developers should start building games in ways that stop that manipulation before it occurs.

Dealing with data regrets

Many games, if not most, are built with systems to track how players use them. The developers can then use that information to alter storylines, change difficulty levels, and help justify new content.

That data is usually isolated. For example, an XCOM game may track whether you choose between two missions. But it’s hard to deduce much about the player’s personality based on that simple decision.

Where things get more precise is that, in some cases, games have been collecting more personal data. Choices that reveal things about the player’s personality, like dialogue choices or even literal personality tests, get recorded and stored.

Privacy experts, and some developers, are now afraid that information can be connected to a patchwork of online services and used in suspicious ways. That’s why if Silent Hill: Shattered Memories launched today it would be different, says writer Sam Barlow.

“There would definitely be additional challenges [in how we’d develop it],” he says.

The game opens by asking players to take a psychometric evaluation, and the game’s content changes based on those answers. For instance, Barlow says, some players will always have respect for authority figures. If the Silent Hill personality test detects that, players will see a policeman who at first presents as an ally — but is ultimately belligerent and rude.

Players who inherently distrust authority figures — according to their psychological test — will see a caring, helpful police character who genuinely cares ... and then leaves the player to fend for themselves. This approach uses personality traits to subvert expectations and heighten drama, Barlow says.

“I was capturing all this data and then analyzing it later, and it honestly felt like you were spying on someone,” Barlow says.

Barlow remembers showing the game at E3 and watching players become more nervous as they read questions like, “Have you ever cheated on a partner?” At the same time, Barlow says, it was exciting to see the interactive storytelling emerge in a way that was “direct and personal.”

Yet in the years since, Barlow has questioned how he can create interactive stories that keep data anonymous.

“It makes you think twice about what information you’re collecting,” says Barlow.

“There’s a sequence in Shattered Memories where you’re walking down a corridor, and it normally takes about 15 seconds. But there is a conversation between the main character and his wife for 30 seconds — the game tracks if you listen or ignore it,” he says.

“Based on that, and some other variables, it gives you the ending that you deserve.”

It’s these types of miniscule decisions that flesh out a psychometric profile of a player. While this isn’t a regulated psychological test by any means, Barlow says, the consequences of collecting the data are becoming starker.

“What happens if in 10 years you don’t get a job because the game reveals you’re not a team player?” he says.

“I’ve spoken to marketing people who are really interested in getting into interactive storytelling, and the reasoning there is traditionally a lot of money tied up in it,” says Barlow.

Netflix experimented with “choose your own adventure” style narrative in 2018 with the release of an interactive “Black Mirror” episode, in which the company records decisions made by viewers. Tracking user choices also plays a huge role in marketing: by gathering many tiny decisions made by viewers (or players), companies like Netflix can build profiles about not just their personalities but even the types of products and services they like or dislike. That data can also be sold.

Netflix has known about your TV habits for years. What’s different now is how microscopic these systems are becoming. Rather than knowing whether you prefer Gilmore Girls or Breaking Bad, Netflix — using interactive narrative — can now understand: do you want the storyline where Walter White kills his enemies, or lets them free?

“It’s so easy to extrapolate any data point. If you have a husband and wife, tracking when they play games could result in an analysis of how long they spend time together. You could measure the health of that relationship,” Barlow says.

Those assumptions make these surveys all the more exciting, and dangerous, says Barlow. And it means making a game like Silent Hill: Shattered Memories comes with moral questions about what data is collected, and why.

“If this was on an iPhone,” says Barlow, “in theory you’re attaching your game with a psychological profile.”

There is no anonymous data

Privacy experts and developers note that every piece of data a company records can be cross referenced against other databases. Lone data points such as decisions made within a game may be benign when isolated, but combined with other sets become powerful representations of behaviour and psychology.

This is a fear that, in part, drives Josh Sawyer, design director at Obsidian. He worked on Fallout: New Vegas, which also used a type of personality test on players.

Like Barlow, Sawyer says he’d change design decisions if New Vegas was released today.

“We would think about it differently,” Sawyer says. “We look at putting telemetric data collection in all of our games, but we always have conversations about data security. We ask people if they want to opt into data collection by default.”

While Sawyer is quick to point out the test was not based on any psychological framework and was only intended as a way for players to pick their characters’ traits, the test still serves as an interesting questionnaire. For instance, players are asked whether they prefer certain traits — like honesty — over others such as humility.

“There was no psychological rigour behind it,” he says, but he notes this is where the problem falls.

If this information is collected and used in future algorithms, the idea that programs, apps, or services could infer psychological data based on tests that have no psychological rigor puts an enormous weight on game designers, says Barlow.

“One of the more frightening things is seeing Amazon algorithms that could figure out if a woman were pregnant or if a man were gay ... and not from anything that was overt in what they were doing, but from these secondary things that no one would have seen on their own and figured it out,” says Barlow.

In one infamous case, Target sent one teenager a catalog with baby products by predicting she was pregnant based on previous shopping behaviour.

“That’s the sort of stuff where I feel like we need to be very very conscious and cautious about what data we’re collecting and what data we’re sharing because it can really be used in ways that go far beyond what it appears to be on its face.”

Nick Yee says data corresponding to personality traits gathered during these tests would be fairly crude and generally unreliable, but whoever has it may understand that and use it anyway.

This is not a hypothetical. The Chinese Government’s recent introduction of a social credit system relies on thousands of data inputs, including frivolous spending, or even playing too many video games. In the future, one city government has said it will impose penalties for in-game cheating.

Nintendo’s Miitomo app has also experimented with this type of data collection, asking users about their preferences for different products. While Nintendo said it would not sell the information to third parties, it was possible for users to see relevant ads on other Nintendo devices connected by a digital account.

Yee says it’s not a stretch to believe future credit systems like those in China could include data points like in-game decision-making.

“If you repeatedly, when given the chance to take a less aggressive solution, take the more risky option ... does that say something about your credit readiness to a lender?” says Yee.

Breaking the barrier between the real world and the digital world

“If we were doing any telemetrics now, I would be cautious about the information from games being used and collected,” says Sawyer.

“One of the things I’d be very concerned about in a game like New Vegas is [...] we allow players to make choices that are really dark and bleak.”

This isn’t a problem limited to AAA developers. Michael Hicks, an independent developer, released his game The Path of Motus in early 2018. The game experiments with player responses to bullying techniques.

“Almost everyone immediately reacted to aggressors with violence, even though the game never tells you exactly how to play,” says Hicks.

“A very small percentage of people immediately felt bad about using violence and attempted other solutions [...] while most folks had an “aha!” moment about 50% through and then changed their tactics.”

Hicks says while his game uses minimal player tracking and isn’t connected to personal profiles, he was thinking during development that the information collected represented a type of survey. Though, like New Vegas, the test was not psychologically rigorous or even housed in a personally identifiable database.

Calli Schroeder is an associate attorney in the Data Transactions and Security and Intellectual Property practices at Lewis Bass Williams & Weese. Her position is that regardless of intent, this type of data can be collected and combined with other information.

“There’s just a lot of connections that people don’t think of, and because the tech develops so quickly those connections are easier and cheaper to make now than they were even five years ago,” she says.

Data and addiction

Alex Champandard is an AI expert and a prolific developer, having worked at Rockstar on Max Payne 3, and at Guerrilla on the Killzone series. Far from calling himself an alarmist, he says the creation of psychological profiles based on gaming data is simply the evolution of games and business.

“The privacy side doesn’t worry me so much,” Champandard says. “The big risk is that it becomes weaponized addiction, that you design a game to manipulate someone’s physiology and dopamine responses with content.”

“Sometimes that’s to sell you and put you in the game. But if we combine that with procedural systems, we can basically make a perfect storm,” says Champandard. “Imagine micro-targeted cigarettes that could deliver the cigarette right in your fingers the minute you’re feeling the most vulnerable.”

Ubisoft — where Yee was formerly an in-house data expert — has gathered information on Assassin’s Creed for years by asking players directly to grade specific missions after finishing them. That type of information could be packed in multiple ways to entice users to return.

Yee is packaging up information given by players themselves, and using those as types of “personas,” or broad profiles that developers can use to better target their games. The information is anonymous, and has no emails or names attached, but it can inform hyper-targeted advertising campaigns.

This is the valuable information Yee trawled through, including information made public by Blizzard through the World of Warcraft “Armory.” This hive of data was made public years ago, and allowed any user to view a character’s name, details about past activities, the class they preferred to play ... even how many hugs they had given in-game.

Yee says he can’t reveal much about specific work with clients, but cites one case where his company helped with a game made by Codename Entertainment, Crusaders of the Lost Idols. The game was an “idle clicker,” meaning it didn’t require much input other than players clicking a button.

Using his player models, Yee was able to show the company that the majority of “idle clicker” players also enjoyed more complex games like Diablo 3 and EVE Online.

Analyzing public Steam data helped Yee and his co-founder come to the conclusion that idle clickers and these games have something in common: the feeling of progression as players progress from one level to the next. That emphasis on progression informed Codename’s marketing campaign, and the result was double digit sales growth.

As Yee points out, this is fairly standard user research behaviour.

What isn’t standard, he says, is how this digital data is being used to create footprints of players’ activity and to inform an intricate web of sales funnels.

In a 2011 academic article documenting an experiment within World of Warcraft, Yee even raised the possibility of privacy concerns for online worlds.

“... before the WoW Armory, players could make the case that they had a reasonable expectation of privacy in WoW,” Yee wrote.

“... this expectation is no longer reasonable.”

A look ahead

One of the barriers to creating psychological profiles in games, Yee says, is that many people play different types of games for different reasons. Translating behaviour from one game to the next often doesn’t fly.

“There are some games with limited customization, so you can’t understand someone’s preferences,” he says. “Others it’s easy.”

“But the bigger problem is translating that behavior [...] in some games you may be more competitive, for example, than in others.”

Champandard says we shouldn’t become complacent, though. That the moment when in-game behaviour becomes targetable is coming faster than we imagine … and artificial intelligence will help it along.

“What AI programmers have been doing for many years can be plugged in and used for more data,” he says.

“If someone was to do it that way [...] it would have big consequences.”