Structural problem (11/11)
Well, this is it. This is surely not the first case of such a ToS violation that played out well, but it's clearly one of the biggest, and we will do a broader analysis of "similar" games using our anomaly detection tool later, in other articles - but for now, let's conclude the results. The NPC mechanic worked and affected reviews by a value between 27% and 50%. The text alone carried 17/27% of the added abnormal reviews. The link to the Steam page opening inside the game in a Steam browser carried 17% out of 27% of the added reviews. Perhaps the Steam-logo head and "Gabe Newbot" play a role too, but there is no way to establish how important it is. The effect took place mostly in positive reviews, without much effect on the negatives, but since the game has a 94% rating, the sample size is not big enough to draw clear conclusions. The revenue-review relation is not established and we can't tell how much additional revenue was acquired, but it is safe to assume it was not negligible. A properly executed Call-to-action can be an effective tool, practically doubling your review count in specific cases. The most interesting part for us is not the calculations themselves - it is the reaction. The majority of people under the original post were defending the game, since it is good. Valve took no visible action nor made any public statement about it. The ToS rules remain vague and their enforcement is questionable. This creates a structural problem, where games that are already popular on release (Far Far West had a lot of wishlists at the start) are incentivized to bend the ToS, gaining a substantial review boost and a non-negligible revenue boost by benefiting from the feedback mechanisms of the system, and largely with the audience's approval. But whether this is a problem to solve or not is not our business. Our job is to continue mining knowledge to help developers with better analysis and prediction tools. Join the Pneumatix waitlist to get early access to our tools and stay in the loop on new articles and updates.
1/1