The Role of App Reviews in Mobile Engineering
Inevitably in your career in mobile application development, you’ll be asked about user reviews. App reviews are often the only outlet users have for giving feedback on an experience, and because of that, the mobile component of a service will often become the focus of a large amount of scrutiny. Today we’ll talk about how to positively an effectively use app reviews to improve your service, and maybe more importantly, how not to.
Reviews are not Always reviews
Since app reviews are often a user’s only outlet for giving feedback, they will frequently use to review to express concerns and problems with some element of a service that has nothing at all to do with the app they’re reviewing. Users will leave an app review because the website doesn’t work, or a customer service agent couldn’t help them, or their account is locked. Although these issues have nothing to do with the app, they’re all things I’ve seen app reviews focus on. It’s a great idea to triage these problems and reach out to users to try to help them, but certainly this falls outside the purview and responsibility of the engineering team. The most user-focused organizations might have entire teams dedicated solely to analyzing and responding to app reviews, or sometimes the support or product departments will handle it. Developers simply do not have the skills or access to do this effectively.
Reviews are not Analytics
Obviously, there are app reviews that actually review or provide feedback on the app itself. These can be incredibly valuable for finding intermittent or device specific issues that aren’t easily reproducible in the lab. But, as unpopular a position as it may be, devoting engineering time to address feedback in user reviews is generally a losing battle. The rationale for this is simple numbers. If you have a million users and 10 developers, you simply cannot devote developer time to every problem in every bad review (for math reasons). To be frank, it’s unrealistic to expect an engineering organization to devote any development time to directly address problems evidenced only by user reviews (some much more likely candidates for finding and fixing production issues are Analytics, Crash Reporting, and Beta Tests).
However, for better or worse, app reviews are the most visible and easily accessible metric for measuring the success of your app, and they will inevitably be used for that, regardless of the statistical insignificance and utter unreliability of the source. So you will almost certainly have to address app reviews in some way.
What to Do
The key to making efficient use of reviews is developing trends. Identifying user review trends is the only way to prioritize issues and smartly allocate development effort. There are lots of tools out there to help you do this. The Google Play Store Console does its best to categorize reviews for you, and the benchmarks section is actually pretty interesting:
In it, the console identifies topics using keywords in reviews. For instance, if a user mentions “login,” “account,” or “username,” that review gets lumped into the account topic. This can be a really efficient way to spot trends, or at least identify areas that might need some extra attention.
Of course, there’s always the tried and true method of just reading through the reviews and categorizing them yourself. Having a team dedicated to this can reveal surprising insights about your app, and the issues users are facing. This is arguably the best way to spot trends, but also the most time and resource intensive.
Takeaway
The point of all this is that, user reviews are frequently misused and over-valued by certain parts of technology organizations, but they can be used to effectively diagnose broader issues. You’ll need to work with your stakeholders to identify thresholds and metrics that you agree constitute an actionable problem, and develop ways to monitor and measure them together. You’ll never get out from under the shadow of user reviews, but you can use them to shine a light on your product in a positive way.