Fault in your stars: An Analysis of Android App Reviews

• Main Authors: Aralikatte, Rahul, Sridhara, Giriprasad, Gantayat, Neelamadhav, Mani, Senthil
• Format: Journal Article
• Language: English
• Published: 11-08-2018
• Subjects: Computer Science - Computation and Language; Computer Science - Learning
• DOI: 10.48550/arxiv.1708.04968

Mobile app distribution platforms such as Google Play Store allow users to
share their feedback about downloaded apps in the form of a review comment and
a corresponding star rating. Typically, the star rating ranges from one to five
stars, with one star denoting a high sense of dissatisfaction with the app and
five stars denoting a high sense of satisfaction.
Unfortunately, due to a variety of reasons, often the star rating provided by
a user is inconsistent with the opinion expressed in the review. For example,
consider the following review for the Facebook App on Android; "Awesome App".
One would reasonably expect the rating for this review to be five stars, but
the actual rating is one star!
Such inconsistent ratings can lead to a deflated (or inflated) overall
average rating of an app which can affect user downloads, as typically users
look at the average star ratings while making a decision on downloading an app.
Also, the app developers receive a biased feedback about the application that
does not represent ground reality. This is especially significant for small
apps with a few thousand downloads as even a small number of mismatched reviews
can bring down the average rating drastically.
In this paper, we conducted a study on this review-rating mismatch problem.
We manually examined 8600 reviews from 10 popular Android apps and found that
20% of the ratings in our dataset were inconsistent with the review. Further,
we developed three systems; two of which were based on traditional machine
learning and one on deep learning to automatically identify reviews whose
rating did not match with the opinion expressed in the review. Our deep
learning system performed the best and had an accuracy of 92% in identifying
the correct star rating to be associated with a given review.