Do Incentivized Reviews Poison the Well? Evidence from a Natural Experiment at Amazon.com

Authors

Jaecheol Park
Arslan Aziz (Meta Platforms, Inc.)
Gene Moo Lee (University of British Columbia)

Abstract

The rapid growth in e-commerce has led to a concomitant increase in consumers’ reliance on digital word-of-mouth to inform their choices. As such, there is an increasing incentive for sellers to solicit reviews for their products. The literature has examined the direct and indirect effects of incentivized reviews on subsequent organic reviews within consumers who received incentives. However, since incentivized reviews and reviewers are often only a small proportion of a review platform (only 1.2% in our sample), it is important to understand whether their presence and absence on the platform affect the organic reviews from other reviewers who have not received incentives, which are often in the majority. We theorize two underlying effects that incentivized reviews can generate on other organic reviews: the herding effect from imitating incentivized reviews and the disclosure effect from the increased trust or skepticism by explicit incentive disclosure statements. Those two effects make organic reviews either follow or deviate from incentivized reviews. Using Bidirectional Encoder Representations from Transformers (BERT) to identify incentivized reviews and a natural experiment caused by a policy change on Amazon.com in October 2016, we conduct difference-in-differences with propensity score matching analyses to identify the effects of banning incentivized reviews on organic reviews. Our results suggest the disclosure effects are salient: banning incentivized reviews has positive effects on organic reviews in terms of frequency, sentiment, length, image, and helpfulness. Moreover, we find that the presence of incentivized reviews has poisoned the well for organic reviews regardless of the incentivized review ratio and that the effect is heterogeneous to product quality uncertainty. Our findings contribute to the literature on online review and platform design and provide insights to platform managers.

Keywords

Incentivized reviews, online reviews, disclosure, herding, platform design

Presentations

• UBC Sauder MIS Seminar Series 2021
• KrAIS Research Workshop (KrAIS) 2021
• Workshop on Information Systems and Economics (WISE) 2021
• Pacific Asia Conference on Information Systems (PACIS) 2022
• Boston University Platform Strategy Research Symposium (BU Platform Strategy) 2022
• Conference on Information Systems and Technology (CIST) 2022
• Bright Internet Global Summit (BIGS) 2022