The Case of the Mysterious 0.7-Star Disappearance: A Swiggy Product Manager’s Quest for Answers

Image Source: https://wafflebytes.com/

Problem Statement — As the Product Manager of Swiggy, your task is to investigate the sudden drop in Google Play Store rating from 4.5 to 3.8 over the past two weeks. Your objective is to identify the underlying cause of this issue and develop effective solutions to resolve it.

Disclaimer: The purpose of this case study is to provide educational value by presenting hypothetical situations and potential solutions. It is important to note that this is purely for educational purposes.

Although it may be alluring to immediately devise solutions for the problem, our initial approach will involve breaking down the issue into smaller sub-problems in order to gain a comprehensive understanding of it.

Assumption: Given that we have been tasked with identifying the underlying reason for the decrease in ratings, we will proceed with the assumption that the root cause can be traced back to the mobile application and will focus our investigation accordingly.

A decline in ratings on Google Play Store may be indicative of heightened user dissatisfaction within the last two weeks, although the specific factors contributing to this phenomenon are presently unknown. To delve deeper into the issue, let us first begin by examining the user experience and journey on the mobile application.

Swiggy’s Food Delivery App User Journey

The user journey provides insight into the different interaction points, objectives, and behaviours of users. Any problems that arise from these points of contact could potentially have severe consequences on the overall user experience. For example, a technical glitch during the account setup process for new users could hinder their ability to utilise the application altogether (Product Managers typically possess a thorough comprehension of their users and their journey prior to implementation).

Assumption: Our focus as the Product Manager for Swiggy’s operations in India is centred on issues and factors specific to India. Specifically, our case study will concentrate on the Swiggy India APK.

In an effort to pinpoint potential problem(s), we proceed to brainstorm the listed factors below:

Internal factors refer to the actions of the organisation’s internal components, such as technology, marketing, human resources, and operations. It is essential to pose appropriate questions to stakeholders to gain a comprehensive understanding of these factors.

External factors are beyond the control of the organisation and are not necessarily a result of internal actions. These factors may or may not have an impact on the organisation.

The questions I pose to stakeholders are…

It’s important to bear in mind that Swiggy relies on its partnerships with delivery agents and restaurants to provide food. As a result, our questionnaire takes their experience into account as well.

(Practically speaking, some of these questions may already have answers known to the Product Manager.)

Based on the assumed responses to the questions listed above, the following are noteworthy:

• We have not implemented any significant changes to the app UI/UX within the last three weeks. However, we did introduce Pay-Later payments mode (beta) last week, along with minor updates to the food search algorithm three weeks ago.
• Recently, Swiggy has modified the compensation process for delivery partners in three major states in India. The company now requires delivery partners to maintain a certain number of active hours (by remaining online and available for deliveries) during the day to qualify for bonuses.
• Inflation has caused a surge in fuel and cooking gas prices within the past month.

From the aforementioned responses, we have formulated the following hypotheses:

1. Should the new Pay Later payment mode experience frequent failures, it may result in customer dissatisfaction due to wasted time and repeated attempts at payments.
2. Potential issues with the updated food search algorithm may lead to customer dissatisfaction as it may become challenging for customers to search for the desired food items.
3. Increased commissions have caused Swiggy’s delivery partners to become dissatisfied and log out of the app. This may have created a shortage of delivery partners, resulting in an imbalance between demand and supply and potential delays in deliveries.
4. The rise in cooking gas and fuel prices has negatively affected Swiggy’s restaurant and delivery partners, reducing their margins and causing dissatisfaction. As a result, they may give low ratings to the app. (Note: It’s unclear if the restaurant and delivery partners are also end-users of the app, so I have removed the implicit assumption.)

Now that we have formed the hypotheses, our next step is to utilize the data and analytics available to us. Our objective is to identify any abnormalities in the metrics and reviews.

Metrics/Data/Logic : Given that our purpose is educational, we will make certain assumptions about the metrics and present them as follows.

Analysis of Reviews on Play Store using Qualitative Methods

To address the notable decline in user ratings, our initial approach is to conduct a qualitative analysis of reviews on Play Store. The objective is to identify common themes of dissatisfaction related to specific issues or a combination of issues. To ensure impartiality, the analysis should be conducted by independent research teams. The team may begin their investigation by focusing on reviews with lower ratings (1–3) that include comments.

Assumption — The decrease in application ratings began just two weeks ago and has been declining gradually.

Evident Outcome

1. Over the past 48 hours, approximately 10,000 users have given us a 1-star rating (1 out of 5).
2. A significant number of these users are expressing concerns about average delivery time, the time calculated from the order received, to prepare and pack, to collect and deliver the order. This metric found a 20% dip in the last 10 days.

Analysis of Swiggy’s Social Presence

As there are ongoing strikes in certain cities and states, we will include an examination of prevailing social media and press patterns. Notably, there has been a Twitter trend for the past five days with the hashtag #WhatIsThisSwiggy. This trend has generated 21,000 posts from 19,200 users and has received 195,000 likes.

Evident Outcome

1. Over the past five days, there has been a significant drop of 20% in the number of Daily Active Users (DAU). Half of this reduction is attributed to two major states in India. Furthermore, there has been a decline in the delivery executive’s activity, measured by DAU, by approximately 32% in the aforementioned states.
2. The average waiting time is a metric utilized to gauge the time it takes for a user to enter their desired food item and purchase it. Recently, Swiggy has observed a 10% increase in this metric across the board, with 70% of the increase being attributed to four major states in India. To gain further insights into the rise of the average waiting time metric, we have combined these findings with the ‘Search Timeout’ per user. The results indicate a 40% increase in the Search Timeout metric.

Analysis of Swiggy’s Dashboard Metrics

It’s time to review the metrics on our dashboards to gain insights into how users truly feel.

Evident Outcome

1. Following the introduction of the new pay later payment mode, a bug in the Android application caused transactions under Rs. 300 to go unrecorded, resulting in a 32% increase in the failed order rate.
2. Over the past week, the bounce rate appears to have risen by 8%, with a focus on drop-offs from various points, including the updated entry-landing point at the offers section.
3. There were significant delivery delays and wait times, with a shortage of delivery riders causing users from over 9 major cities to experience a 45% longer wait time than usual.
4. Customers are increasingly dissatisfied with paying extra charges, in addition to the product cost, due to rising inflation. Swiggy has not made any adjustments to the delivery fee, and in some cases, these charges represent 25% to 35% of the product cost. Additionally, the absence of discount offers on the platform exacerbates the issue.

Based on these findings, we can either further develop or dismiss particular hypotheses as appropriate.

1. If the entry landing page was causing significant issues, it would result in a spike in bounce rates, but the rates have remained normal. Moreover, since the page has only been active for a week, it is premature to consider removing it. As a result, we can dismiss this hypothesis.
2. The recent payment gateway software update, particularly for Android users, caused significant frustration and contributed to a surge in failed orders, making it a probable root cause of the problem.
3. The unavailability of riders during peak order times resulted in numerous drop-off sessions and negative ratings, making it another root cause of the issue at hand.
4. The lack of available riders during peak ordering periods resulted in multiple instances of order cancellations and poor ratings, making it a significant contributor to the problem.

Uncovering the Root Cause

After conducting a comprehensive investigation into all potential factors that may have contributed to this issue, analyzing various metrics and sources with both logic and data, we have determined that with a recent update to the payment gateway for Android users, leading to a surge in failed orders, are the root causes of the problem. Additionally, it is clear that the lack of sufficient logistics and manpower to handle the surge in deliveries has frustrated users, resulting in longer waiting times than usual.

My Solutions

1. Handling Issues/Bugs: As we have discovered an issue with the payment gateway, it is crucial to address this problem promptly. We should collaborate with quality analysts and tech leads to develop a plan of action to resolve the issue.
2. Handling the Delivery partners: To control the damage caused by the new compensation system, our initial step should be to confer with the legal team and executives to create a plan. We must emphasize our commitment to delivering top-notch services to our partners.
3. Handle the irrational reviews: Upon analyzing the negative reviews received on Play Store within the last 3 days, we have identified that many of these reviews are baseless and not related to the functioning of the app. As we do not have control over the rating system, we should approach Google to help filter out the irrelevant ratings and remove them.

Once we have developed our solutions, we will refer back to the metrics to confirm the desired improvements.

Disclaimer: The purpose of this case study is to provide educational value by presenting hypothetical situations and potential solutions. It is important to note that this is purely for educational purposes.