The Limits to Improvement

Problem

In my previous job, I stumbled across an interesting problem. When conducting user research, I noticed that a certain section of our users couldn’t book a direct flight. At that time, I was leading the air products team responsible for helping users find the best flight deals. Our user research showcased that a significant number of users had to make multiple bookings to reach the desired destination. They had to book two flights separately using two weblinks and going through their user journeys one by one, consecutively.

It was taking a lot of time. Our users were unable to avail offers due to insufficient transaction value with one flight while struggling to find the relevant information and cancellation policies. Spending all that time exploring different possibilities and doing their own research took the fun out of their travel and made it quite cumbersome. Our vision as an organization was to make travel fun, and we wanted to come up with a solution that would make it so.

Actions taken

We looked into our flight data for the last 30 days to learn how many users who booked connecting flights had to wait at the airport for more than three to four hours. We found out that 10 to 12 percent of our users were experiencing that, which indicated a serious problem. To put it more concretely, if 20 000 tickets were booked on our platform, some 2 000 people were going through this.

To better understand the problem, we had to take a look at different things: flight schedules, prices, shortest travel time to an optimal price, etc. We did some research, collected some rudimentary data, and came up with a crude equation that stated that every 15 minutes of waiting equals 15 Indian rupees. We built an algorithm that could tell us the money value of time. For example, if we could help our users save time, how much money they would be willing to attribute to that, or how much time the users can forgo for a fair price. Obviously, some price-conscious users would be willing to trade their time for money.

Also, a portion of our users struggled to discover multi-airline flights. So to address all those problems, we built an API that would show users direct flights, but also other opportunities with connecting flights showcasing how much money they could save in return for waiting longer in transit.

However, we encountered a number of challenges we had a hard time dealing with. For example, terminal changes were frequent and abrupt. Also, terms and conditions were different for different airlines -- some would allow 20k baggage and others 10k, some would give 100 percent refund for cancelations, while others would refund zero. On top of that, we had some technical difficulties related to latency. When a user would try to fetch all the flights from the different airlines, all those permutations and combinations were slowing the API down.

When we finally launched the product in the pilot phase, a number of complaints were filed by users who were unaware of the challenges described above. Some of the problems we managed to remedy. Every time a user would try to book any multi-airline itinerary, a pop-up would appear with information of baggage and cancelation policies, and only after a user would check it off would they be able to move forward.

Our product was initially designed to be used in India, but we soon expanded it to international flights for Indian users. We introduced visa terms and conditions to address the main pain point of Indian travelers. By expanding to international flights, we managed to make a conversion jump of 6 to 7 percent in our flight consumer journey.

Lessons learned

• We could have done user testing much better. If we had done it more extensively, many of the user complaints received in the pilot phase could have been circumvented.
• Over time, the product didn’t gain much traction. At best, ten percent of our audience was using it, which was in stark contrast to the forecasted 80 to 90 percent. Maybe we failed at marketing or were unable to explain our value proposition clearly. We did a concept validation and user testing that was promising, but things simply didn’t work out in the end. Though it was an average product with average chances for success, it didn’t work as planned.