Dealing With the Catastrophic Failure of an Important Product
18 May, 2021

CTO at 95 Percent Group LLC
Problem
I’ve been a part of EdTech for many years, watching education evolve with technology as the industry develops.
In one of my earlier roles, I walked into a situation that was less than ideal. One of our products was a “mission-critical system”. It was, essentially, an ERP system for many school districts at once. There were some issues that prevented it from working, resulting in performance issues. Schools were not receiving the funding that they were owed by the state, due to the fact that our system was dropping the ball and losing some of those schools in translation. Each state had their own list of regulations that were upheld by the system, which only made matters more complicated.
On top of that, the teachers using these systems to document the grades of their students were finding themselves with missing entries. The problem affected a lot of schools and a lot of people. It was a really bad situation.
Actions taken
Technically, our team was eventually able to resolve the problems preventing the system from working. My job, a role that I had just stepped into, involved mediating with each patron firsthand in order to perform a bit of damage control. We wanted to stay out of court and out of the local paper.
One of my ways of trying to see eye-to-eye with our customers involved going to each one individually in order to really hear what they were trying to say. I sought to talk them down from the ledge. I was able to buy some time in this way; it would not have been possible without developing some sense of empathy for their plight.
The old notion is that the customer is always right. When it came down to it, my company sold them a service that they were not satisfied with. The discussions were very difficult at times; some of them began with the client actually yelling at me after leveling with them on the truth of the disagreement. We never blamed them; it was always the company who would step in and take the responsibility for the incongruence between what was expected and what we had delivered.
The next step involved giving them real solutions to the problems that they were having with our company and the products that we had shared with them. It was a really hard time, but I learned so much from the people that I had met.
Lessons learned
- If you’re not fully-versed on the type of business that you wish to involve yourself with, you should always seek out a professional consultant to advise you.
- Lean fully into the empathy that you have for the people who you serve. Being completely honest and transparent with a customer that you are having trouble reaching in some way will often buy you real time, so many months or years that you can spend repairing the damage and the relationship.
- As a representative of the service that you provide, you cannot back away from the responsibility if things happen to go wrong at some point.The customer has to come first, especially in an industry like education.
- Your errors will always result in your most valuable lessons.
Discover Plato
Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader
Related stories
10 December
Supporting principles on why being data led (not driven) helps with the story telling.
Vikash Chhaganlal
Head of Engineering at Xero
13 October
A high performance team refers to “ a group of goal-focused individuals with specialized expertise and complementary skills who collaborate, innovate and produce consistently superior results.”

Praveen Cheruvu
Senior Software Engineering Manager at Anaplan
2 August
Viswa Mani Kiran Peddinti, Sr Engineering Manager at Instacart, walks through his experience scaling a team, product and his skills as a leader.

Viswa Mani Kiran Peddinti
Sr Engineering Manager at Instacart
12 July
Vineet Puranik, Senior Engineering Manager at DocuSign, discusses the impact of roadmaps, organization, and proper management for your teams to procure growth.

Vineet Puranik
Senior Engineering Manager at DocuSign
1 July
Saikrishna Desaraju, Engineering Manager at Marks & Spencer, draws from his personal experience to advise new managers on thriving in their roles.

Saikrishna Desaraju
Engineering Manager at Deliveroo