Acting Quickly To Resolve an Incident
9 April, 2021

Engineering Manager at Carta
Problem
Recently we had an internal incident that required a prompt response. Our long-standing customer notified us that they found a bug on our platform. They were under a tight deadline to deliver some documents to authorities by means of our platform. The bug disabled them to locate the documents, let alone to send them.
Actions taken
A couple of our engineers were tasked to investigate what happened and why. While it was a matter of the greatest urgency, they were stuck for a few hours, unable to understand what had happened. I was called in to help them out.
I had a strong guess what the problem was. Being calm and composed allowed me to coordinate our efforts in the right direction. I organized the team in no time; I had someone checking on one piece of the application, someone else looking at the logs while I was talking to our point of contact that was communicating with the client. At that moment, I understood that having the most accurate pieces of information was critical; the more accurate the information, the more precisely we could detect the problem. I instructed the team to report to me frequently about what was going on. After collecting enough information, I was confident that my original guess was correct. We were able to identify what happened to our customer’s documents and merely had to recover them.
By quickly resolving the incident, we managed to turn a disadvantageous situation in our favor and further strengthen the relationship with the client.
In addition, I made sure to turn this experience into institutional knowledge. In a situation of great urgency, when every second counts, one can’t often share what they are doing. But I seized the first opportunity after the incident was resolved to reach out to engineers who were originally tasked to fix the problem and offered to explain what I did step-by-step. Other people also joined and were appreciative of my efforts to make our actions the collective learning experience.
Lessons learned
- By acting quickly to resolve an incident, one can turn the disadvantageous situation into a great opportunity. Not only did I fix the problem promptly without jeopardizing the relationship with the customer, but our rapid response and commitment to be at our customer’s service strengthen our relationship.
- I was able to share step-by-step what I did, so next time when someone on the team encounters the same problem, we can be even quicker to resolve the incident.
Discover Plato
Scale your coaching effort for your engineering and product teams
Develop yourself to become a stronger engineering / product leader
Related stories
21 March
A short overview of a very effective leadership assessment by Jack Welch, that is easily transferred to other industries is the 4Es of leadership – energy, energize, edge, and execution.

Ramesh Dewangan
CEO at Quantum Vision Consulting
21 March
Based on an awesome book titled "Deep Work" by Cal Newport we provide provide a brief overview of the Rules for Focused Success in a Distracted World.

Ramesh Dewangan
CEO at Quantum Vision Consulting
21 March
Is it possible to be too empathetic? If you overdo it, it can be an energy sucker.

Melanie Zens
Delivery & Operations / Digital Transformation / Innovation at Marais Consulting Inc
20 March
Learn about 10 rules from the wisdom of these long-living residents from Ogimi, a small village in Okinawa, Japan. You could interpret the rules as the lifestyle habits that enable the senior residents of Ogami to live long and enjoy their ikigai.

Ramesh Dewangan
CEO at Quantum Vision Consulting
7 March
3 ways leaders can cultivate relationships that lead to better products.

Guy Jenkins
SVP Global Customer Experience at Salesforce