Approaching Predictive Analytics Problems

Problem

We were developing a predictive analytics product for healthcare, the aim of which was to promote preventive care by delivering people an overview of their health status and future, including severity for severe diseases. It could detect high risk or specific outcomes within the 3 - 5 years. With the use of highly sophisticated AI, we worked to get high specificity and accuracy with the scores. That is more on the technical side, but how do we convey the message when it comes to communicating that to the consumer side?

Let’s say our product discovered someone with a high risk of colon cancer, but when we came to that, we needed to think about how they would react to this kind of observational insight. While some consumers might show interest in checking in with a physician, others may simply freak out. In that way, we could cause more damage than good. After all, we did not want people to break down mentally. Our issues with the product revolved around communicating the right message to our end users.

Actions taken

In order to understand the type of message we wanted to convey to our consumers and patients, we conducted surveys and focus groups. Once we gathered the information, we took a few measures. First of all, instead of writing down the scores, whereby someone may have 9-points for getting colon cancer in the next 3 years, we changed the whole algorithm.

It would work in a way that would compare populations and not just give scores. As an alternative, it would rather say we were comparing vast amounts of data, and we found that people who are similar to you, health-wise, were at more risk for a particular disease. We were generally addressing people and not directly pointing at someone. It took us a while to change the whole algorithm system for a successful migration, but in the end, it was effective.

We changed the message that we were communicating. There was a huge implication in talking about the risk itself, so what we did was we talked about the biomarkers. We pointed out some of the things like, for the case of colon cancer, instead of telling the consumer that they were at a higher risk, we would indicate the level of hemoglobin. This was one of the most effective solutions we came up with. It was a reverse engineering algorithm that helped both physicians and our consumers. They did not just get the black box results but also the logical reasoning and the results behind it.

Last but not least, we also separated what we communicated with our consumers versus physicians. We realized that we could communicate some messages to a certain level with customers, but anything beyond that should go to the physicians themselves. As the professional in the loop, the physician should then be the people communicating with the consumers/patients. For instance, if our AI was unsure how severe the risk was to be 一 the absolute risk is high, but the relative risk is low 一 the machine would not communicate it with the patient. Instead, the physician would decide whether they wanted to talk to the patient about it or not.

Lessons learned

• Communicating messages in the health industry are not binary and all about 0’s and 1’s. So, anyone doing so should rather be careful and confident about doing it in the right way.
• Some of the observations you may give out to people do have many magnitudes, and you need to consider that. For you, it might be a data set that you are working on, but for some people, it could be life-changing.
• Technology is here to serve the needs of people and not the other way around. To illustrate, when we were serving the score given by the number, it upset both the consumers and the physicians because they were only getting a number. We worked on the black box, extracted the information to find the actual factors.