Dr. Anant Madabhushi is developing deep-learning software that could predict how diseases develop.
In October, NephCure awarded Dr. Anant Madabhushi a NEPTUNE Ancillary Studies Grant for his work on computational imaging of kidney pathology slides.
The Nephrotic Syndrome Study Network, or NEPTUNE, is a long-term observational study that has gathered health data and biological samples from close to 2,000 glomerular disease patients nationwide. Researchers can apply for grants to conduct research on this de-identified patient data. Besides having helped fund the creation of NEPTUNE, NephCure also now helps provide the funding that make a number of the ancillary studies possible.
We recently spoke with Dr. Madabhushi to learn more about what he and his team will use the NEPTUNE data to study, and how he sees his work contributing to the future of precision medicine in the kidney field.
NKI: Could you give us a brief synopsis of your work and what you’ll be using this award to study?
Dr. Madabhushi: Over the last 12-13 years, my group has been developing the technology and algorithms for analysis and computational characterization of tissue images. When someone gets a biopsy, their tissue slide has historically been read by a pathologist. But now with improved technology, we can scan these slides and create digital images. We can then start to train a computer to create a predictive model that is able to look at the digital images and identify patterns. Doing this could tell us about disease presence and aggressiveness and potentially about response to therapies, which is a big deal. Being able to use a computer to figure out, from a routinely acquired tissue slide alone, if there are patterns which may tell us which patients may or may not respond to a particular therapy is of immense value to a clinician. It allows for potentially better therapeutic management for the patient, potentially obviating the need for more aggressive therapies in patients who may not receive added benefit from them.
So that’s been a large part of our work over the last 12 years. Most of our work has been focused in the cancer domain. I’m really excited about the NephCure award because it represents our first foray into the non-oncology space: we’re looking at kidney disease, which is somewhat novel to me.
The project that we are pursuing with the NephCure award is to develop a set of tools and algorithms—software—that will start to analyze tissue images of kidney biopsies. With these tools, the computer has the potential to start to recognize patterns of aggressive disease. The ability to do that in the context of kidney disease is huge, because there’s still so much that we don’t know about kidney disease. We still don’t know so many things about response to therapy and outcome and prognosis in the kidney disease space.
The long-term vision is that these tools will allow a clinician to be able to prognosticate, based on kidney biopsies, how a patient is going to do and what treatment might be appropriate for them. I need to qualify that we have one year. So the question is: what are we going to be able to do in one year, and how does that set the stage for the long-term vision and realization of the bigger picture of kidney precision medicine?
The goal of this specific project is to develop a set of tools that will allow end-users, that is, nephrologists, pathologists, to use this software to create very deeply annotated mark-ups and portraits of the different structures in kidney pathology images. For instance, in kidney pathology images, we know there are different individual substructures, like glomeruli, proximal tubules, capillaries, and so on. The goal is to train the computer so that it can go in and start to identify all of these structures on its own.
Once we’ve created a deeply annotated data set of images with all these structures identified, now one can start to ask specific hypotheses. For instance, once the computer has identified where all the glomeruli are, can I figure out whether there is an association with treatment response or recurrence of disease, just based on the number or location of glomeruli?
So what we have is essentially a sort of a pattern-detection pilot project. Hopefully by the end of it, we’ll have these patterns mapped out by the computer, and then the nephrology and nephropathology communities can start to look at those patterns and say yes, these patterns seem to be important in transplant rejection, these patterns seem to be critical in chronic kidney disease or nephrotic syndrome. The goal is to create the enabling technology that allows clinicians and pathologists to go in and start asking those prognostic and predictive questions.
NKI: Is this software that your team is developing something that, down the line, will get smarter with more data?
Dr. Madabhushi: That’s absolutely right: the more you give it, the smarter it becomes. The problem is, it’s not just about giving it any data, you have to give specific data to the computer. In other words, if you want the computer to recognize what a glomerulus and a capillary are, you have to have an expert, a pathologist, sit down and map out these specific structures and then feed the image to the computer. The computer then looks at a large number of these and starts to learn and recognize these structures and then starts to provide the output.
Let’s think about that for a second. Who are the people that are going to provide these annotations? They would be expert pathologists, nephrologists, and nephropathologists. These are not people with a lot of time.
I’ll give you an example: we published a paper in Nature Scientific Reports looking at how to train a computer to recognize patterns of invasive breast cancer on tissue slide images. That work took four years, because we had 600 slides. On each of the 600 slides, we had to have a pathologist sit down and manually mark up where the cancer was. Those annotated slides were then used to train the computer. A lot of these algorithms are very dependent, not just on images and data, but on manual annotations of the data to become better and better.
The pathologists and clinicians who are going to be able to do these annotations just don’t have the time. They don’t have the bandwidth to be able to sit and spend hours doing the markups and the annotation.
What is the alternative? You need to be able to make the computer quick and efficient in the way it learns. One of the critical attributes of what we’re developing is a very lightweight version of this learning infrastructure. In other words, what if we could train the computer with just four or five examples? If you could, in 20-25 seconds, mark up what you want the computer to find on four or five slides, and then the computer rapidly uses just those four or five examples to create a network and a prediction and give you the results.
Now, because the computer learned with just four or five examples, it’s probably not going to do a great job. But that’s okay, because what the end-user can now do is edit the results from the computer. Let’s say the computer found the majority of the glomeruli, if that’s what we’re going after, but it missed a few. Now the user can go in and say, “I see that the computer missed it here and here, so I will mark those up. I also see that in these few places, it seemed to identify something as a glomerulus when it wasn’t. So I will erase that because that result isn’t right.” So now with another 20-30 seconds of interaction, we’ve cleaned up the results from the computer. The computer then takes that cleaned up result and re-learns. What we’ve done is created a very efficient way for the computer to learn that is not taxing on the end-user. The initial result is not great, but with a few iterations back and forth, the computer can very quickly start to become very efficient and accurate.
The other advantage of this model is that it makes it very generalizable. I won’t need to spend three or four years creating a dedicated glomerulus detector. In one day, I could have the software learn what a glomerulus looks like. This is a big deal, because what would have previously taken years, we can maybe do in a few weeks.
That is really what is exciting about this. The computer is able to do the bulk of its own self-learning: We’re teaching the computer to be more effective in learning rapidly.
NKI: You mentioned that you hadn’t really worked with the kidney before. How did you get involved with kidney pathology and get connected with NEPTUNE?
Dr. Madabhushi: Three people: Laura Barisoni, John Sedor, and Michael Feldman. They got me into thinking about kidney pathology. Michael Feldman is a pathologist at the University of Pennsylvania. I’ve been working with him for a long time on a number of different cancers. He called me up one day and said, “We’ve got to start thinking about the kidney.” Before I knew it, I was connected to Laura Barisoni, who is one of the world’s leading nephropathologists at the University of Miami. And then, Dr. John Sedor, a nephrologist literally down the road at the Cleveland Clinic who I had not met before, stopped by my office one afternoon, and we started talking about the kidney and kidney pathology.
So this conversation started about a year ago. It’s actually quite unbelievable. I met John and we started to brainstorm about opportunities in the kidney space. This work is the culmination of those conversations and the frequent meetings that we have. It’s been their persistence, resilience and passion that got me excited about kidney pathology. I’d been very happy working in the oncology space. This was them appreciating the utility of the tools that we’ve been developing in the oncology space and realizing that these tools could be transformative in the kidney pathology space. I really credit these three folks for bringing me over to the kidney side of things.
I will say, this award is a real shot in the arm. It really energizes us. We’ve been doing this work off and on using cobbled resources. Andrew Janowczyk, Yu Zhou, and Jeff Nirschl from my group have done an outstanding job in creating the initial software infrastructure to get us where we are currently. Now, given the fact that we have this award, it really allows us to increase the tempo and rev things up. We can use this as a basis to launch potentially even bigger projects and create a bigger operation in the kidney pathology space.
We had a great time chatting with Dr. Madabhushi about his work and are very much looking forward to seeing his team’s advances in kidney pathology imaging. Stay tuned to www.NephCure.org for updates on their research and other news from the field.
Dr. Anant Madabhushi is the F. Alex Nason Professor II of Biomedical Engineering and the Director of the Center for Computational Imaging and Personalized Diagnostics at Case Western Reserve University.