A Solution in Search of a Problem: Designing AI Tools With the Rare Disease Community
Something has been weighing on me. I sit at the intersection of a few roles: I am a data scientist and an application developer, and I am the mother of a child with a rare, poorly understood, and currently incurable disease. From all of those seats, I have grown uneasy about how thoughtlessly some AI applications are being designed, including many that are advertised as “patient centered.”
I often warn my students not to be the scientist with a solution in search of a problem. We fall for a beautiful new tool and then look for somewhere to point it. That excitement is real, and I do not want to shame it, because I feel it too. Clinicians are excited in a different way: they have problems they genuinely need solved, and they can see what these tools might do for the patients in front of them. I share all of it. I wake up every day grateful for these tools. I believe that AI will finally help explain what happened to my son Charlie: why he had infantile spasms, why at sixteen he lives with multiple disabilities, why he still has frequent, life-threatening seizures, why I have never heard him speak a sentence.
Even so, I feel conflicted, because I work alongside teams who are just as excited as I am, and because I do not think even us “experts” always understand the tools we are building, and for whom we are building these tools. I worry that we are making mistakes that could hurt people. Here is the story of why I believe that.
The night I first heard "infantile spasms"
Sixteen years ago, my son Charlie began showing the signs of infantile spasms, though I did not know that at the time. It started at mealtimes. He would turn his head away from the spoon while I fed him. The pediatrician thought he was teething.
Then he became inconsolable. I will never forget standing at an open house for my older daughter's new preschool while Charlie screamed the entire time. Other parents looked over, not unkindly, but with the instinct parents have: something is wrong with that baby. We were comforting him, doing everything right, and still he screamed. I called the pediatrician again. Teething, he said, and he gently implied that I was an anxious mother. When I expressed concern about his development, his “too happy” affect, he said I was unreasonably comparing my son to his exceptionally bright older sister. Any parent knows the feeling I mean: you know when something is wrong with your child.
The strange movements came next. He would drop his head to one side. One evening we were sitting in the living room, not eating, and he did it again, the head drop, now coming in clusters, I now know to be an epileptic spasm. I had no vocabulary for any of this. I was a data analyst at a major academic institution, but I did not work in epilepsy, and I had never seen anything like it. This is a point I come back to often: most people are not experts in whatever rare disease finds them, and that has nothing to do with how good they are at using the internet. Vulnerability to poor information about a rare condition reaches everyone, including doctors, including PhD-level scientists. None of us knows much about these diseases until one arrives in our own home.
One thing led to another. I was sent from one hospital to another, and we arrived at the second one at night. They transferred us without telling me why, and by then I was frightened. They put Charlie and me in a small side room, really more of a closet with a pediatric stretcher in it. He could climb and crawl, and the bed was not safe for him, so I stood with one hand on him the whole time.
I had come straight from work. Earlier that day I had mentioned my nervous feeling to some physicians at my institution, not neurologists but knowledgeable enough, and they were the ones who told me the words: ask for an EEG. At first I misheard it as EMG. I did not know anything. Those of us who breastfeed and go back to work know that you cannot always wear at the office what you would wear at home to breastfeed a baby who is breastfeeding constantly, and Charlie was breastfeeding constantly, I now think to comfort himself. So there I was, in my work clothes, in a closet-sized room, trying to keep him calm and breastfeed him awkwardly in a silk blouse.
After a while, there was a knock, and three medical students or junior trainees came in to ask me questions. Those of us who work in healthcare know what that means. It means something is wrong, that this is the kind of case people want to see. They asked me about Charlie's history and his behaviors. I do not remember the questions; it is all a blur. When they finished, they left.
A resident occasionally looked in on us. He told me we would have to wait until morning, because no one could do an EEG overnight. I was trying to reach my then-husband at home on a BlackBerry. I asked the resident what he thought was going on, because I truly had no idea. In passing, he said they thought it was infantile spasms, that an EEG would confirm it, and that it would have to wait until morning. Then he walked out.
What the first search result told me
With one hand on Charlie, who would not settle, and screamed so much he was almost whispering, I typed "infantile spasms" into that BlackBerry. The first thing that came up was a peer-reviewed article by Dr. W. Donald Shields, "Infantile Spasms: Little Seizures, Big Consequences." I know now that it is a genuine and valuable contribution to the literature. In that moment, one line took my legs out from under me:
Infantile spasms is associated with a significant risk of mortality and morbidity.
It rested that claim on Riikonen's series, which had tracked 214 children for two to three and a half decades: close to a third of them died over the course of the follow-up, many before the age of three. It was the word mortality I could not get past, and the fact that a third of the children in that study had died, and I read every bit of it as being about my son.
My legs gave out. Charlie is going to die. I called my then-husband, who was home with our daughter. I called my parents, who started crying, and dropped everything to drive overnight from upstate New York to Baltimore, to say goodbye to Charlie and comfort me. The next twenty-four hours are a blur. They moved us to the neurology ward, where Charlie had an EEG the next morning. He had a stuffy nose, so everyone was in contact precautions and I could not see a single face. I called Charlie's godmother to come sit with me, because I had read that my son was going to die, and I could not read any further, and I was trying to keep him safe and comforted at the same time.
I remember thinking, I need to pull it together. I need to look good. I can't break down. I need to be smart. I need to hold it together so I can make good decisions for Charlie. I need to hold it together so I can make good decisions for Charlie. It’s important that these doctors see me as smart and rational. I need to keep it together for Charlie.
All day I had been asking about infantile spasms, and the senior doctors kept asking me where I had heard that phrase. I could tell they were unhappy that someone had said it to me, yet no one would give me information, because the EEG had to be read and discussed first. Finally, that evening, almost a full day after the resident's words, a team came in fully gowned, six or seven of them, or fewer, I could not tell, all faceless behind their precautions. They confirmed it: Charlie had infantile spasms. It felt like a gun going off next to my ear. I went down in a heap and said, over and over, moaning like an animal. “Charlie is going to die.”
One of them stepped closer and asked where I had heard that. I reached for the article, shaking, trying to find the website again. I did not need to. He knew exactly what I meant, because that is what came up when he searched "infantile spasms," even through a trusted resource like PubMed, which is what I had used, because I was an epidemiologist, and I know exactly where to find the “credible” information.
Why "credible sources" were not enough
Here is what the doctors could not tell me at the time, and what almost no one could: we knew very little about the causes, the triggers, the mechanisms, or what predicted whether one child would do better than another. What the framing of that era did say was that if no cause could be found, a child was expected to do better, not worse. They could not find a cause in Charlie, and nothing I described seemed to fit a known one, so they told me he would probably have a better course than I feared, and probably would not die, at least not right away.
It would later turn out that Charlie had a malformation in his brain that no one can explain, which they believe may have a genetic root that has still not been identified. Sixteen years on, after watching the number of genes we can test for grow from a handful to hundreds, no one can yet tell me what caused his spasms. Last month I finally arranged to send old tissue from one of his two brain surgeries to a scientist I had tracked down at a conference, one I attended and paid for out of my own pocket, in the hope of finally learning what happened to him. If you think families like mine turn to the internet for answers, you have no idea how far past the internet we will go to understand our child's condition.
So do not build a tool that claims confidence it has not earned. I know myself well enough to know that if this were happening to me now, and a tool were branded as smarter than whatever I could find on my own, I would take it seriously. If that tool brought up the same article, it would be worse for me than the open web was, not better, because the branding would tell me to trust it. That article had been set aside by a doctor as out of context and out of date, and yet it is exactly the kind of source that an AI chat is likely to bring up.
We tell ourselves that if we feed these systems trusted, credible sources, the output will be sound. For rare diseases, that assumption breaks. These conditions are notoriously underfunded and poorly described. Patient advocacy groups largely exist because the information their families found online was inaccurate, incomplete, and disorganized, and they wanted to address that. If we pull only from so-called credible sources, however we define that, we are frequently pulling the same thin material that families and clinicians already read. The doctors gave me information that we would not consider correct today. It may have been the best reading of the literature then, yet we understand it very differently now.
Make that make sense: we are rushing to put AI inside our applications while doing nothing to improve the quality of the information underneath it. You cannot train a model on data that do not exist. This is why co-production with the patient advocacy community is not optional, and it is the heart of the ethical concern about the confidence these tools imply. There is no fiercer champion of AI than a parent whose child has an undiagnosed or poorly understood disease. We embrace these tools, and our embrace carries an expectation that the tool knows more than we do. If the same model sits underneath a chatbot that a tech leader says was built specially for us, and it returns the same article a family could have found alone, it has given them nothing new. It has only made the old answer feel more certain, wrapped in a confident tone. What have we done?
What a better tool could have done
Picture a well-built epilepsy resource, made with the community, with a chat feature. I type in the only words I have, the ones the resident gave me: infantile spasms. What could it have done differently?
It could have started by telling me that this is complex and multifactorial, that I would find many frightening things online, and that because the condition is so varied, I could not take a statistic from one group and apply it to my child. I cannot take numbers from children with tuberous sclerosis, a common cause, and assume they describe a child with Gould Syndrome, or a child with an SCN2A variant. A group number does not describe an individual, and a number for one cause does not transfer to another. Just knowing that would have steadied me.
Then it could have told me the single most useful thing: have this conversation with your clinician, and here are the words to say. That one sentence, first, would have changed everything about that night. It could also give the clinician a fact sheet prepared by the scientific advisory board of a patient advocacy group, so no one loses precious moments searching for one.
The other thing it could have given me right away was a link to the online community that other parents had built, which is where I found nearly all of my comfort and information in those early days. On my own, that resource took me weeks to find.
Where would language like that come from? From the community. We would come together and agree on how to talk with a parent who has just heard the words infantile spasms, because we have all lived some version of the night I just described. My story is not singular. A group of people who have searched the internet in terror for their own child's condition would know exactly what to write, and we would test it, and we would all be able to stand behind it.
There is one more thing I came to understand that no chatbot could have told me, though the community could. Over the years I have met literally thousands of families affected by infantile spasms. Among them are many whose children largely recovered, including many who underwent hemispherectomy, the surgery that removes half of the brain, and who have gone on to college, to careers, to full lives, getting married…even though those stories rarely make the headlines but are the stories we all cling to in those early years. If you built a tool with the community, we would make sure you found that; and you would not find that on PubMed, StatPearls, or UpToDate.
There is one more piece I want to add. The article had described two risks, death and serious developmental disability, but in my panic I had read only the part about death. What the team offered me, once they finally spoke with me, was their reading of Charlie in particular. Given what they did and did not know about his underlying cause, they did not expect those mortality figures to describe him, and they thought the more likely reality for him was the developmental disability rather than death. My heart soared. Charlie will be severely disabled, but he is probably going to live. In a way that may sound strange to someone outside this, I was flooded with relief, because I would love Charlie through anything. It never once crossed my mind to grieve his intellectual disability for its own sake. As long as I had him, and he was not suffering, we were going to be all right. A candid tool, built and vetted by the community, could have helped me hold both truths at once, the stark statistic and the reasons it might not be my son's story, hours earlier than a gowned team with no faces finally did.
Questioning the treatment
Over the years, I came to question the treatments Charlie was and was not offered. Infantile spasms has a narrow set of first-line therapies, treatment should begin quickly, and surgery should be considered for the right candidates. Many children never receive the right regimen, for reasons that include gaps in provider education, fear of the side effects, and the cost of the medication. Imagine if, during the years I suspected he was not being offered the right options, the community had already assembled the information that an AI drew from, so it could give me that picture in a clear, responsible, usable way. That would have been a gift.
I am not saying we should avoid AI. I am saying that when the output of an AI tool merely reflects the status quo back at us, that is not good enough, and it is not responsible. That is us enjoying our tools and congratulating ourselves on a clever product.
The community is already building
In the past few years, I have watched an explosion of innovation coming straight from patient advocacy groups in how they use AI. At the same time, I have seen an assumption in parts of the research and clinical development world that patients do not know how to use these tools themselves, that handing them essentially the same capabilities they could get anywhere and calling it new is somehow enough. “Look, here is this thing called ChatGPT, but this one has a purple ribbon, so you know it's for epilepsy.” No. Not good enough. Families already know how to use these tools, maybe better than we do. I have watched groups build entire knowledge graphs, coding their way through it by trial and instinct. If your child has one of these diseases, nothing will stop you. You will learn, and you will become an expert.
So if you are wondering why a community rejects your tool, or there is no traffic coming to your site despite announcing it at conferences, posters, abstracts, and webinars, consider that they are not naive, and that they could probably build something better themselves.
If you do not involve the patient community from day one, the only conclusion I can draw is that you do not care about the people who will use your product. Try to imagine any other business approaching product development this way. Imagine designing a car without asking a single driver what features that they want that car to have. Of course you would ask them, because you want to sell cars. But if you do not ask the people who would use your AI product what they think, or whether it would help them, or you push back on their concerns about your “trusted sources,” I can only conclude that you do not care whether they use it or like it. That tells me a great deal about you in general, not just about this product.
If you want to do this well, if you want to use these tools the way our talents are meant to be used, here is what I would ask.
Design considerations for co-produced tools
Co-produce from the first day. Build with patient advocacy groups and participants, not for an imagined version of them. When a group or a single participant tells you something concerns them, treat it as a finding, not a nice-to-have.
Improve the information underneath, not only the interface. A chatbot running on the same model a family could use for free, drawing on the same thin literature, gives them nothing new; it only wraps the old answer in more authority. The value has to come from better, community-vetted information at the foundation.
Represent the state of the literature truthfully, including how thin it is. Say clearly when little is known, and do not imply a certainty the field does not have.
Do not move statistics across causes. A number from tuberous sclerosis does not describe a child with SCN2A or Gould Syndrome, and a group number does not describe any one child. Build that warning in.
Point people toward their clinician, and give them the words for that conversation. Often the most useful first response is help starting the right discussion.
Test with a real range of people before release. Include people who are frightened, exhausted, and new to the vocabulary, which describes most of us at the moment of diagnosis.
Make it easy to share. Let a family show a clinician the exact output they saw, word for word, so the tool supports the conversation instead of standing in for it.
Date what you show, and cite where it came from. Let a reader see when something was true and check it themselves.
Disclose what was made by AI and what was written by a human. I learned the importance of this when I vibe coded a knowledge graph with Claude for my internal team; it generated very smart-sounding summaries that were often wrong, misleading, or out of date. I only knew because I am a domain expert, but if I had not read it carefully, even I would have missed it.
I have seen at least one early effort that takes this seriously. The Dravet Syndrome Foundation is developing an AI-powered Dravet Syndrome ontology out of exactly the kind of co-production model I have been describing. I had the great privilege of testing it, and I am struck by the care and responsibility going into the work.
What we are called to do
I work across several diseases through my teaching and my advising, but especially ALS, and I know firsthand how difficult it is to do this work well. Every day I wake up grateful for these tools, and I am personally doing things I could not have done without them. But when we pour our effort into the engineering and the fun of what else we can make these systems do, without ever asking who is on the other end reading the result, and then we call ourselves patient centered, that is not it, folks. That is not what our talents are for. That is not why we are here.
This goes out to all the well-meaning people who simply never thought of it this way. Let us do it together, as a community. We are pioneers. Last week I was still coordinating the testing of a piece of my son's brain, still trying, sixteen years later, to learn what happened to him. I would have given anything, that night in the closet-sized room, for a tool that told me the truth gently: that this is complicated, that the frightening number I had just read might have nothing to do with my child, and that the next thing to do was talk to his doctor, and here is how to have that conversation. We can build that. We owe it to the people who will search these words late at night, with one hand on a child who will not stop screaming, and one hand on their phone.
Source
W. Donald Shields, MD. Infantile Spasms: Little Seizures, Big Consequences.Epilepsy Currents. 2006;6(3):63–69.
