Language is central to how the brain communicates and makes meaning. Because the brain is, in many ways, the original architecture for language models, the rise of large language models (LLMs) opens new possibilities for understanding both human cognition and artificial intelligence. To explore these connections, Benjamin Hayden, professor of neurosurgery at Baylor College of Medicine and adjunct professor of electrical and computer engineering and linguistics at Rice University, and Suzanne Kemmer, associate professor of linguistics and cognitive sciences at Rice, organized a symposium at the BioScience Research Collaborative.
The event, LLMs and the Brain, featured researchers from Rice, Baylor College of Medicine, the University of Texas, the Georgia Institute of Technology, the University of Montreal and other institutions with talks spanning topics from language processing in the brain to interpretability in AI systems. Supported by Rice’s Creative Ventures Fund, ENRICH, Neuroengineering Initiative, Department of Linguistics and Program in Cognitive Sciences, the symposium highlighted the university’s role in fostering collaboration between neuroscience and AI in close partnership with the Texas Medical Center.
The conversation around brain research extends beyond the university and is unfolding at the state level. On Nov. 4, Texas voters will decide on Proposition 14, which would fund the Dementia Prevention and Research Institute of Texas (DPRIT) with $3 billion over 10 years, creating the largest state-funded dementia research program in the country.
To learn more about the goals of the symposium and the wider importance of brain research, Rice News spoke with Hayden for a Q&A session.
What inspired the creation of this event, and why bring LLMs and brain research into dialogue now?
Suzanne Kemmer and I have gotten very excited about the amazing capabilities of large language models. We are very optimistic about their utility as scientific tools. In particular, we think that they have the power to answer basic questions about how language works and about how the brain implements language. We wanted to invite some of the best and the brightest people doing that kind of research to Houston to teach us about what’s going on and hopefully galvanize new research ideas at TMC.
The agenda included speakers from neuroscience, computer science, linguistics and engineering. What makes this intersection so compelling at this moment in research?
LLMs are basically computers that generate language. Remarkably, when you look inside of them, you see a lot of processes that look a lot like what the brain is doing as well.
As a neuroscientist, you always want model organisms that are easier to study than humans. For neuroscience of language and neuroscience of concepts, LLMs are basically a model organism. But to understand them, you need computer scientists and engineers.
How do you see this symposium contributing to broader conversations in both neuroscience and AI?
We invited a range of speakers with different disciplinary backgrounds but who are already all facing in the same direction. Bringing them together makes them talk to each other and talk the same language. We hope that doing that creates the kind of friction that leads to sparks. In the days since the symposium, we have seen multiple potential collaborations percolating among the speakers and attendees.
What are some of the open challenges in using LLMs to learn about human cognition?
To me, the biggest challenge is that we just don’t have enough brain data. You ideally would want hundreds of hours of speech and thousands of neurons at the same time. That would really help us understand how the brain does language.
Another one is the lack of good theories. Right now, we are kind of in awe about how LLMs do what they do. Some things they do are better than expected. So another big challenge is figuring out exactly how they work, which takes good clever computer science.
Many of your lab’s projects touch on mental health, from depression to addiction. How might the convergence of brain research and AI lead to better treatments for psychiatric or neurological disorders?
Diagnosis and prognosis are always difficult. We’d love to have a blood pressure test for depression or dementia, something that takes five minutes and gives a good reliable sense of the overall status. LLMs can analyze language use and may be able, if programmed the right way, to give a measure like blood pressure. They could warn you when you need to get up and exercise, take your medication or take a nap.
I’m very optimistic about neuromodulatory treatment for psychiatric diseases. But making that work means we need to measure how people are doing in response to the treatment they are getting. In other words, to “close the loop” we need to measure how well the neuromodulation is working — turn it up or turn it down? I predict that LLMs will be a critical ingredient in that process.
What are the ethical or translational stakes of connecting LLMs to brain science, particularly when it comes to patient care?
These are very high, and we always take care to think through all the risks and consequences of our approach before each step of the research. It’s important that we maintain trust from the patients and the community. The stakes here are higher because LLMs are, in many ways, this mysterious thing that no one understands — that makes people feel slightly more cautious about it, and we need to respect that.
This event brings together early career scholars and trainees alongside senior researchers. How important is this mix for building the next generation of cross-disciplinary science?
It’s essential. The younger people always have the best ideas and the most innovative approaches and are often the first to tell us when the old ideas don’t work anymore. The older scientists have the ability to see the broader playing field, so bringing them together is vital.
What do you hope participants take away from this symposium, and what do you hope the broader Rice and TMC community learns from it?
Besides the scientific ideas, I hope they get a sense of how powerful LLM-based approaches are in so many different domains, even ones they didn’t think of. Right now they are this amazing Swiss Army knife, and we are discovering new uses every day. Getting all those together in one place is really valuable.
Looking forward, how do you see collaborative frameworks like the Rice Neuroengineering Initiative shaping the field where neuroscience and AI meet?
It’s going to be absolutely essential. These new technologies are leading to new cross-disciplinary fields that are composed of elements from other fields. We need to make it easier to collaborate, including across institutions, to make progress in the future.
What are your thoughts on Proposition 14 and the potential creation of DPRIT?
I’m very strongly in support. We are at a point where technology is advancing rapidly and biomedical research is struggling to keep up. But that creates an opportunity, where for a small investment in biomedical research we can get a huge multiplier, because so much of the technology is already solved.
After the conversation with Hayden, Kemmer added the following remarks on the symposium’s goals and the expanding dialogue between language, neuroscience and AI:
I agree with Ben’s observation about the remarkable potential for cross-fertilization of disciplines that the emergence of large language models gives us. Combined with recent technological developments in brain research that Ben alluded to, some pioneered here in Texas, we are truly at a new jumping-off point.
The symposium allowed people pursuing many different approaches to hear from each other and get ideas they could use to advance their own research as well as the collective project of understanding how the brain performs various functions.
For decades linguists and psychologists have been trying to model how linguistic information is structured and processed during language use and how language processing relates more generally to functions like perception, memory and general reasoning. Neuroscientists have studied the possible localization of linguistic processing in particular brain areas with a variety of methodologies. In recent years, a picture has been gradually emerging of how these various brain functions are perhaps less tightly localized and much more intertwined during neural processing than previously thought. Now, with new noninvasive technologies, researchers can observe language processing in a much more direct way by monitoring brain activations at a very fine-grained level during language use.
With the emergence of LLMs, researchers can come together and expand knowledge from across disciplines with new information obtained from modeling language with this new technology. Furthermore, comparing the results with real-time language processing in actual brains at the level of neural circuits is really exciting.
The potential for rapid advances in knowledge of how the brain works and applications of this knowledge to specific problems like dementia, addiction, PTSD, epilepsy, Parkinson’s and other afflictions of the brain are truly immense.
The symposium showed how rapid progress is being made along various fronts. What is emerging is a road map for how these problems can be tackled and solved in a shorter time frame than we could have envisioned even five years ago.