The groundbreaking collaboration between poet Nick Flynn and media artist David Rokeby unveiled a fusion of poetry and artificial intelligence (AI) at an April 12 event at Rice University’s Moody Center for the Arts. Titled “THEY ARE FALLING ALL AROUND ME / Postcards from the Edge of Latent Space: Exploring New Sites of Creativity with Generative AI,” the performance showcased a live demonstration of an AI program that generated illustrations in real time in response to Flynn’s poetry.
“We’ve been working on this in my lab for about two and a half years,” said Rokeby, director of the Bank of Montreal Lab for Creative Research in the Arts, Performance, Emerging Technologies and AI at the University of Toronto.
Rokeby’s expertise in AI technology was instrumental in bringing the program nicknamed “Voice Scroll” to life. Its ability to interpret and respond to spoken words with visual imagery added a new dimension to traditional poetry readings.
“There’s a series of poems that we’ve been training the machine on, so it’s a set list,” said Flynn, a University of Houston English professor. “Each poem has its own set of ambient prompts, which control where the machine can go, so it doesn’t just go wild. It creates the mood that we want for the poem.”
Flynn has received numerous awards for his work, including the prestigious PEN award. His latest poetry collection “Low" served as the inspiration for the AI program’s dataset.
The live aspect of the performance was crucial with Flynn highlighting how each rendition of the poem yielded unique results from the AI program.
“The speed that this happens means that you can explore it kind of intuitively, almost the way you would have a conversation with somebody,” Rokeby said. “That changes it so that you can get a better feel, I think, for what it does badly and what it does really well.”
He and Flynn admitted there is plenty of both in the process so far, but what comes with that is a sense of unpredictability and imperfection.
“We’re really at the very beginning of it, which is very exciting because it’s still really wonky,” Flynn said. “It makes a lot of mistakes, like creating six-fingered people.”
“As an artist, you’re often bending away from the real to be expressive in different ways,” Rokeby said, explaining that what engineers might label glitches, he and Flynn view as features. “We’re trying to hold on to that wild moment, the wild moment full of all the possibilities, because when you ask for things to be perfect, you let go of a lot of creative possibilities.”
The event not only showcased the creative potential of AI but also sparked discussions about its ethical implications and artistic possibilities. Following the performance, Flynn, Rokeby and Rice professor Anthony Brandt engaged in a conversation exploring these themes.
“At the Moody, we are interested in exploring the intersection of the arts and technology, which made this program a perfect fit for our spring season,” said Alison Weaver, the Moody’s executive director. “Providing a platform to present experimental projects such as this one is at the core of our mission as an innovative institution.”
Funded in part by the Cynthia Woods Mitchell Center for the Arts and supported by the BMO Lab and the Moody, the event marked a pioneering exploration of the intersection between human creativity and machine intelligence.