Birth of the A(I)uthor

My guest on the show today is Tuhin Chakrabarty, an assistant professor of computer science at Stony Brook University and the culprit behind my first ever trigger warning on eminent Americans.Here we go: if a big part of your identity is vested in being a writer, and you are already experiencing existential dread about the prospect of AI getting better than you at writing, you may want to not listen to this podcast, or at least you may need to take a Xanax or a gummy before listening. Because Tuhin’s work, training AI to produce super high quality samples of high literary writing, is existentially threatening. It goes right at the heart of what we wordsmiths do. Just to give a sense, here’s a passage from a New Yorker article by Vauhini Vara that explores what Tuhin has found:I asked Chakrabarty to run an informal version of his experiment on my writing, with a twist: I would pit his model directly against me. To start, he fine-tuned a model on my published writing, much as he’d done in the formal experiment. Then I sent him four short excerpts from a novel that I’m currently writing. No one else had read these excerpts; they had never been published or circulated. There was no way that a large language model could have seen them before.The narrator of my novel in progress is an Indian American ex-journalist. She runs a nonprofit that publishes stories from immigrant and refugee women, but it’s strapped for funding, so she courts an Indian American venture capitalist as a potential donor. Chakrabarty used an L.L.M. to create content summaries of the excerpts I’d sent him. (One representative sentence, from a summary about the narrator’s journaling habit, explains, “A pivotal memory is introduced: in ninth grade, the narrator’s mother read this journal, an act seen as a profound betrayal.”) Finally, he gave the summaries to his fine-tuned model, and he asked it to compose passages “in the style of Vauhini Vara.”Going into all this, I was self-assured, even smug. I’d always felt that my style was original and, more important, that my books were totally distinct from one another. I figured that, even if the A.I. model could imitate my past books, it couldn’t predict the style of the novel in progress. So, when Chakrabarty sent me the A.I.-generated imitations, I was genuinely confused. Like Díaz and Nunez, I found lots of stylistic details—rhythm, verbiage—annoying. But the text produced by the model was eerily close to mine. Reading some of its lines next to my own, I couldn’t remember which was which. Unlike Díaz or Nunez, I even preferred some of the doppelgänger’s versions. My style seemed to be more consistent across projects than I’d realized.I sent four passages to some readers who’d liked my previous books, explaining that half were mine and half were the model’s. I wanted them to guess which were which. … The first of my readers to respond was Dana Mauriello, my best friend from college and an accomplished tech entrepreneur. “Truth: this was terrifying!” she wrote. “I was so nervous that I would say that AI wrote something that you wrote, and you would be insulted!!!” Her anxiety, it turned out, was justified. She didn’t get any of them right.Dana blamed this partly on her not being a writer. But, of my seven readers, none correctly identified more than half the passages. One of the last people I heard from was the novelist Karan Mahajan, a professor of literary arts at Brown University. He and I learned to write together in college, along with Tony Tulathimutte, and have been sharing drafts with each other ever since. He’s among the most perceptive writers and readers I’ve met. “Oof, this was really confusing and mindmelting,” Karan wrote. Then he, too, misidentified all four excerpts.To be clear, Tuhin isn’t claiming that AI, except under the very specific, rather contrived circumstances of his experiments, can already match the best human writers. And he doesn’t know if they ever will be able to match or surpass us. I don’t know how much consolation that affords, though; what he has done is threatening enough. Moreover, in this episode we’re doing what is sometimes the most anxiety producing thing of all when it comes to fraught questions: We’re sitting in the uncertainty, and poking at it. How does his method work? What does and doesn’t it prove? Why did he think to do it in the first place? What are its implications? What technical challenges would need to be solved for it to move from short sample