Discover more from Ankur’s Newsletter
Artificial Intelligence in Writing
Hello readers —
Starting this week, I will provide an editorial on an important topic in artificial intelligence every week instead of my usual curated list of AI news. Readers have asked for this type of editorial to help them better understand how the latest developments in AI will affect them, both in business and in their personal lives. My goal is to inform, to inspire, and to elicit good conversation on all things AI with this editorial and would love your feedback along the way.
As always, thanks for reading.
P.S. There is now a search bar on the home page to help you find the content you need.
Artificial Intelligence in Writing
In its October 14, 2019 issue, The New Yorker released a piece titled, “Can A Machine Learn To Write For the New Yorker?” If AI could write as well as a writer for The New Yorker - one of the best-regarded publications in the world - does technology obviate the need for humans to write at all?
Before I answer that, let’s answer the most basic question first.
Where is artificial intelligence in writing today?
Artificial intelligence is remarkably good at predicting the next set of words in a sentence given the set of words it has seen in context. Anyone that has used Google’s Smart Compose can attest to this. We have come a long way from the more rudimentary “predict next word” features such as Apple’s QuickType and Google’s Smart Reply, which many of you use in your everyday lives.
The field of natural language processing (NLP), the subfield of artificial intelligence that involves text and speech, has advanced dramatically in the past four years and especially since last fall. Artificial intelligence is now able to generate much longer text - given previous context - and some of the longer text is reasonably coherent. To test for yourself, play with this generative text demo. Enter your own content to start and then press tab to generate new content.
AI generates coherent content at shorter sequences but becomes increasingly less coherent at longer sequences, entering into a dreamlike Faulkner or Daliesque state. In other words, AI is nowhere near the writing prowess of a professional writer at The New Yorker but, given the pace of advances in NLP and the compounding returns from this, AI could become stunningly good at most ordinary forms of writing in the next few years.
How did AI become “good” at writing?
Until more recently, researchers in this space believed the best way to advance AI was for experts to teach machines what they knew about language. In other words, these experts would design a set of rules for how to process language - this approach is knowledge-based or expert-based. Unfortunately, language is incredibly complicated - with many edge cases - and experts could not code enough rules fast enough to advance NLP in a material way. Progress in the field stalled.
The current state-of-the-art approach in NLP is data-based and relies on a branch of AI known as machine learning. This computer science-driven approach requires:
Lots of textual data (such as Wikipedia or Reddit data).
A specialized neural network architecture that handles long-range dependencies in text known as the Transformer.
Training for very long periods of time using lots of compute; one of the largest language models to date, XLNet, took 2000 GPU days to train.
This data-based approach dominates the space now, easily eclipsing the advances previously made by expert-based systems (read this The New York Times piece on Google Translate in 2016 for more). Feed more data into this neural net architecture for longer training times, and you will have a better-performing AI system.
Where do we go from here?
Despite being able to generate text well at shorter sequences - and somewhat at longer sequences - AI does not truly understand language. AI is using a probability distribution generated from learning from lots of data to predict the next word or sequence of words that follows a particular context the AI has been given. To evolve materially, AI will need to truly understand the meaning of language to form coherent arguments and thoughts as it writes.
That being said, AI is still very good at generating text that is seemingly real and could fool people on a mass scale. This problem only gets worse as AI gets smarter.
AI is also capable of generating good text summaries from data. Today, many post-game summaries in sports are generated by AI without readers even realizing it. The summaries are pretty basic and procedural, drawing on statistics from the game played, and are perfect for today’s AI to handle.
As AI continues to advance, it will be able to generate more of the written content we see. More of the writing humans do may come from AI rather than being generated by humans themselves. Imagine being able to feed an AI a few bullet points arguing in favor of or against a topic of your choice and then having the AI generate a coherent narrative weaving in those arguments.
That bring us full circle. To answer the original question, if AI could write as well as a writer for The New Yorker, humans would not need to write much at all. We are far from that reality, but we should expect more of the writing we do - especially the more basic variant - to come from AI in the years to come.
Here are some firms that are doing notable work in natural language and in generating text.
Grammarly: Assist human writers.
Quill: Make students better writers.
Narrative Science: Automate data-driven financial analyst-type written reports.
ScriptBook: Create screenplays and stories.
Persado: Generate creative marketing messages and monitor performance.
Phrasee: Perform copywriting at scale.
A future where AI empowers human to write better and more quickly is not one to be afraid of but one that we should champion and be proud of having created.