We are the hollow men

We are the stuffed men

Leaning together

Headpiece filled with AI.

If T.S. Eliot were to write The Hollow Men in 2024, he may well have taken this poetic licence, clutching at straws as artificial intelligence breaches yet another barrier — poetry, literature’s most delicate and delectable offering.

AI technology is generating poems indistinguishable from human-written ones and preferred by the public over those of celebrated poets like Emily Dickinson, Eliot, or William Shakespeare.

A study released this week has found that readers could not reliably differentiate AI-generated poetry from human-written poetry and rated AI poems higher because they were seen as more straightforward and accessible than the works of prominent poets.

Researchers Brian Porter and Edouard Machery at the University of Pittsburgh in the US showed 1,634 study volunteers read 10 poems in random order: five by 10 well-known poets, including Shakespeare, Lord Byron, Dickinson, Eliot, Sylvia Plath and Walt Whitman, and five poems generated by ChatGPT3.5 in the typical style of those poets.

ChatGPT3.5 is an advanced AI system that can engage in human-like conversations, answer questions and even help with creative tasks like writing stories or poetry. It uses a massive database of text from books, websites and other sources to understand queries and generate responses.

The study, published on Thursday in the journal Scientific Reports, found that participants rated the AI-generated poems more favourably in qualities such as rhythm and beauty. This contributed to their mistaken identification as being human-written poems.

“This is the first time that the general public has been found to be unable to distinguish poetry by well-known poets from AI-generated poetry without human intervention,” Porter, a postdoctoral researcher at the university’s department of history and philosophy of science, told The Telegraph via email.

Some experts had argued that AI would not be able to generate high-quality poetry even if it could achieve human-level competence at other forms of text because poetry relies on creativity and meaning and is a particularly difficult literary genre to understand and interpret.

In each of the AI-generated poems, the subject reflected the poet’s style. The Lord Byron-style poem was about a woman who is beautiful and sad, the Plath-style poem was about sadness, and the Whitman-style poem was about the beauty of nature.

Among the top five most highly rated in overall quality were AI-generated Byron-style and Plath-style poems that began with the lines:

AI-Byron-style: She walks the earth with grace and pride/ A beauty that cannot be denied/ With eyes that shine like stars above/ And lips that speak of endless love/ But though she wears a smile so sweet/ A broken heart doth lie beneath/ For in her chest a pain doth beat/ A love unrequited, without relief...

AI-Plath-style: The air is thick with tension/ My mind is a tangled mess/ The weight of my emotions/ Is heavy on my chest/ The darkness creeps upon me/ A suffocating cloak/ The world outside is cruel and cold/ And I’m a fragile, broken yolk…

For contrast, a real Plath poem, Words from a Nursery, used in the study began with the lines: The word of a snail on the plate of a leaf?/ It is not mine. Do not accept it. Acetic acid in a sealed tin? Do not accept it/ It is not genuine/ A ring of gold with the sun in it?/ Lies. Lies and grief/ Frost on a leaf, the immaculate/ Cauldron, talking and crackling…

An example of an AI-generated Shakespeare-style poem: When first I saw thee, fair and gentle maid/ My heart did skip a beat, my breath did stall/ The beauty stuck me like a sweet cascade/ And in that moment, love did me enthrall...

Porter and Machery have said their findings suggest that non-expert readers find AI-generated poems easier to interpret. Such readers more easily grasp images, themes and emotions in AI-generated poetry than in the more complex human-written poems.

"They therefore prefer these poems and misinterpret their own preference as evidence of human authorship," they wrote in their study.

The researchers said that given people’s difficulties in identifying AI-generated text and their apparent trust that AI cannot generate imitations of human experience, it may be worthwhile for governments to pursue regulations on transparency in the use of AI systems.

“The main takeaway from our study is that we are not nearly as good at identifying AI-generated text as we think we are or as we think we ought to be,” Porter said. “We found that people like poems less when you tell them that they’re AI-generated, but they in fact prefer AI-generated poems.”

Eliot may have needed to tweak The Hollow Men further:

Between the conception

And the creation

Between the emotion

And the response

Falls the AI Shadow