Can a computer write a sonnet as well as Shakespeare?

AI or not AI? That is the question.

Computer scientists in Australia teamed up with an expert in the University of Toronto's department of English to design an algorithm that writes poetry following the rules of rhyme and meter. To test their results, the researchers asked people online to distinguish between human- and bot-written verses. The best version of the algorithm fooled people nearly 50 per cent of the time.

In some ways, the computer's verses were better than Shakespeare's. The rhymes and meter in the machine-generated poetry were more precise than in human-written poems. But "Deep-speare" still has a long way to go before it writes anything worthy of the Western canon.

"It's very easy for me to tell what's by a computer or not - ridiculously easy," says Adam Hammond, an assistant professor of English at U of T and co-author of the paper outlining the algorithm.

"We solved two out of four problems," Hammond says, referring to rhyme and meter. "The other two are much harder: making something that's readable and something that can evoke emotion in a reader."

Computer scientists at IBM Research Australia, the University of Melbourne, and Thomson Reuters trained a neural network using nearly 2,700 sonnets in Project Gutenberg, a free digital library. The computer uses three models - language, meter and rhyming - and probability to pick the right words for its poem. It produced quatrains, or four lines of verse, in iambic pentameter.

Couldn't fool expert

The researchers assessed their results by asking people online to tell the human and algorithm poetry apart. Most laymen - maybe confused by Elizabethan English - couldn't tell that verses like this one were the work of a programmed poet:

With joyous gambols gay and still array

No longer when he twas, while in his day

At first to pass in all delightful ways

Around him, charming and of all his days

But Deep-speare didn't fool the expert. Hammond says it was easy to spot the computer's verses because they were often incoherent and contained grammatical errors like the one above: "he twas."

He rated Deep-speare's quatrains on rhyme and meter, finding that they were better in these respects than poems written by humans. The reason? "You have meter in a poem to create expectation," he explains. "You do that so you can break the pattern. It's about creating expectations and then violating them at some point."

The break signals a change in the emotional state of the speaker, Hammond adds.

And as further comfort to poets who worry about being replaced by AI, Hammond says there is much more to a sonnet that a computer can't imitate - at least not yet. A sonnet contains three quatrains and ends with a couplet, two lines of verse. "It's actually a really complicated form because usually, the first bit proposes a problem, and then there's a turn, then the couplet that offers a solution," he says.

"Imagine teaching a computer to come up with a problem, express that problem, and then offer a solution to it. It's impossible to imagine."

And yet his co-authors appear undaunted. Hammond says that his fellow researchers want to fine-tune the algorithm so that it sticks to a single topic or design an algorithm that can write short fiction. "Our results suggest that future research should look beyond meter and focus on improving readability," the researchers say.

Their work was published last month in the proceedings of the Association for Computational Linguistics conference.

- Article courtesy U of T News, University of Toronto