In Apr 2019, I managed to fine-tune OpenAI’s GPT-117M on poetry and wrote an article for TowardsDataScience about it. I then turned this into an Instagram poet called OSUN, giving it a persona and matching it with photos that I edited.
I think I know where I got the Tang Dynasty poetry bit. I remember reading a novel in school - an English translation of a long and almost soap-operatic work about a Chinese court. I don’t remember much of the details, but I remember the poetry the characters wrote to each other.
At this point I was working on the language overview paper with Nisansa, so I knew the basics. I wanted to try my hand at something more complex. OpenAI had just dropped the GPT-2 paper in Feb, so the moment the 117M model became available, I dashed into it and began retraining it to generate poetry. The outputs were interesting enough for me to blog about.
Simultaneously, I found myself somewhat amused by the rise of Instagram poetry and the reactions from authors and friends on my feed. So I ended up generating large numbers of poems with the fine-tuned model, sifting through them for good ones, and slowly uploading them to Instagram to see how people reacted:
Faces were generated with StyleGAN (see This Person Does Not Exist), except for the one Game of Thrones image. It was a chance to learn both some cutting-edge tech and teach myself Python (I operated mostly in R until that point).
To my surprise, it grew a very small following (100+ people). I used some more Python to automate its actions - following other poets, liking their work, following people back, etc. It would have been interesting to grow this experiment further, but at the time I was more interested in seeing what other things I could fine-tune the model into. GPT-117 Yeats was a mess.
When the GPT-2 345M model weights were released, I upgraded and reached out to Dr Samuel Peralta to see if we could explore a human + AI collaboration. Dr Peralta is an interest polymath; an Emmy-Award winning physicist, entrepreneur, science fiction writer and poet. He sent me his translations of Pablo Neruda, one of his favourite poets, and I fine-tuned OSUN further.
August, 2020: it was ready. I didn’t know what to do with it.
https://twitter.com/Semaphore/status/1292598634169733120
And then, to my immense surprise, Samuel decided to send this to the moon, alongside a couple other short stories of mine: a first version of the first chapter of Claws and Effect, and a short story titled “Beatnik”, set in the world of the Writing Contest. See more in: https://www.lunarcodex.com/
The Codex itself has grown over the years: from Samuel’s books, science fiction anthologies and the OSUN collaboration, to 30,000 artists of all stripes in four capsules, all of which will hopefully preserve our art for millions of years to come.
“Our hope is that future travelers who find this capsule will discover some of the richness of our world today,” Dr. Peralta said. “It speaks to the idea that, despite wars and pandemics and climate upheaval, humankind found time to dream, time to create art.”
The Peregrine Collection represents creative artists from all over the globe, including from Canada, the US, the U.K., Ireland, Belgium, Australia, Sri Lanka, Singapore, and the Philippines. It includes a collaborative human-AI work of poetry between Dr. Peralta and OSUN, an OpenAI-based machine programmed by Sri Lankan author and researcher Yudhanjaya Wijeratne.
Later, the same model gave me the poetry for the main character of The Salvage Crew - it was exactly the kind of melancholic, slightly alien voice I was looking for. It was also how I learned Python and how to set up and configure these large models - which in turn gave me my first exhibition, Auto | Biography.