I looked a gift horse in the mouth
And it's not a camel.
Could an AI chat app read Farsi better than I could? Years ago, I had the privilege of taking Farsi at UC Berkeley Extension, where they actually taught the script as well as spoken language. Should be an easy win for the AI chat apps though, because I can't really read Farsi anymore. But the trick is that a lot of Persian calligraphy is very interpretive.
A Persian friend brought this gift from Iran. The ghazal and its translation were shared with me at the time. I thought it would be an interesting experiment to see if transliteration of this was beyond the AI chat apps I use on a daily basis. I took a photo and passed it to ChatGPT, Claude, and Gemini. This was a while back, and all AI failed to identify the text as text.
Today, I tried again. All five models (ChatGPT 4o, ChatGPT 3o, Claude Opus 4, Claude Sonnet 4, and Gemini 2.5 Pro) could identify that there is a Hafez poem inscribed therein. Claude—both models—commented that the artwork was beautiful, with "intricate calligraphy and decorative elements," but could not identify the poem and suggested I consultant expert.
ChatGPT 3o got it right. ChatGPT 4o gave a false result. After several exchanges, ChatGPT 4o conceded that "Gemini is correct. The poem in your image is "اگر آن ترک شیرازی..." That is: If that Shirazi Turk takes my heart in hand, I would give away Samarkand and Bukhara for her Hindu beauty spot." And shared that its "earlier guess was based on the overall silhouette and structure... I didn’t zoom in to recognize the distinctive سمرقند و بخارا phrase."
It should not be lost that I am completely blown away that these translation services are available on my little phone. And this isn't a benchmark report, just a couple of thoughts:
1. ChatGPT 4o returned a confabulated answer, likely based on some threshold of confidence. But, couldn't I just use 3o? "If that Turk of Shiraz could only take my heart in hand, I’d give Samarkand and Bukhara for the dark mole on her cheek." Both 4o and 3o use the pronoun "her". Persian is a gender-neutral language.
Gemini's translation was stricter, "If that Turk of Shiraz would take my heart in hand, For his dark mole I would give Samarkand and Bukhara." As my friend tried to explain to me once, in "Farsi gender-neutral doesn't mean 'he or she'; it's all 'he'." ;-)
Given the pronouns as mechanics in proxy, however, "their" might be considered a more culturally neutral transliteration. Or not. I mean can you separate language and culture?
2. When asked to describe the photo, ChatGPT 3o gave an astonishingly detailed description, including identifying the horse. Both ChatGPT 4o and Gemini described it as a camel. With such a complex piece of artwork, it's difficult to distinguish all of the elements. But that's clearly a horse. And horses were symbolic and commonly used in Hafez's area of discourse. Camels? A conflation of the Arab and Persian cultures.
Yep, today, I looked a gift horse in the mouth; and it’s not a camel.