5 ms·

A reasoning model is not meant for writing poetry. It's not very useful to evaluate it on such tasks.

mediaman 2y ago
It's not clear that writing poetry is a bad use case. Reasoning models seem to actually do pretty well with creative writing and poetry. Deepseek's R1, for example, has much better poem structure than the underlying V3, and writers are saying R1 was the first model where they actually felt like it was a useful writing companion. R1 seems to think at length about word choice, correcting structure, pentameter, and so on.
- 1propionyl 2y ago
  Indeed. I would assume that a reasoning model would do far better at things like actually maintaining meter or rhyme scheme, something that models (even with good attention mechanisms) generally do very poorly at.
- theturtle32 2y ago
  Ok, that makes some sense. I guess I was thinking more about the creative and abstract nature of poetry, the free flowing kind, not so much about rigid structures of meter and rhyme.
  - fivestones 2y ago
    Ha! So did you mean to have your answer shift into a poem half way through, or was that accidental? Nice.
mquander 2y ago
I tried to tell my English teachers that all through high school but it never worked.
aprilthird2021 2y ago
To be blunt, an AI isn't a good tool for writing poetry either. At least, not the kind people read as a high literature form. For commercials, jingles, Hallmark cards, etc. sure
DonHopkins 2y ago
There exists poetry that requires a lot of mathematical understanding! This is "literally" (and I mean literally in the literary sense) from a Stanislaw Lem story about an electronic bard, translated from Polish by Michael Kandel: https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.html https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.h... Prompt: A love poem, lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly, with a little topology and higher calculus, if need be. But with feeling, you understand, and in the cybernetic spirit. Response: Come, let us hasten to a higher plane, Where dyads tread the fairy fields of Venn, Their indices bedecked from one to n, Commingled in an endless Markov chain! Come, every frustum longs to be a cone, And every vector dreams of matrices. Hark to the gentle gradient of the breeze: It whispers of a more ergodic zone. In Riemann, Hilbert or in Banach space Let superscripts and subscripts go their ways. Our asymptotes no longer out of phase, We shall encounter, counting, face to face. I'll grant thee random access to my heart, Thou'lt tell me all the constants of thy love; And so we two shall all love's lemmas prove, And in our bound partition never part. For what did Cauchy know, or Christoffel, Or Fourier, or any Boole or Euler, Wielding their compasses, their pens and rulers, Of thy supernal sinusoidal spell? Cancel me not -- for what then shall remain? Abscissas, some mantissas, modules, modes, A root or two, a torus and a node: The inverse of my verse, a null domain. Ellipse of bliss, converse, O lips divine! The product of our scalars is defined! Cyberiad draws nigh, and the skew mind cuts capers like a happy haversine. I see the eigenvalue in thine eye, I hear the tender tensor in thy sigh. Bernoulli would have been content to die, Had he but known such a squared cosine 2 phi! From The Cyberiad, by Stanislaw Lem. Translated from Polish by Michael Kandel. Here's a previous discussion of Marcin Wichary's translation of one of Lem's stories from Polish to English. He created the Lem Google Doodle, and he stalked and met Stanislaw Lem when he was a boy. Plus a discussion of Michael Kandel's translation of the poetry of the Electric Bard from The First Sally of Cyberiad, comparing it to machine translation: https://news.ycombinator.com/item?id=28600200 https://news.ycombinator.com/item?id=28600200 Stanislaw Lem has finally gotten the translations his genius deserves: https://www.washingtonpost.com/entertainment/books/stanislaw-lem-has-finally-gotten-the-translations-his-genius-deserves/2020/03/03/3227b18c-5982-11ea-ab68-101ecfec2532_story.html https://www.washingtonpost.com/entertainment/books/stanislaw... >Lem’s fiction is filled with haunting, prescient landscapes. In these reissued and newly issued translations — some by the pitch-perfect Lem-o-phile, Michael Kandel — each sentence is as hard, gleaming and unpredictable as the next marvelous invention or plot twist. It’s hard to keep up with Lem’s hyper-drive of an imagination but always fun to try.
frontalier 2y ago
good thing we didn't prompt it because we want to read a poem then

It's not clear that writing poetry is a bad use case. Reasoning models seem to actually do pretty well with creative writing and poetry. Deepseek's R1, for example, has much better poem structure than the underlying V3, and writers are saying R1 was the first model where they actually felt like it was a useful writing companion. R1 seems to think at length about word choice, correcting structure, pentameter, and so on.

1propionyl 2y ago
Indeed. I would assume that a reasoning model would do far better at things like actually maintaining meter or rhyme scheme, something that models (even with good attention mechanisms) generally do very poorly at.
theturtle32 2y ago
Ok, that makes some sense. I guess I was thinking more about the creative and abstract nature of poetry, the free flowing kind, not so much about rigid structures of meter and rhyme.
- fivestones 2y ago
  Ha! So did you mean to have your answer shift into a poem half way through, or was that accidental? Nice.

mquander 2y ago

I tried to tell my English teachers that all through high school but it never worked.

aprilthird2021 2y ago

To be blunt, an AI isn't a good tool for writing poetry either. At least, not the kind people read as a high literature form. For commercials, jingles, Hallmark cards, etc. sure

DonHopkins 2y ago

There exists poetry that requires a lot of mathematical understanding! This is "literally" (and I mean literally in the literary sense) from a Stanislaw Lem story about an electronic bard, translated from Polish by Michael Kandel: https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.html https://www.donhopkins.com/home/catalog/lem/WonderfulPoems.h... Prompt: A love poem, lyrical, pastoral, and expressed in the language of pure mathematics. Tensor algebra mainly, with a little topology and higher calculus, if need be. But with feeling, you understand, and in the cybernetic spirit. Response: Come, let us hasten to a higher plane, Where dyads tread the fairy fields of Venn, Their indices bedecked from one to n, Commingled in an endless Markov chain! Come, every frustum longs to be a cone, And every vector dreams of matrices. Hark to the gentle gradient of the breeze: It whispers of a more ergodic zone. In Riemann, Hilbert or in Banach space Let superscripts and subscripts go their ways. Our asymptotes no longer out of phase, We shall encounter, counting, face to face. I'll grant thee random access to my heart, Thou'lt tell me all the constants of thy love; And so we two shall all love's lemmas prove, And in our bound partition never part. For what did Cauchy know, or Christoffel, Or Fourier, or any Boole or Euler, Wielding their compasses, their pens and rulers, Of thy supernal sinusoidal spell? Cancel me not -- for what then shall remain? Abscissas, some mantissas, modules, modes, A root or two, a torus and a node: The inverse of my verse, a null domain. Ellipse of bliss, converse, O lips divine! The product of our scalars is defined! Cyberiad draws nigh, and the skew mind cuts capers like a happy haversine. I see the eigenvalue in thine eye, I hear the tender tensor in thy sigh. Bernoulli would have been content to die, Had he but known such a squared cosine 2 phi! From The Cyberiad, by Stanislaw Lem. Translated from Polish by Michael Kandel. Here's a previous discussion of Marcin Wichary's translation of one of Lem's stories from Polish to English. He created the Lem Google Doodle, and he stalked and met Stanislaw Lem when he was a boy. Plus a discussion of Michael Kandel's translation of the poetry of the Electric Bard from The First Sally of Cyberiad, comparing it to machine translation: https://news.ycombinator.com/item?id=28600200 https://news.ycombinator.com/item?id=28600200 Stanislaw Lem has finally gotten the translations his genius deserves: https://www.washingtonpost.com/entertainment/books/stanislaw-lem-has-finally-gotten-the-translations-his-genius-deserves/2020/03/03/3227b18c-5982-11ea-ab68-101ecfec2532_story.html https://www.washingtonpost.com/entertainment/books/stanislaw... >Lem’s fiction is filled with haunting, prescient landscapes. In these reissued and newly issued translations — some by the pitch-perfect Lem-o-phile, Michael Kandel — each sentence is as hard, gleaming and unpredictable as the next marvelous invention or plot twist. It’s hard to keep up with Lem’s hyper-drive of an imagination but always fun to try.

frontalier 2y ago

good thing we didn't prompt it because we want to read a poem then