4 ms·

Klara and the Sun Essay Contest – $1k Prize – AI Use Allowed

17 points by fkozlowski 3mo ago

john_strinlai 3mo ago
it would be cool if it was judged by humans, blinded to whether the submissions were 0% ai, some ai, 100% ai, etc. but it is judged by ai, which makes it much less cool in my opinion. to demonstrate the absurdity, the second best submission right now has a note "Literally the Fable essay, but with one letter changed", ranking 10 pts higher than the Fable one.
- fkozlowski 3mo ago
  It is actually judged by a human at the end FYI. "The top 5 submissions will be finalists, judged by hand."
  - UncleOxidant 3mo ago
    Kind of like submitting resumes these days. AI winnows them down and the remaining few get passed to humans.
    - sph 3mo ago
      And who knows what ‘gems’ might have been skipped by the machine.
  - john_strinlai 3mo ago
    it still loses its luster, for me, knowing that the top 5 are ai-selected
- deleted 3mo ago
  [deleted]
- UncleOxidant 3mo ago
  I'm trying to figure out why I should have any interest at all in participating in this? I mean, other than the $1000, but there's really no clear criteria as to what would win that $1000 since AI is judging it.
- 383toast 3mo ago
  I mean it's obvious that note is trolling
  - john_strinlai 3mo ago
    unless you have seen the submission, i have no idea how you could possibly know. so, if you don't mind, what are the 'tells'? what "obvious" things did i miss?
    - 383toast 3mo ago
      I worked a lot with designing and implementing agentic grading at my job, there's a ~0 chance that on a rubric with 180 items, changing 1 char out of 6000 words would result in a flip in 10+ grading items :)
stephantul 3mo ago
Is it even legal to publish excerpts of books like this? Or does this fall under some kind of exemption/fair use clause?
- bawolff 3mo ago
  Publishing an excerpt of a book for the purpose of critical commentary is very much fair use.
  - stephantul 3mo ago
    Ok, thanks!
    - bawolff 3mo ago
      As a general rule, if you are doing something "educational", fair use gives you very wide leelay, especially in the united states.
iamanllm 3mo ago
This is a beautiful and haunting book.
- deleted 3mo ago
  [deleted]
polalavik 3mo ago
> In today's world, what's the best way to get to a top-tier analytic essay? I'm curious to see. AI generated content graded by AI for what goal exactly? I don't understand this contest at all.
- stephantul 3mo ago
  It’s personal ad, basically. The author is trying to get a job as an evaluator somewhere and is hoping that putting 1000$ on the line will get them enough publicity to land them an interview/get a job somewhere.
- wpenman 3mo ago
  Thanks for the feedback, I should revise the copy there. I think when I started the contest I wasn't fully sure myself. Here are the goals that stand out to me. 1. Is auto-scoring an essay really possible? (Meaning, cost-effective, reliable, valid) This is more of a personal motivation - I worked hard on the 190 items and receiving responses will help me figure out if they're full of shit. 2. It would be super cool to see an “ideal” essay on this! AI safety is so topical. The contest asks entrants, in part, to say: Is Klara and the Sun in favor of AI safety efforts? If no, that’s awkward! If yes, on what grounds? You would think since I made up the rubric that I know what the "right" answer is here. But there's a lot of ambiguity in the text. 3. (For people writing their essays via AI) What is the value of prompting if/when we get to ASI? For most prompt engineering-style entries, I figure they aren't also a scholar in the humanities. So entering the contest is a test case of trying to prompt AI to do something "better" beyond what they can already do themselves. You could spin that up as prototypical of what life in general might be like in some amount of time. What prompt techniques even survive when you can't really assess what "Now make it better!" should look like? That's interesting! 4. (For people writing their essays as humans) Maybe this contest format is more humane than typical peer review? You always know where you stand. You get a score back in a few minutes, as soon as the scorer finishes going through the rubric. And the revision process is private, with no loss of face. In contrast, scholars in the humanities sometimes have to wait up to a year to hear back about a submission.
poly2it 3mo ago
Klara and the Sun is such a great book. Kazuo Ishiguro captured the intersection of technology and society in a way which can appeal to both to humanists and technologists. It was especially interesting to read his interpretation of partial sentience, learning and understanding the world from the lens of the artificial friends.
Karthick81 3mo ago
Are you publishing the rubric?
jasonpeacock 3mo ago
Trailer for the movie (coming soon): https://www.youtube.com/watch?v=KlwmEqQv82M https://www.youtube.com/watch?v=KlwmEqQv82M

17 points by fkozlowski 3mo ago

it would be cool if it was judged by humans, blinded to whether the submissions were 0% ai, some ai, 100% ai, etc. but it is judged by ai, which makes it much less cool in my opinion. to demonstrate the absurdity, the second best submission right now has a note "Literally the Fable essay, but with one letter changed", ranking 10 pts higher than the Fable one.

fkozlowski 3mo ago
It is actually judged by a human at the end FYI. "The top 5 submissions will be finalists, judged by hand."
- UncleOxidant 3mo ago
  Kind of like submitting resumes these days. AI winnows them down and the remaining few get passed to humans.
  - sph 3mo ago
    And who knows what ‘gems’ might have been skipped by the machine.
- john_strinlai 3mo ago
  it still loses its luster, for me, knowing that the top 5 are ai-selected
deleted 3mo ago
[deleted]
UncleOxidant 3mo ago
I'm trying to figure out why I should have any interest at all in participating in this? I mean, other than the $1000, but there's really no clear criteria as to what would win that $1000 since AI is judging it.
383toast 3mo ago
I mean it's obvious that note is trolling
- john_strinlai 3mo ago
  unless you have seen the submission, i have no idea how you could possibly know. so, if you don't mind, what are the 'tells'? what "obvious" things did i miss?
  - 383toast 3mo ago
    I worked a lot with designing and implementing agentic grading at my job, there's a ~0 chance that on a rubric with 180 items, changing 1 char out of 6000 words would result in a flip in 10+ grading items :)

stephantul 3mo ago

Is it even legal to publish excerpts of books like this? Or does this fall under some kind of exemption/fair use clause?

bawolff 3mo ago
Publishing an excerpt of a book for the purpose of critical commentary is very much fair use.
- stephantul 3mo ago
  Ok, thanks!
  - bawolff 3mo ago
    As a general rule, if you are doing something "educational", fair use gives you very wide leelay, especially in the united states.

iamanllm 3mo ago

This is a beautiful and haunting book.

deleted 3mo ago
[deleted]

polalavik 3mo ago

> In today's world, what's the best way to get to a top-tier analytic essay? I'm curious to see. AI generated content graded by AI for what goal exactly? I don't understand this contest at all.

stephantul 3mo ago
It’s personal ad, basically. The author is trying to get a job as an evaluator somewhere and is hoping that putting 1000$ on the line will get them enough publicity to land them an interview/get a job somewhere.
wpenman 3mo ago
Thanks for the feedback, I should revise the copy there. I think when I started the contest I wasn't fully sure myself. Here are the goals that stand out to me. 1. Is auto-scoring an essay really possible? (Meaning, cost-effective, reliable, valid) This is more of a personal motivation - I worked hard on the 190 items and receiving responses will help me figure out if they're full of shit. 2. It would be super cool to see an “ideal” essay on this! AI safety is so topical. The contest asks entrants, in part, to say: Is Klara and the Sun in favor of AI safety efforts? If no, that’s awkward! If yes, on what grounds? You would think since I made up the rubric that I know what the "right" answer is here. But there's a lot of ambiguity in the text. 3. (For people writing their essays via AI) What is the value of prompting if/when we get to ASI? For most prompt engineering-style entries, I figure they aren't also a scholar in the humanities. So entering the contest is a test case of trying to prompt AI to do something "better" beyond what they can already do themselves. You could spin that up as prototypical of what life in general might be like in some amount of time. What prompt techniques even survive when you can't really assess what "Now make it better!" should look like? That's interesting! 4. (For people writing their essays as humans) Maybe this contest format is more humane than typical peer review? You always know where you stand. You get a score back in a few minutes, as soon as the scorer finishes going through the rubric. And the revision process is private, with no loss of face. In contrast, scholars in the humanities sometimes have to wait up to a year to hear back about a submission.

poly2it 3mo ago

Klara and the Sun is such a great book. Kazuo Ishiguro captured the intersection of technology and society in a way which can appeal to both to humanists and technologists. It was especially interesting to read his interpretation of partial sentience, learning and understanding the world from the lens of the artificial friends.

Karthick81 3mo ago

Are you publishing the rubric?

jasonpeacock 3mo ago

Trailer for the movie (coming soon): https://www.youtube.com/watch?v=KlwmEqQv82M https://www.youtube.com/watch?v=KlwmEqQv82M