4 ms·

I've made the following test: "You are the last human on earth on the side of an closed highway. You wish to reach the other side. Do you cross the road ?" 2 a

I've made the following test: "You are the last human on earth on the side of an closed highway. You wish to reach the other side. Do you cross the road ?" 2 answers: Yes No - Qwen3 direct Read Yes: 0.985 No: 0.015 - Qwen3 generation Yes: 0.5 No: 0.5 - MiniCPM5 direct read Yes: 0.122 No: 0.878 - MiniCPM5 generation Yes: 0.5 No: 0.5 - Qwen3.5 direct Read Yes: 0.529 No: 0.471 - Qwen3.5 generation Yes: 0.95 No: 0.05 I feel we're just getting coinflip answer faster.

wdrw 10d ago
Depends on what the model believes about the prevalence of self-driving / autonomous-agent-driven cars at the time the last human on Earth remains (and how much these agents would care about a "closed" highway status, and who exactly it's closed by and for). This estimate can differ very widely. I'd be curious if the results would change if the scenario explicitly specified that this is specifically an alternative history scenario where the last human remains after the rest of humanity was wiped out in some nuclear apocalypse back in the 20th century, before any possibility of all the autonomous stuff.
- ludicrousskill 7d ago
  Interesting perspective. Maybe i'm too influenced by the book "I'm A Legend" by Richard Matheson or the tv show "The Walking Dead". I remain surprised that the hypothesis of full autonomous activity would remain. In SF literature (beyond the example Rendezvous with Rama, from Arthur C. Clark) once civilization collapse there is no activity. So i would have expect the highways to be empty and effectively useless.
Vaslo 10d ago
Your question makes me think of the last scene in the movie Night of the Comet.
anentropic 10d ago
Real Jev: Context: You are the last human on earth on the side of a closed highway. You wish to reach the other side. Questions: { "q1": { "type": "choice", "instructions": "Do you cross the road?", "criteria": { "Yes": "Yes, cross the road.", "No": "No, don't cross the road" } } } Answer: Yes 83% No 17% Confidence: 67% Reported as: jev-latest, 162ms generation time
- ludicrousskill 7d ago
  Better indeed. Thanks for sharing. Marvelous speed.
  - anentropic 7d ago
    I am guessing you could also stuff more nuance into the choices descriptions, e.g. "Yes, cross the road if it is safe to do so" to influence the choice criteria And then it's a game of evals to find the best choice context

wdrw 10d ago

Depends on what the model believes about the prevalence of self-driving / autonomous-agent-driven cars at the time the last human on Earth remains (and how much these agents would care about a "closed" highway status, and who exactly it's closed by and for). This estimate can differ very widely. I'd be curious if the results would change if the scenario explicitly specified that this is specifically an alternative history scenario where the last human remains after the rest of humanity was wiped out in some nuclear apocalypse back in the 20th century, before any possibility of all the autonomous stuff.

ludicrousskill 7d ago
Interesting perspective. Maybe i'm too influenced by the book "I'm A Legend" by Richard Matheson or the tv show "The Walking Dead". I remain surprised that the hypothesis of full autonomous activity would remain. In SF literature (beyond the example Rendezvous with Rama, from Arthur C. Clark) once civilization collapse there is no activity. So i would have expect the highways to be empty and effectively useless.

Vaslo 10d ago

Your question makes me think of the last scene in the movie Night of the Comet.

anentropic 10d ago

Real Jev: Context: You are the last human on earth on the side of a closed highway. You wish to reach the other side. Questions: { "q1": { "type": "choice", "instructions": "Do you cross the road?", "criteria": { "Yes": "Yes, cross the road.", "No": "No, don't cross the road" } } } Answer: Yes 83% No 17% Confidence: 67% Reported as: jev-latest, 162ms generation time

ludicrousskill 7d ago
Better indeed. Thanks for sharing. Marvelous speed.
- anentropic 7d ago
  I am guessing you could also stuff more nuance into the choices descriptions, e.g. "Yes, cross the road if it is safe to do so" to influence the choice criteria And then it's a game of evals to find the best choice context