3 ms·

"Simulate surveys" makes it sound like a satirical AI product

callalex 3y ago
I was seriously searching through their webpage expecting a reference to the “conjoined triangles of success” to help indicate that this is all an elaborate self-aware prank. I was sorely disappointed to learn there is zero self-awareness here.
- famouswaffles 3y ago
  Self awareness for what lol ? If anything, existing research indicates you can just skip basic surveys and go for complex simulacra experiments. https://news.ycombinator.com/item?id=36868552 https://news.ycombinator.com/item?id=36868552
msoad 3y ago
Can't believe this is coming from PhD students from Princeton University! It's so obviously flawed.
- famouswaffles 3y ago
  Is it ? What little research we have of this kind of phenomena points towards this being very valid. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? (https://arxiv.org/abs/2301.07543 https://arxiv.org/abs/2301.07543) Out of One, Many: Using Language Models to Simulate Human Samples (https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899)
- ljm 3y ago
  A bayesian model will use previous data to calculate probabilities, so using AI to calculate a survey based on previous surveys sounds like a logical evolution no? That's something you might typically do by hand when running a survey and, say, comparing it to a benchmark. The problems will be similar to most AI problems I suppose: people who don't really understand the limitations of AI or the results it produces take the output of AI as gospel. My own thought is: what does it mean to 'simulate' a survey if the outcome is that people treat it as a 'live' or 'empirical' one?
- dang 3y ago
  "Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

I was seriously searching through their webpage expecting a reference to the “conjoined triangles of success” to help indicate that this is all an elaborate self-aware prank. I was sorely disappointed to learn there is zero self-awareness here.

famouswaffles 3y ago
Self awareness for what lol ? If anything, existing research indicates you can just skip basic surveys and go for complex simulacra experiments. https://news.ycombinator.com/item?id=36868552 https://news.ycombinator.com/item?id=36868552

msoad 3y ago

Can't believe this is coming from PhD students from Princeton University! It's so obviously flawed.

famouswaffles 3y ago
Is it ? What little research we have of this kind of phenomena points towards this being very valid. Large Language Models as Simulated Economic Agents: What Can We Learn from Homo Silicus? (https://arxiv.org/abs/2301.07543 https://arxiv.org/abs/2301.07543) Out of One, Many: Using Language Models to Simulate Human Samples (https://arxiv.org/abs/2209.06899 https://arxiv.org/abs/2209.06899)
ljm 3y ago
A bayesian model will use previous data to calculate probabilities, so using AI to calculate a survey based on previous surveys sounds like a logical evolution no? That's something you might typically do by hand when running a survey and, say, comparing it to a benchmark. The problems will be similar to most AI problems I suppose: people who don't really understand the limitations of AI or the results it produces take the output of AI as gospel. My own thought is: what does it mean to 'simulate' a survey if the outcome is that people treat it as a 'live' or 'empirical' one?
dang 3y ago
"Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html