4 ms·

ok, i guess this isn't something we are supposed to admit, but there are many things i have learned over the years which have been distilled to thought-bites, r

ok, i guess this isn't something we are supposed to admit, but there are many things i have learned over the years which have been distilled to thought-bites, right? i often want to comment just short things, slogans almost. these AI have been invaluable in beefing up my slogans into longer statements with the un-distilled facts laid out in bullet points. it's pretty helpful as an argument de-compression tool. but that has pretty serious epistemological concerns. and to the people who are concerned about that - well i think that's funny... maybe take a walk down the science of how poor memory really is. i guess i just have a very attenuated view of human (including my own) rationality. it's waaaaay more inflated for the average human than we're willing to admit. i made a comment early on in December about the oncoming epistemological crisis and damn has it been really fun to watch unfold the last few months.

luckylion 4y ago
An advanced specialized Model could probably create fleshed out arguments from your thought-bites and subtly manipulate you by weaving other positions you don't/didn't have into the argument. "You're right, because xyz is correct and that makes your argument correct", making you more likely to accept xyz as correct, because we like being right and if accepting xyz makes you right in this very moment, that's a small price to pay. it feels like that would require much more sophistication, but then again who hasn't been amazed how intelligent ChatGPT & friends feel?
- LesZedCB 4y ago
  the RLHF has indeed made them quite sycophantic. i don't believe in any of the 'latent space wario' stuff, but i really enjoy using it explicitly to get clarification and explanation from opposing viewpoints. i would love to see some experiment where a debate is held/moderated by an LLM which can help find common ground in highly contentious viewpoints. the post-mortem from each side would be a fascinating read.

luckylion 4y ago

An advanced specialized Model could probably create fleshed out arguments from your thought-bites and subtly manipulate you by weaving other positions you don't/didn't have into the argument. "You're right, because xyz is correct and that makes your argument correct", making you more likely to accept xyz as correct, because we like being right and if accepting xyz makes you right in this very moment, that's a small price to pay. it feels like that would require much more sophistication, but then again who hasn't been amazed how intelligent ChatGPT & friends feel?

LesZedCB 4y ago
the RLHF has indeed made them quite sycophantic. i don't believe in any of the 'latent space wario' stuff, but i really enjoy using it explicitly to get clarification and explanation from opposing viewpoints. i would love to see some experiment where a debate is held/moderated by an LLM which can help find common ground in highly contentious viewpoints. the post-mortem from each side would be a fascinating read.