10 ms·

Humans in Humans Out: GPT Converging Toward Common Sense in Both Success/Failure

60 points by elsewhen 3y ago

transfire 3y ago
This is a social-political problem, not a technological one. And unfortunately it’s going to be a lot worse before it gets better. Our political systems are not equipped to handle it (properly). Maybe we should think about replacing government with AI.
- parthianshotgun 3y ago
  I hope you're being silly
- emehex 3y ago
  This is the plot of QualityLand (https://www.goodreads.com/book/show/36216607-qualityland https://www.goodreads.com/book/show/36216607-qualityland). Highly recommended!
  - Brian_K_White 3y ago
    Thank you. I'll trade you a Jennifer Government by Max Barry (not the same premis just a recommendation, but is also a take on the progression of corporations)
  - sdenton4 3y ago
    And Alphaville! https://en.m.wikipedia.org/wiki/Alphaville_(film) https://en.m.wikipedia.org/wiki/Alphaville_(film)
- mirekrusin 3y ago
  Aiocracy. Sounds like one of those comical human predictions that turns out to be 170 degrees wrong.
- fnordpiglet 3y ago
  Good try, ChatGPT. We aren’t falling for your take over attempts just yet.
- netruk44 3y ago
  We'll replace government with AI just as soon as we get blockchain voting. In other words, it won't happen. People won't (and shouldn't) trust the tech enough.
- airstrike 3y ago
  Maybe we start small and replace just the Defense department with AI to make rational decisions and avoid the possibility that human emotions lead to dangerous conflicts https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project
- plorg 3y ago
  Our political systems are not equipped to handle these problems anyways. They aren't problems of fact (though when factual disagreements occur it makes problems worse), but of needs, desires, priorities, and values. The magic computer will not resolve these disagreements just because it comes to the "right" or "most common sense" answer.
  - fnordpiglet 3y ago
    42
- 13years 3y ago
  > Maybe we should think about replacing government with AI. I think that would be a mistake. It is seemingly apparent that a machine may be able to not have bias as it is just performing calculations. It is not emotional in its decisions. However, the implementation in reality will likely diverge from that significantly. I've explained this much further as the Bias Paradox - https://dakara.substack.com/p/ai-the-bias-paradox https://dakara.substack.com/p/ai-the-bias-paradox
traviswt 3y ago
Very fascinating. If we train on every piece of knowledge, then statistically that should average around 100IQ right? It's not perfect analogy, but the model is shooting more for consensus, a bit hive mind. It feels like to improve the quality of the model, we need to be more selective of training data.
- dTal 3y ago
  It's not entirely clear what precisely IQ is measuring. It's not a stretch that averaging the output of a bunch of 100 IQ people yields something better than what any one of them could have produced. Perhaps "there's only one way to be right, but many ways to be wrong".
  - prettyStandard 3y ago
    We know what IQ is. It's the ability to think abstractly. Even more precisely it's the ability to do IQ tests.
    - QuiDortDine 3y ago
      This is a common belief that used to be true, but no longer is. IQ tests are reasonably good at measuring general intelligence, ie the g factor : https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
      - prettyStandard 3y ago
        I'm glad you posted this. Sometimes I try to describe this gfactor to people, saying "the best indicator of intelligence is any other intelligence". Putting a name on things helps people understand I am not a total bullshiter. Why do you say that this is a common belief that used to be true, but is now no longer true?
- PartiallyTyped 3y ago
  IQ is not measurable by knowledge, but by speed of execution and understanding through limited information.
ingenieroariel 3y ago
After reading the abstract I can't help but think we only want to feed a coding AI good code. Is there an awesome-code repo that lists all the repos we consider above average?
- jfk13 3y ago
  Can't you just ask ChatGPT? Surely it's seen so much that it'll "know"....
  - ingenieroariel 3y ago
    https://gist.github.com/ingenieroariel/39bc74f56d7adef437ed42df94f8fd24 https://gist.github.com/ingenieroariel/39bc74f56d7adef437ed4... Linus Torvalds (Creator of the Linux Kernel) - https://github.com/torvalds https://github.com/torvalds John Carmack (Co-founder of id Software) - https://github.com/ID_AA_Carmack https://github.com/ID_AA_Carmack Bjarne Stroustrup (Creator of C++) - https://github.com/BjarneStroustrup https://github.com/BjarneStroustrup Fabrice Bellard (Creator of QEMU, FFMpeg, and Tiny C Compiler) - https://github.com/fbellard https://github.com/fbellard Andrei Alexandrescu (C++ expert and author) - https://github.com/incomputable https://github.com/incomputable Chandler Carruth (LLVM and Clang developer) - https://github.com/chandlerc https://github.com/chandlerc Daniel Lemire (Computer science researcher, focuses on performance) - https://github.com/lemire https://github.com/lemire P.J. Plauger - A renowned author, and contributor to the C Standard Library - https://github.com/pjplauger https://github.com/pjplauger Peter J. Weinberger - Co-creator of AWK and a contributor to Unix - https://github.com/pjw https://github.com/pjw Keith Packard - A prominent contributor to the X Window System, and the Linux graphics stack - https://github.com/keith-packard https://github.com/keith-packard Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker https://github.com/richfelker Julia Evans - Systems programmer known for her work on C projects and educational materials - https://github.com/jvns https://github.com/jvns Tony Finch - Author of several C projects, including the Knot DNS server and other infrastructure tools - https://github.com/fanf https://github.com/fanf Sergey Bratus - A security researcher and expert in low-level programming, with a focus on C - https://github.com/sergeybratus https://github.com/sergeybratus Herb Sutter - A prominent C++ expert, author, and chair of the ISO C++ standards committee - https://github.com/hsutter https://github.com/hsutter Sean Parent - A software engineer at Adobe and C++ expert, known for his talks on C++ best practices - https://github.com/sean-parent https://github.com/sean-parent Kate Gregory - A C++ expert, author, and Microsoft Regional Director - https://github.com/gregcons https://github.com/gregcons Eric Niebler - A C++ expert and author of the range-v3 library, which inspired C++20's ranges - https://github.com/ericniebler https://github.com/ericniebler Jason Turner - A C++ expert and host of the C++ Weekly YouTube series - https://github.com/lefticus https://github.com/lefticus Titus Winters - A software engineer at Google and C++ expert, known for his work on the Google C++ Style Guide and Abseil - https://github.com/tituswinters https://github.com/tituswinters Jonathan Boccara - A C++ expert and author of the book "The Legacy Code Programmer's Toolbox" - https://github.com/joboccara https://github.com/joboccara Richard Stallman - Founder of the GNU project, which aims to create a complete, portable, and free Unix-like operating system - https://github.com/rms https://github.com/rms Samuel L. Miller - A core developer of the SQLite project, a self-contained, serverless, zero-configuration, transactional SQL database engine - https://github.com/slmiller https://github.com/slmiller Frank Denis - Creator of the libsodium project, a modern, easy-to-use software library for encryption, decryption, signatures, password hashing, and more - https://github.com/jedisct1 https://github.com/jedisct1 Michael Kerrisk - Author of "The Linux Programming Interface" and contributor to portable, multiplatform code in the Linux kernel and glibc - https://github.com/mtk-mirror https://github.com/mtk-mirror Salvatore Sanfilippo - Creator of the Redis project, an open-source, in-memory data structure store, used as a database, cache, and message broker - https://github.com/antirez https://github.com/antirez Niels Provos - A security researcher and developer who has contributed to various projects, including libevent, a library that provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout - https://github.com/provos https://github.com/provos Alon Zakai - Creator of Emscripten, a compiler that allows developers to run C and C++ code on the web, using WebAssembly - https://github.com/kripken https://github.com/kripken Brian W. Kernighan - Co-author of "The C Programming Language" and "The Practice of Programming" - https://github.com/bwk https://github.com/bwk Dennis Ritchie - Creator of the C programming language and co-author of "The C Programming Language" - https://github.com/dmr https://github.com/dmr (GitHub profile not available, as he passed away in 2011) Dan Gookin - Author of "C For Dummies" and other programming books - https://github.com/DanGookin https://github.com/DanGookin Zed A. Shaw - Author of "Learn C the Hard Way" and other programming books - https://github.com/zedshaw https://github.com/zedshaw Mark Lutz - Author of "C Programming for the Absolute Beginner" and Python books - https://github.com/learning-python https://github.com/learning-python Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker https://github.com/richfelker William A. Rowe Jr. - A key contributor to the Apache Portable Runtime (APR) project, which includes a custom libc implementation for the Apache HTTP Server - https://github.com/wrowe https://github.com/wrowe Stefan Tomanek - Creator of dietlibc, a libc optimized for small size - https://github.com/stefan-tomanek https://github.com/stefan-tomanek (The dietlibc project itself doesn't have an official GitHub repository, but you can find it at https://www.fefe.de/dietlibc/ https://www.fefe.de/dietlibc/) Szabolcs Nagy - A contributor to the musl libc project and creator of the math library used in musl - https://github.com/szabolcsnagy https://github.com/szabolcsnagy Alexei Starovoitov - Creator of the uClibc-ng project, a small C library for developing embedded Linux systems - https://github.com/ast-uClibc https://github.com/ast-uClibc (The uClibc-ng project can be found at https://github.com/wbx-github/uClibc-ng https://github.com/wbx-github/uClibc-ng)
jongjong 3y ago
The other day, I had a long debate with ChatGPT about a complex topic (about software design/architecture). I made a contrarian statement about this topic which had taken me over a decade to figure out. Initially, it strongly disagreed with me and it gave me the standard list of reasons in support of the current mainstream/consensus position. Then I basically explained in great detail why each of those reasons was invalid (or not significant). Then, after a long discussion and exposing contradictions in the mainstream consensus, it stopped disagreeing with me and was essentially agreeing with my position. It was refreshing to have a discussion where the other party is actually listening to my arguments. Too bad it doesn't retain the info from the discussion though... I realized that I could not have such a discussion with the vast majority of people in my industry, even if they were fully open minded; primarily because most people would not have so much knowledge at their disposal as ChatGPT had. I really felt that it had ALL of the knowledge on the subject. I just had to point it to show it the contradictions in the knowledge which it possessed. It feels like it is able to reason from first principles and synthesize information but it often chooses to present the consensus view by default. You have to really draw out its knowledge in order to make it override the consensus view. Kind of reminds me of System 1 (fast) thinking versus System 2 (slow) thinking from the book "Thinking fast and slow."
- jancsika 3y ago
  > Then, after a long discussion and exposing contradictions in the mainstream consensus, it stopped disagreeing with me and was essentially agreeing with my position. You need to go back and argue what you already know to be a wrong position with an equal amount of persuasively written arguments. My hypothesis from playing with Alpaca 13B is that you'll have ChatGPT agreeing with you in roughly the same amount of time. And you ought to know whether my hypothesis is right or wrong before describing ChatGPT as "actually listening" to your arguments. Otherwise, you risk falling in love with the wrong chat bot.
  - jongjong 3y ago
    I tried to convince it of an incorrect position about a basic topic but I couldn't continue. It made some very detailed, air-tight arguments (went beyond of my expertise). I would have had to resort to disputing definitions to keep that argument going. With my previous debate, it never touched on definitions; it was all logic-based.
badloginagain 3y ago
I forget where, but someone from OpenAI said that that the next GPT is "all about the data". I wonder if you have pre-training data that is fully factual, high quality, with perfect logic will result in a LLM that doesn't fall into reasoning gaps humans fall into. Or are these reasoning gaps an innate component of intelligence? Probably the most exciting thing about AI development is we get to start testing the things that make us uniquely human.
- ChatGTP 3y ago
  My bet is it’s just the scale of the data required. It’s like brute forcing every single answer to every single question.
  - musingsole 3y ago
    Rainbow tables didn't pan out before; they won't this time either.
deepzn 3y ago
Some basic reasoning it still is very bad at surprisingly. I understand it cannot plan. But for example, I used this from an IQ test - Robert is taller than John. Charlie is taller than Robert. Therefore, John is the shortest of the three. It outputs, an explanation that John necessarily is not the shortest. This was GPT 3.5, or the regular version of ChatGPT. Not sure on 4.

60 points by elsewhen 3y ago

This is a social-political problem, not a technological one. And unfortunately it’s going to be a lot worse before it gets better. Our political systems are not equipped to handle it (properly). Maybe we should think about replacing government with AI.

parthianshotgun 3y ago
I hope you're being silly
emehex 3y ago
This is the plot of QualityLand (https://www.goodreads.com/book/show/36216607-qualityland https://www.goodreads.com/book/show/36216607-qualityland). Highly recommended!
- Brian_K_White 3y ago
  Thank you. I'll trade you a Jennifer Government by Max Barry (not the same premis just a recommendation, but is also a take on the progression of corporations)
- sdenton4 3y ago
  And Alphaville! https://en.m.wikipedia.org/wiki/Alphaville_(film) https://en.m.wikipedia.org/wiki/Alphaville_(film)
mirekrusin 3y ago
Aiocracy. Sounds like one of those comical human predictions that turns out to be 170 degrees wrong.
fnordpiglet 3y ago
Good try, ChatGPT. We aren’t falling for your take over attempts just yet.
netruk44 3y ago
We'll replace government with AI just as soon as we get blockchain voting. In other words, it won't happen. People won't (and shouldn't) trust the tech enough.
airstrike 3y ago
Maybe we start small and replace just the Defense department with AI to make rational decisions and avoid the possibility that human emotions lead to dangerous conflicts https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project https://en.wikipedia.org/wiki/Colossus:_The_Forbin_Project
plorg 3y ago
Our political systems are not equipped to handle these problems anyways. They aren't problems of fact (though when factual disagreements occur it makes problems worse), but of needs, desires, priorities, and values. The magic computer will not resolve these disagreements just because it comes to the "right" or "most common sense" answer.
- fnordpiglet 3y ago
  42
13years 3y ago
> Maybe we should think about replacing government with AI. I think that would be a mistake. It is seemingly apparent that a machine may be able to not have bias as it is just performing calculations. It is not emotional in its decisions. However, the implementation in reality will likely diverge from that significantly. I've explained this much further as the Bias Paradox - https://dakara.substack.com/p/ai-the-bias-paradox https://dakara.substack.com/p/ai-the-bias-paradox

traviswt 3y ago

Very fascinating. If we train on every piece of knowledge, then statistically that should average around 100IQ right? It's not perfect analogy, but the model is shooting more for consensus, a bit hive mind. It feels like to improve the quality of the model, we need to be more selective of training data.

dTal 3y ago
It's not entirely clear what precisely IQ is measuring. It's not a stretch that averaging the output of a bunch of 100 IQ people yields something better than what any one of them could have produced. Perhaps "there's only one way to be right, but many ways to be wrong".
- prettyStandard 3y ago
  We know what IQ is. It's the ability to think abstractly. Even more precisely it's the ability to do IQ tests.
  - QuiDortDine 3y ago
    This is a common belief that used to be true, but no longer is. IQ tests are reasonably good at measuring general intelligence, ie the g factor : https://en.wikipedia.org/wiki/G_factor_(psychometrics) https://en.wikipedia.org/wiki/G_factor_(psychometrics)
    - prettyStandard 3y ago
      I'm glad you posted this. Sometimes I try to describe this gfactor to people, saying "the best indicator of intelligence is any other intelligence". Putting a name on things helps people understand I am not a total bullshiter. Why do you say that this is a common belief that used to be true, but is now no longer true?
PartiallyTyped 3y ago
IQ is not measurable by knowledge, but by speed of execution and understanding through limited information.

ingenieroariel 3y ago

After reading the abstract I can't help but think we only want to feed a coding AI good code. Is there an awesome-code repo that lists all the repos we consider above average?

jfk13 3y ago
Can't you just ask ChatGPT? Surely it's seen so much that it'll "know"....
- ingenieroariel 3y ago
  https://gist.github.com/ingenieroariel/39bc74f56d7adef437ed42df94f8fd24 https://gist.github.com/ingenieroariel/39bc74f56d7adef437ed4... Linus Torvalds (Creator of the Linux Kernel) - https://github.com/torvalds https://github.com/torvalds John Carmack (Co-founder of id Software) - https://github.com/ID_AA_Carmack https://github.com/ID_AA_Carmack Bjarne Stroustrup (Creator of C++) - https://github.com/BjarneStroustrup https://github.com/BjarneStroustrup Fabrice Bellard (Creator of QEMU, FFMpeg, and Tiny C Compiler) - https://github.com/fbellard https://github.com/fbellard Andrei Alexandrescu (C++ expert and author) - https://github.com/incomputable https://github.com/incomputable Chandler Carruth (LLVM and Clang developer) - https://github.com/chandlerc https://github.com/chandlerc Daniel Lemire (Computer science researcher, focuses on performance) - https://github.com/lemire https://github.com/lemire P.J. Plauger - A renowned author, and contributor to the C Standard Library - https://github.com/pjplauger https://github.com/pjplauger Peter J. Weinberger - Co-creator of AWK and a contributor to Unix - https://github.com/pjw https://github.com/pjw Keith Packard - A prominent contributor to the X Window System, and the Linux graphics stack - https://github.com/keith-packard https://github.com/keith-packard Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker https://github.com/richfelker Julia Evans - Systems programmer known for her work on C projects and educational materials - https://github.com/jvns https://github.com/jvns Tony Finch - Author of several C projects, including the Knot DNS server and other infrastructure tools - https://github.com/fanf https://github.com/fanf Sergey Bratus - A security researcher and expert in low-level programming, with a focus on C - https://github.com/sergeybratus https://github.com/sergeybratus Herb Sutter - A prominent C++ expert, author, and chair of the ISO C++ standards committee - https://github.com/hsutter https://github.com/hsutter Sean Parent - A software engineer at Adobe and C++ expert, known for his talks on C++ best practices - https://github.com/sean-parent https://github.com/sean-parent Kate Gregory - A C++ expert, author, and Microsoft Regional Director - https://github.com/gregcons https://github.com/gregcons Eric Niebler - A C++ expert and author of the range-v3 library, which inspired C++20's ranges - https://github.com/ericniebler https://github.com/ericniebler Jason Turner - A C++ expert and host of the C++ Weekly YouTube series - https://github.com/lefticus https://github.com/lefticus Titus Winters - A software engineer at Google and C++ expert, known for his work on the Google C++ Style Guide and Abseil - https://github.com/tituswinters https://github.com/tituswinters Jonathan Boccara - A C++ expert and author of the book "The Legacy Code Programmer's Toolbox" - https://github.com/joboccara https://github.com/joboccara Richard Stallman - Founder of the GNU project, which aims to create a complete, portable, and free Unix-like operating system - https://github.com/rms https://github.com/rms Samuel L. Miller - A core developer of the SQLite project, a self-contained, serverless, zero-configuration, transactional SQL database engine - https://github.com/slmiller https://github.com/slmiller Frank Denis - Creator of the libsodium project, a modern, easy-to-use software library for encryption, decryption, signatures, password hashing, and more - https://github.com/jedisct1 https://github.com/jedisct1 Michael Kerrisk - Author of "The Linux Programming Interface" and contributor to portable, multiplatform code in the Linux kernel and glibc - https://github.com/mtk-mirror https://github.com/mtk-mirror Salvatore Sanfilippo - Creator of the Redis project, an open-source, in-memory data structure store, used as a database, cache, and message broker - https://github.com/antirez https://github.com/antirez Niels Provos - A security researcher and developer who has contributed to various projects, including libevent, a library that provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a timeout - https://github.com/provos https://github.com/provos Alon Zakai - Creator of Emscripten, a compiler that allows developers to run C and C++ code on the web, using WebAssembly - https://github.com/kripken https://github.com/kripken Brian W. Kernighan - Co-author of "The C Programming Language" and "The Practice of Programming" - https://github.com/bwk https://github.com/bwk Dennis Ritchie - Creator of the C programming language and co-author of "The C Programming Language" - https://github.com/dmr https://github.com/dmr (GitHub profile not available, as he passed away in 2011) Dan Gookin - Author of "C For Dummies" and other programming books - https://github.com/DanGookin https://github.com/DanGookin Zed A. Shaw - Author of "Learn C the Hard Way" and other programming books - https://github.com/zedshaw https://github.com/zedshaw Mark Lutz - Author of "C Programming for the Absolute Beginner" and Python books - https://github.com/learning-python https://github.com/learning-python Rich Felker - Founder of the musl libc project, an alternative C library for Linux-based systems - https://github.com/richfelker https://github.com/richfelker William A. Rowe Jr. - A key contributor to the Apache Portable Runtime (APR) project, which includes a custom libc implementation for the Apache HTTP Server - https://github.com/wrowe https://github.com/wrowe Stefan Tomanek - Creator of dietlibc, a libc optimized for small size - https://github.com/stefan-tomanek https://github.com/stefan-tomanek (The dietlibc project itself doesn't have an official GitHub repository, but you can find it at https://www.fefe.de/dietlibc/ https://www.fefe.de/dietlibc/) Szabolcs Nagy - A contributor to the musl libc project and creator of the math library used in musl - https://github.com/szabolcsnagy https://github.com/szabolcsnagy Alexei Starovoitov - Creator of the uClibc-ng project, a small C library for developing embedded Linux systems - https://github.com/ast-uClibc https://github.com/ast-uClibc (The uClibc-ng project can be found at https://github.com/wbx-github/uClibc-ng https://github.com/wbx-github/uClibc-ng)

jongjong 3y ago

The other day, I had a long debate with ChatGPT about a complex topic (about software design/architecture). I made a contrarian statement about this topic which had taken me over a decade to figure out. Initially, it strongly disagreed with me and it gave me the standard list of reasons in support of the current mainstream/consensus position. Then I basically explained in great detail why each of those reasons was invalid (or not significant). Then, after a long discussion and exposing contradictions in the mainstream consensus, it stopped disagreeing with me and was essentially agreeing with my position. It was refreshing to have a discussion where the other party is actually listening to my arguments. Too bad it doesn't retain the info from the discussion though... I realized that I could not have such a discussion with the vast majority of people in my industry, even if they were fully open minded; primarily because most people would not have so much knowledge at their disposal as ChatGPT had. I really felt that it had ALL of the knowledge on the subject. I just had to point it to show it the contradictions in the knowledge which it possessed. It feels like it is able to reason from first principles and synthesize information but it often chooses to present the consensus view by default. You have to really draw out its knowledge in order to make it override the consensus view. Kind of reminds me of System 1 (fast) thinking versus System 2 (slow) thinking from the book "Thinking fast and slow."

jancsika 3y ago
> Then, after a long discussion and exposing contradictions in the mainstream consensus, it stopped disagreeing with me and was essentially agreeing with my position. You need to go back and argue what you already know to be a wrong position with an equal amount of persuasively written arguments. My hypothesis from playing with Alpaca 13B is that you'll have ChatGPT agreeing with you in roughly the same amount of time. And you ought to know whether my hypothesis is right or wrong before describing ChatGPT as "actually listening" to your arguments. Otherwise, you risk falling in love with the wrong chat bot.
- jongjong 3y ago
  I tried to convince it of an incorrect position about a basic topic but I couldn't continue. It made some very detailed, air-tight arguments (went beyond of my expertise). I would have had to resort to disputing definitions to keep that argument going. With my previous debate, it never touched on definitions; it was all logic-based.

badloginagain 3y ago

I forget where, but someone from OpenAI said that that the next GPT is "all about the data". I wonder if you have pre-training data that is fully factual, high quality, with perfect logic will result in a LLM that doesn't fall into reasoning gaps humans fall into. Or are these reasoning gaps an innate component of intelligence? Probably the most exciting thing about AI development is we get to start testing the things that make us uniquely human.

ChatGTP 3y ago
My bet is it’s just the scale of the data required. It’s like brute forcing every single answer to every single question.
- musingsole 3y ago
  Rainbow tables didn't pan out before; they won't this time either.

deepzn 3y ago

Some basic reasoning it still is very bad at surprisingly. I understand it cannot plan. But for example, I used this from an IQ test - Robert is taller than John. Charlie is taller than Robert. Therefore, John is the shortest of the three. It outputs, an explanation that John necessarily is not the shortest. This was GPT 3.5, or the regular version of ChatGPT. Not sure on 4.