4 ms·

It’s that good in your experience? Enough that it is close to writing something as nice as the Linux Kernel we have now? How close is close? Reading comments li

It’s that good in your experience? Enough that it is close to writing something as nice as the Linux Kernel we have now? How close is close? Reading comments like these makes me feel like I’m swallowing crazy pills. It legitimately makes me feel like I’m using it wrong. I do Android and iOS native development and some IoT and it gives me really wrong code very often. To the point that I don’t see much difference between chatgpt and gpt4. But you say it’s close to just “write me a unix-like kernel”

starbugs 3y ago
It's not close and I also see no big difference between GPT 3.5 and 4. Don't get hyped. I am sure it will eventually happen, but calling it close is very optimistic.
londons_explore 3y ago
Not close to as in "it can nearly write it correctly", but close to as in "I believe within a small number of years, gpt-4-like tools would be able to write you a unix-like kernel from scratch and have it actually work, with no human input". I think we already have most of the pieces in place: * big language models that sometimes get the right answer. * language models with the ability to write instructions for other language models (ie. writing a project plan, and then completing each item of the plan, and then putting the results together). * language models with the ability to use tools (ie. 'run valgrind, tell the model what it says, and then the model will modify the code to fix the valgrind error') * language models with the ability to summarize large things to small. * language models with the ability to review existing work and see what needs changing to meet a goal, including chucking out work that isn't right/fit for purpose. With all these pieces, it really seems that with enough compute/budget, we are awfully close...
- gwoolhurme 3y ago
  That seems to also miss the intentionality that goes into some things in the kernel as well… I understand now you mean when a feedback of LLMs are improved on. I guess fair enough there, no idea if that will work till we see it. However I think the problem of a Unix-like kernel is a lot less trivial due to the human intentionality that goes into some choices as well as bit-banging optimization.
  - londons_explore 3y ago
    > human intentionality that goes into some choices Many choices are made at design time to make the right tradeoffs between complexity, speed, etc. But with AI-designed things, complexity is no longer an issue as long as the AI understands it, and you no longer need to think too much about speed - just implement 100 different designs and pick the one which does best on a set of benchmarks also designed by the AI.
    - kaba0 3y ago
      Current AIs can’t reason about trivial, 2 years old human cognitive things, let alone multi-million lines of code bases.
      - chaxor 3y ago
        This is a misguided conclusion, as they are entirely different systems. Don't compare apples and oranges like this or you'll be very disappointed.
- kaba0 3y ago
  Can any of these do any real reasoning? I feel this would just result in some Goldberg machine of producing terrible shit the big majority of the time, but may be impressive in some edge cases when the star constellations are right.
- Zanfa 3y ago
  It could also be a case of Tesla's full self-driving that's perpetually 2 years away. Progress in AI isn't linear so you can't extrapolate based on historic data. We really don't know if we're just a small step away from AGI or if we'll be stuck with the current crop of LLMs, with only incremental improvements over the next decade.
jakear 3y ago
Personally I couldn't even get it to draw me a regular pentagon in CSS. It was happy to draw hexagons and call them pentagons. It was even happy to go back and fix its mistakes when I informed it that it was making hexagons, not pentagons. It, of course, readily accepted that I was correct and it had indeed drawn a hexagon, but this time it'd be different. This time, it'd draw a pentagon. And... repeat.
- im3w1l 3y ago
  It really struggles with arithmetic, so that's kind of a worst case problem for it though.
  - jakear 3y ago
    Sure, but understanding you need to have 5 vertices to make a pentagon isn't exactly high-brow. Certainly not compared to making a Unix kernel from scratch.
    - im3w1l 3y ago
      I don't think high-brow / low-brow is a useful framework for understanding these models. They are good at certain things and bad at others and those don't correspond neatly to what a human finds easy and hard.
      - deleted 3y ago
        [deleted]
svaha1728 3y ago
I agree. You can also go to to github.com/torvalds/linux click on code -> Download zip and have the Linux Kernel. I’m not sure what having ChatGPT write it for you would give you. Maybe 10,000 hours of frustration after you realize how far from the mark you really are?

starbugs 3y ago

It's not close and I also see no big difference between GPT 3.5 and 4. Don't get hyped. I am sure it will eventually happen, but calling it close is very optimistic.

londons_explore 3y ago

Not close to as in "it can nearly write it correctly", but close to as in "I believe within a small number of years, gpt-4-like tools would be able to write you a unix-like kernel from scratch and have it actually work, with no human input". I think we already have most of the pieces in place: * big language models that sometimes get the right answer. * language models with the ability to write instructions for other language models (ie. writing a project plan, and then completing each item of the plan, and then putting the results together). * language models with the ability to use tools (ie. 'run valgrind, tell the model what it says, and then the model will modify the code to fix the valgrind error') * language models with the ability to summarize large things to small. * language models with the ability to review existing work and see what needs changing to meet a goal, including chucking out work that isn't right/fit for purpose. With all these pieces, it really seems that with enough compute/budget, we are awfully close...

gwoolhurme 3y ago
That seems to also miss the intentionality that goes into some things in the kernel as well… I understand now you mean when a feedback of LLMs are improved on. I guess fair enough there, no idea if that will work till we see it. However I think the problem of a Unix-like kernel is a lot less trivial due to the human intentionality that goes into some choices as well as bit-banging optimization.
- londons_explore 3y ago
  > human intentionality that goes into some choices Many choices are made at design time to make the right tradeoffs between complexity, speed, etc. But with AI-designed things, complexity is no longer an issue as long as the AI understands it, and you no longer need to think too much about speed - just implement 100 different designs and pick the one which does best on a set of benchmarks also designed by the AI.
  - kaba0 3y ago
    Current AIs can’t reason about trivial, 2 years old human cognitive things, let alone multi-million lines of code bases.
    - chaxor 3y ago
      This is a misguided conclusion, as they are entirely different systems. Don't compare apples and oranges like this or you'll be very disappointed.
kaba0 3y ago
Can any of these do any real reasoning? I feel this would just result in some Goldberg machine of producing terrible shit the big majority of the time, but may be impressive in some edge cases when the star constellations are right.
Zanfa 3y ago
It could also be a case of Tesla's full self-driving that's perpetually 2 years away. Progress in AI isn't linear so you can't extrapolate based on historic data. We really don't know if we're just a small step away from AGI or if we'll be stuck with the current crop of LLMs, with only incremental improvements over the next decade.

jakear 3y ago

Personally I couldn't even get it to draw me a regular pentagon in CSS. It was happy to draw hexagons and call them pentagons. It was even happy to go back and fix its mistakes when I informed it that it was making hexagons, not pentagons. It, of course, readily accepted that I was correct and it had indeed drawn a hexagon, but this time it'd be different. This time, it'd draw a pentagon. And... repeat.

im3w1l 3y ago
It really struggles with arithmetic, so that's kind of a worst case problem for it though.
- jakear 3y ago
  Sure, but understanding you need to have 5 vertices to make a pentagon isn't exactly high-brow. Certainly not compared to making a Unix kernel from scratch.
  - im3w1l 3y ago
    I don't think high-brow / low-brow is a useful framework for understanding these models. They are good at certain things and bad at others and those don't correspond neatly to what a human finds easy and hard.
    - deleted 3y ago
      [deleted]

svaha1728 3y ago

I agree. You can also go to to github.com/torvalds/linux click on code -> Download zip and have the Linux Kernel. I’m not sure what having ChatGPT write it for you would give you. Maybe 10,000 hours of frustration after you realize how far from the mark you really are?