4 ms·

> responded by writing its own computer vision pipeline Was it "its own" or something that was part of its training material? Don't get me wrong, I find this

> responded by writing its own computer vision pipeline Was it "its own" or something that was part of its training material? Don't get me wrong, I find this all amazing too and makes my work 10x easier and quicker. But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.)

nicky0 3mo ago
It would be "its own" in that it's customized for the specific use case. Your same argument could just as easily be applied to humans. If you write your own code, is it really your own, or is it just based on your own training and other code you've seen?
- teacpde 3mo ago
  I think their point is that there is no new invention here, being able to mimic what a human will do is absolutely impressive, but at the same time it is still mimicking humans. While I just type that out, I do realize my expectations for AI has been constantly lifted by how fast it iterates.
  - Aeolos 3mo ago
    It's mimicking humans only in as far as it is still using tools & programming languages developed by humans, for humans. This is temporary. There is a future where AI builds tools made by AI for AI, evolving at speeds that we may no longer be able to follow. Even with our current nascent technology, we can already observe this happening. See the recent laments on the AI Bun rewrite that "it's a million lines of code that nobody has ever reviewed." Or Cursor building a new source-control system for LLMs, because git is designed for human collaboration speed, not hundreds of changes per second. That's just 4 years in. How much of humanity's code will remain hand-written vs LLM-written 2, 5, 10 years from now?
  - TeMPOraL 3mo ago
    It's mimicking humans in how humans mimick each other in everything they do.
  - regularfry 3mo ago
    This isn't mimicking what a human would do, because it is an exceedingly rare human who would write their own renderer for a task needing a CAD file generating. The human who is capable of doing both is rare, let alone the human who can do it in any sort of reasonable time. Saying "but it's in the training data" is a cop-out: it's in Google too, would I do it? No. I would not.
TeMPOraL 3mo ago
> But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.) But how much to emphasize it, and why? Because frankly, you aren't inventing anything from scratch / first principles, either. None of us are, not frequently and not much anyway. We're primiarily just regurgitating what we've seen in the past and, mixing it with what we see in front of us - and that's true whether it's art or prose or code. Here, I wouldn't be able to do the same thing Claude did now, because I've never written a visual ML pipeline before myself. I know enough basics to get me started searching, and I'm confident I'd be able to cobble something together, but it would be me regurgitating and mixing whatever TensorFlow tutorials or OpenCV docs I found relevant, perhaps even tweaking an example from Github. Now, unless I'd literally just tweak a few lines of config in an existing Github example, no one would begrudge me saying "I wrote my own computer vision pipeline to solve this". So why begrudge LLMs?
- wneje 3mo ago
  [flagged]
- nz 3mo ago
  This is just the rationalist-empiricist[0] tension, from philosophy. Most empiricists (usually people who are _not_ from continental European cultures), take a position that knowledge and creativity come from observation and imitation, while rationalists believe that knowledge and creativity can be endogenous to the mind. One could say that, stereotypically and cartoonishly, empiricists believe that invention is a false concept, and a synonym for discovery, while rationalists would oppose this view. That catch is, if you look at the etymology of words "invention" and "discovery", you will find that they both share the _same_ root: _Ars Inveniendi_. A natural question arises: does this mean that people from a millennium ago did not have a dyad equivalent to our invention-discovery dyad? And the answer, surprisingly, is _no_. Even a millennium ago, the empiricists and rationalists were going at it. If _Ars Inveniendi_ is art of discovery/invention, then its counterpart is _Ars Demonstrandi_ (the art of demonstration/proof). So, how do these differ? Is one just observing/creating and the other just math and language-games? In general, Ars Demonstrandi is about writing down axioms, and then expanding those axioms recursively (similar to rewrite rules in any formal system), until you get to some end-state, or, if there is none, a novel or surprising state. I, personally, call this source-to-sink thinking. Ars Inveniendi is about taking conclusions (often using observations from the physical world) and trying to figure out what axioms can lead to those conclusions. I, (again) personally, call this sink-to-source thinking. Put differently, Ars Inveniendi can help one discover starting points for Ars Demonstrandi. If you read the dialectics (e.g. Plato and friends), you'll find that most of them are just a mutual recursion between Ars Inveniendi and Ars Demonstrandi. I believe (again, I am not a philosopher, and this is just my intuition), that the thing that we call "creativity" and "invention" _emerges_ from the recursive loop[1]. A favorite example: Esperanto (the conlang). It is a language, which is remarkably elegant and consistent and (in my opinion) beautiful, because it _is_ derived from first principles, which were themselves derived from the various languages spoken in Europe (not all of which are Indo-European -- the agglutinative features have more in common with Finno-Ugric and Turkic languages). There is something about it, that makes it _qualitatively_ different (and thus holistically novel) from all other languages (and I speak, fluently, _two_ national languages, that are very different from each other, so I can attest to the difference personally). My guess, is that people will not accept that AI/LLMs are creative or inventive, until they can produce an original[2] (non-plagiarized) artifact that feels the way Esperanto feels. [0]: By Rationalist, I do not mean the "Bay Area Rationalists", who are, in fact, empiricists. [1]: I am unsure if the loop requires only one human, at least two humans, or if it can be fully automated. [2]: Note that Centos are poems made completely out of line-numbers (e.g. fragments from the Iliad, or the bible, etc). Every line is borrowed, yet some of them are considered beautiful and original works of art. Similarly, Labatut and Burroughs and Perec, write using a technique called _the cut-up method_, where they take books, magazines, and newspapers, and superimpose page-fragments, and use that as an inspiration -- they are all considered artists, and good ones. It is unclear to me why LLMs (which seem to be built on the cut-up method, and have cut-ups of all of human knowledge) cannot match these artists. What's missing?
alex43578 3mo ago
Is anything you do on your own? You wrote this response in English, but didn't invent your own language. How dilute of a contribution can someone/something have made and still merit credit?
- oblio 3mo ago
  LOL. Tolkien invented multiple languages. There are/were probably 100 000 languages on this planet. That's without including dialects. We use existing languages not because we love to copy, it's because languages are a medium for communication. They're only useful if they're understood by others. If you wanted to prove a point: writing is much, much harder to invent, yet it was invented independently at least 3 times in history. By definition LLMs can't invent something truly novel like writing. Because nobody trained the first writers. There was no "training set". It's fine, LLMs can still be useful. Science is looking at the gaps and most of the gaps that are useful are small and LLMs can explore those faster than us. That could potentially improve many lives. Though not in this corporate LLM world, most likely.
  - jefftk 3mo ago
    Are you saying your bar for being impressed by an LLM is something equivalent to the initial invention of writing?
    - oblio 3mo ago
      LLMs can be quite impressive with much more mundane accomplishments. The bar for what OpenAI, Anthropic, Google & co are trying to sell me, all of us, yes, for sure. Inventing writing is even too low a bar.
    - teiferer 3mo ago
      > your bar for being impressed by an LLM Please don't move the goal posts. If you read my post again you will notice that I explicitly state that I'm impressed. This is not about whether somebody is impressed but about whether "it created its own Xyz" is a suitable description.
chpatrick 3mo ago
It's definitely not seen the counterexample to the Jacobian conjecture before.
the8472 3mo ago
Please fill out this form to verify you're human: https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg
lanstin 3mo ago
Glue layers are fun and easy to write - and often worth doing.
dorgo 3mo ago
>But it's not like it's inventing this stuff from scratch / first principles. Yes, it's definitelly time for an Alpha Zero moment. An AI which invents everything from first principles. That would be impressive, ... and a bit scary.

nicky0 3mo ago

It would be "its own" in that it's customized for the specific use case. Your same argument could just as easily be applied to humans. If you write your own code, is it really your own, or is it just based on your own training and other code you've seen?

teacpde 3mo ago
I think their point is that there is no new invention here, being able to mimic what a human will do is absolutely impressive, but at the same time it is still mimicking humans. While I just type that out, I do realize my expectations for AI has been constantly lifted by how fast it iterates.
- Aeolos 3mo ago
  It's mimicking humans only in as far as it is still using tools & programming languages developed by humans, for humans. This is temporary. There is a future where AI builds tools made by AI for AI, evolving at speeds that we may no longer be able to follow. Even with our current nascent technology, we can already observe this happening. See the recent laments on the AI Bun rewrite that "it's a million lines of code that nobody has ever reviewed." Or Cursor building a new source-control system for LLMs, because git is designed for human collaboration speed, not hundreds of changes per second. That's just 4 years in. How much of humanity's code will remain hand-written vs LLM-written 2, 5, 10 years from now?
- TeMPOraL 3mo ago
  It's mimicking humans in how humans mimick each other in everything they do.
- regularfry 3mo ago
  This isn't mimicking what a human would do, because it is an exceedingly rare human who would write their own renderer for a task needing a CAD file generating. The human who is capable of doing both is rare, let alone the human who can do it in any sort of reasonable time. Saying "but it's in the training data" is a cop-out: it's in Google too, would I do it? No. I would not.

TeMPOraL 3mo ago

> But it's not like it's inventing this stuff from scratch / first principles. It has seen this kind of tech before by consuming all publicly available source code and books etc. (And that's ok, but let's be honest/clear about that.) But how much to emphasize it, and why? Because frankly, you aren't inventing anything from scratch / first principles, either. None of us are, not frequently and not much anyway. We're primiarily just regurgitating what we've seen in the past and, mixing it with what we see in front of us - and that's true whether it's art or prose or code. Here, I wouldn't be able to do the same thing Claude did now, because I've never written a visual ML pipeline before myself. I know enough basics to get me started searching, and I'm confident I'd be able to cobble something together, but it would be me regurgitating and mixing whatever TensorFlow tutorials or OpenCV docs I found relevant, perhaps even tweaking an example from Github. Now, unless I'd literally just tweak a few lines of config in an existing Github example, no one would begrudge me saying "I wrote my own computer vision pipeline to solve this". So why begrudge LLMs?

wneje 3mo ago
[flagged]
nz 3mo ago
This is just the rationalist-empiricist[0] tension, from philosophy. Most empiricists (usually people who are _not_ from continental European cultures), take a position that knowledge and creativity come from observation and imitation, while rationalists believe that knowledge and creativity can be endogenous to the mind. One could say that, stereotypically and cartoonishly, empiricists believe that invention is a false concept, and a synonym for discovery, while rationalists would oppose this view. That catch is, if you look at the etymology of words "invention" and "discovery", you will find that they both share the _same_ root: _Ars Inveniendi_. A natural question arises: does this mean that people from a millennium ago did not have a dyad equivalent to our invention-discovery dyad? And the answer, surprisingly, is _no_. Even a millennium ago, the empiricists and rationalists were going at it. If _Ars Inveniendi_ is art of discovery/invention, then its counterpart is _Ars Demonstrandi_ (the art of demonstration/proof). So, how do these differ? Is one just observing/creating and the other just math and language-games? In general, Ars Demonstrandi is about writing down axioms, and then expanding those axioms recursively (similar to rewrite rules in any formal system), until you get to some end-state, or, if there is none, a novel or surprising state. I, personally, call this source-to-sink thinking. Ars Inveniendi is about taking conclusions (often using observations from the physical world) and trying to figure out what axioms can lead to those conclusions. I, (again) personally, call this sink-to-source thinking. Put differently, Ars Inveniendi can help one discover starting points for Ars Demonstrandi. If you read the dialectics (e.g. Plato and friends), you'll find that most of them are just a mutual recursion between Ars Inveniendi and Ars Demonstrandi. I believe (again, I am not a philosopher, and this is just my intuition), that the thing that we call "creativity" and "invention" _emerges_ from the recursive loop[1]. A favorite example: Esperanto (the conlang). It is a language, which is remarkably elegant and consistent and (in my opinion) beautiful, because it _is_ derived from first principles, which were themselves derived from the various languages spoken in Europe (not all of which are Indo-European -- the agglutinative features have more in common with Finno-Ugric and Turkic languages). There is something about it, that makes it _qualitatively_ different (and thus holistically novel) from all other languages (and I speak, fluently, _two_ national languages, that are very different from each other, so I can attest to the difference personally). My guess, is that people will not accept that AI/LLMs are creative or inventive, until they can produce an original[2] (non-plagiarized) artifact that feels the way Esperanto feels. [0]: By Rationalist, I do not mean the "Bay Area Rationalists", who are, in fact, empiricists. [1]: I am unsure if the loop requires only one human, at least two humans, or if it can be fully automated. [2]: Note that Centos are poems made completely out of line-numbers (e.g. fragments from the Iliad, or the bible, etc). Every line is borrowed, yet some of them are considered beautiful and original works of art. Similarly, Labatut and Burroughs and Perec, write using a technique called _the cut-up method_, where they take books, magazines, and newspapers, and superimpose page-fragments, and use that as an inspiration -- they are all considered artists, and good ones. It is unclear to me why LLMs (which seem to be built on the cut-up method, and have cut-ups of all of human knowledge) cannot match these artists. What's missing?

alex43578 3mo ago

Is anything you do on your own? You wrote this response in English, but didn't invent your own language. How dilute of a contribution can someone/something have made and still merit credit?

oblio 3mo ago
LOL. Tolkien invented multiple languages. There are/were probably 100 000 languages on this planet. That's without including dialects. We use existing languages not because we love to copy, it's because languages are a medium for communication. They're only useful if they're understood by others. If you wanted to prove a point: writing is much, much harder to invent, yet it was invented independently at least 3 times in history. By definition LLMs can't invent something truly novel like writing. Because nobody trained the first writers. There was no "training set". It's fine, LLMs can still be useful. Science is looking at the gaps and most of the gaps that are useful are small and LLMs can explore those faster than us. That could potentially improve many lives. Though not in this corporate LLM world, most likely.
- jefftk 3mo ago
  Are you saying your bar for being impressed by an LLM is something equivalent to the initial invention of writing?
  - oblio 3mo ago
    LLMs can be quite impressive with much more mundane accomplishments. The bar for what OpenAI, Anthropic, Google & co are trying to sell me, all of us, yes, for sure. Inventing writing is even too low a bar.
  - teiferer 3mo ago
    > your bar for being impressed by an LLM Please don't move the goal posts. If you read my post again you will notice that I explicitly state that I'm impressed. This is not about whether somebody is impressed but about whether "it created its own Xyz" is a suitable description.

chpatrick 3mo ago

It's definitely not seen the counterexample to the Jacobian conjecture before.

the8472 3mo ago

Please fill out this form to verify you're human: https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg https://litter.catbox.moe/3ugm2b0m1divdgxs.jpg

lanstin 3mo ago

Glue layers are fun and easy to write - and often worth doing.

dorgo 3mo ago

>But it's not like it's inventing this stuff from scratch / first principles. Yes, it's definitelly time for an Alpha Zero moment. An AI which invents everything from first principles. That would be impressive, ... and a bit scary.