5 ms·

Finally, my interest in LLMs is piqued! Seems like everyone has been getting excited around the search or code-generation use cases ... or simply trying to mak

Finally, my interest in LLMs is piqued! Seems like everyone has been getting excited around the search or code-generation use cases ... or simply trying to make it say naughty things (boring, not interested, wake up in a few more years), but this is eye opening. The idea of this as a "universal coupler" is fascinating, and I think I agree with the author that we are probably standing at an early-90s-web moment with LLMs as a function call (the technology is kinda-there and mostly-works, and people are trying out a lot of ideas ... some work, some don't). My mind is racing. Thanks for the epiphany moment.

ketzo 4y ago
It’s awesome for formatting and structuring. Copy and paste a bunch of styled JS components -> get back out a single CSS sheet Paste in a markdown document -> get out the same thing in HTML Fun stuff like that.
- chaxor 4y ago
  Ah yes, you now too can have the wonders of pandoc - now on sale! Retail pandoc price of 20MB, now selling for only 3TB with the exclusive offer of ChatGPT!
  - seagullriffic 4y ago
    I don't want to be mean, but this seems like the famous Dropbox/rsync comment. The value here is how easy it is, and the fact that a generalised model can take the place of a (well-engineered) specialised tool.
v7n 4y ago
Bit of a tangent but I also predict an explosive demand for parametric AI-first design and simulation software. OpenSCAD works right now in the sense that the produced source is valid, but asking for even something well documented e.g. an AR-15 lower receiver does not impress.
- TaylorAlexander 4y ago
  I did not realize it could do that. I asked it to generate a cube with a hole through the center and it really struggled to succeed. Interesting concept though!
  - martyvis 4y ago
    ChatGPT has a pretty good handle on G-code (the machine code to drive CNC and 3D printers). A machinist friend of mine and myself got it to write G-code to create a shape from a block of material, adding holes and arrises and so on. To top it off it wrote some decent poetry about 2 mates discussing G-code over a few beers.
- fnordpiglet 4y ago
  Fusion 360 has a generative AI design extension already. Never used it, I’m too much a poor to use something like that.
  - ewalk153 4y ago
    There is a free hobby plan for Fusion 360 that I use for my personal projects.
    - fnordpiglet 4y ago
      I thought it doesn’t have the generative AI extension? When I looked at the plan comparison it was a part of the commercial.
- samplatt 4y ago
  My previous boss (we were in the civil construction industry) has been saying for years that we are on the cusp of an industry-shattering breakthrough with generative engineering. We should be able to, by now, input terrain maps, account for local water ways & rainfall, and Autodesk's Brobdingnagian suite of tools should be able to spit out a whole new suburban package, engineered to local country/city engineering standards. The main thing that's holding is back is the complexity of engineering standards, and its implementation in the software. But all that'll take is time & money.
nl 4y ago
We are months away from being able to do this with images too. All the pieces are there, and multi-modal (smallish) large language + image models are already being used in research labs; eg MS Kosmos-1[1]. Check out the visual IQ test results in the paper. Kosmos-1 is only 1.6B parameters. When that or similar models scale out to 50B_ params they will be pretty amazing. [1] https://arxiv.org/abs/2302.14045 https://arxiv.org/abs/2302.14045
- chrisweekly 4y ago
  Linked article / brief mentions interleaving of text and images as essential for certain kinds of learning; that particular intersection reminds me of the profoundly compelling multimodal book by Nick Sousanis, "Unflattening". Highest possible recommendation. https://en.m.wikipedia.org/wiki/Unflattening https://en.m.wikipedia.org/wiki/Unflattening
  - EZ-Cheeze 4y ago
    I had that book but lost it in LA. This is the page I remembered the most and have shown to the most people: https://twitter.com/nsousanis/status/245176914900299776?lang=en https://twitter.com/nsousanis/status/245176914900299776?lang...
- EZ-Cheeze 4y ago
  That is the truly mind-blowing thing that's coming. I am currently creating reference images for a game which I expect within 6-12 months I'll be able to feed into a multimodal ChatGPT to create 3D assets out of the 2D pics. We'll be able to conjure up worlds at a whim - so start imagining them already!
  - deleted 4y ago
    [deleted]
  - basch 4y ago
    Mark gets a lot of flack for metaverse, but can imagine world design where you start in a blank room and describe what you want around you, and it appears. Like the Matrix loading white room. And with voice recognition and eye tracking (and brain scans), how close are we to “you have to use your hands? It’s like a baby’s toy.”
basch 4y ago
I remember having that epiphany in 2017, reading the original google transformer press release. I remember thinking “they taught computers to understand!, and someday soon i’ll be able to translate anything into anything.” And not like translating “languages” but translating contexts and concepts. LLMs really shine when you try and combine two previously untouched things or map one bit of knowledge it a common pattern. Eg “summarize Mary Shelly’s Frankenstein as if it was an episode synopsis of Magic School Bus.” “Now do the same for Dracula.” I remember my mind racing so much that day. With all the ideas of things it would unlock. And I still managed to under imagine just how many different types of knowledge, and different layers of knowledge and language, they would be able to encode into one compact dataset.
- novaRom 4y ago
  LLMs will be used to discover "not so obvious" connections in science. I think we will see first scientific discoveries done by LLMs very soon (in fact, such works are probably already happening). Expect major breakthroughs in physics, math, cosmology, biology, etc. published in Nature and authored by LLMs.
  - OOPMan 4y ago
    Cranking that hype level to the next stage eh?
JRagone 4y ago
This is where I see the Semantic Web, semantic interoperability, and ontology alignment spaces coming to life. Who needs to standardize data when you can stick an AI in the middle of two services to do the translation for you? And then suddenly the interoperability of the metaverse doesn't seem so insurmountable. I made a post last summer that speculated on this, but I didn't think this future was so near. Or even possible really. https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Semantic_interoperability https://en.wikipedia.org/wiki/Semantic_interoperability https://en.wikipedia.org/wiki/Ontology_alignment https://en.wikipedia.org/wiki/Ontology_alignment https://metaverse-research-590.web.app/posts/communication/ https://metaverse-research-590.web.app/posts/communication/

ketzo 4y ago

It’s awesome for formatting and structuring. Copy and paste a bunch of styled JS components -> get back out a single CSS sheet Paste in a markdown document -> get out the same thing in HTML Fun stuff like that.

chaxor 4y ago
Ah yes, you now too can have the wonders of pandoc - now on sale! Retail pandoc price of 20MB, now selling for only 3TB with the exclusive offer of ChatGPT!
- seagullriffic 4y ago
  I don't want to be mean, but this seems like the famous Dropbox/rsync comment. The value here is how easy it is, and the fact that a generalised model can take the place of a (well-engineered) specialised tool.

v7n 4y ago

Bit of a tangent but I also predict an explosive demand for parametric AI-first design and simulation software. OpenSCAD works right now in the sense that the produced source is valid, but asking for even something well documented e.g. an AR-15 lower receiver does not impress.

TaylorAlexander 4y ago
I did not realize it could do that. I asked it to generate a cube with a hole through the center and it really struggled to succeed. Interesting concept though!
- martyvis 4y ago
  ChatGPT has a pretty good handle on G-code (the machine code to drive CNC and 3D printers). A machinist friend of mine and myself got it to write G-code to create a shape from a block of material, adding holes and arrises and so on. To top it off it wrote some decent poetry about 2 mates discussing G-code over a few beers.
fnordpiglet 4y ago
Fusion 360 has a generative AI design extension already. Never used it, I’m too much a poor to use something like that.
- ewalk153 4y ago
  There is a free hobby plan for Fusion 360 that I use for my personal projects.
  - fnordpiglet 4y ago
    I thought it doesn’t have the generative AI extension? When I looked at the plan comparison it was a part of the commercial.
samplatt 4y ago
My previous boss (we were in the civil construction industry) has been saying for years that we are on the cusp of an industry-shattering breakthrough with generative engineering. We should be able to, by now, input terrain maps, account for local water ways & rainfall, and Autodesk's Brobdingnagian suite of tools should be able to spit out a whole new suburban package, engineered to local country/city engineering standards. The main thing that's holding is back is the complexity of engineering standards, and its implementation in the software. But all that'll take is time & money.

nl 4y ago

We are months away from being able to do this with images too. All the pieces are there, and multi-modal (smallish) large language + image models are already being used in research labs; eg MS Kosmos-1[1]. Check out the visual IQ test results in the paper. Kosmos-1 is only 1.6B parameters. When that or similar models scale out to 50B_ params they will be pretty amazing. [1] https://arxiv.org/abs/2302.14045 https://arxiv.org/abs/2302.14045

chrisweekly 4y ago
Linked article / brief mentions interleaving of text and images as essential for certain kinds of learning; that particular intersection reminds me of the profoundly compelling multimodal book by Nick Sousanis, "Unflattening". Highest possible recommendation. https://en.m.wikipedia.org/wiki/Unflattening https://en.m.wikipedia.org/wiki/Unflattening
- EZ-Cheeze 4y ago
  I had that book but lost it in LA. This is the page I remembered the most and have shown to the most people: https://twitter.com/nsousanis/status/245176914900299776?lang=en https://twitter.com/nsousanis/status/245176914900299776?lang...
EZ-Cheeze 4y ago
That is the truly mind-blowing thing that's coming. I am currently creating reference images for a game which I expect within 6-12 months I'll be able to feed into a multimodal ChatGPT to create 3D assets out of the 2D pics. We'll be able to conjure up worlds at a whim - so start imagining them already!
- deleted 4y ago
  [deleted]
- basch 4y ago
  Mark gets a lot of flack for metaverse, but can imagine world design where you start in a blank room and describe what you want around you, and it appears. Like the Matrix loading white room. And with voice recognition and eye tracking (and brain scans), how close are we to “you have to use your hands? It’s like a baby’s toy.”

basch 4y ago

I remember having that epiphany in 2017, reading the original google transformer press release. I remember thinking “they taught computers to understand!, and someday soon i’ll be able to translate anything into anything.” And not like translating “languages” but translating contexts and concepts. LLMs really shine when you try and combine two previously untouched things or map one bit of knowledge it a common pattern. Eg “summarize Mary Shelly’s Frankenstein as if it was an episode synopsis of Magic School Bus.” “Now do the same for Dracula.” I remember my mind racing so much that day. With all the ideas of things it would unlock. And I still managed to under imagine just how many different types of knowledge, and different layers of knowledge and language, they would be able to encode into one compact dataset.

novaRom 4y ago
LLMs will be used to discover "not so obvious" connections in science. I think we will see first scientific discoveries done by LLMs very soon (in fact, such works are probably already happening). Expect major breakthroughs in physics, math, cosmology, biology, etc. published in Nature and authored by LLMs.
- OOPMan 4y ago
  Cranking that hype level to the next stage eh?

JRagone 4y ago

This is where I see the Semantic Web, semantic interoperability, and ontology alignment spaces coming to life. Who needs to standardize data when you can stick an AI in the middle of two services to do the translation for you? And then suddenly the interoperability of the metaverse doesn't seem so insurmountable. I made a post last summer that speculated on this, but I didn't think this future was so near. Or even possible really. https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Semantic_Web https://en.wikipedia.org/wiki/Semantic_interoperability https://en.wikipedia.org/wiki/Semantic_interoperability https://en.wikipedia.org/wiki/Ontology_alignment https://en.wikipedia.org/wiki/Ontology_alignment https://metaverse-research-590.web.app/posts/communication/ https://metaverse-research-590.web.app/posts/communication/