6 ms·

I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems lik

I think a lot of AI stuff is really cool and promising, but am dismayed by how impressed people are by it sometimes, especially the visual output of systems like DALL-E. Seems like machines are testing our intelligence, rather than the other way around.

celestialcheese 3y ago
Simple things that don't seem impressive to you, but are impressive to others, may be because you haven't experienced the first hand difficulty of doing that thing pre-transformers/GPT. For example, I get _unbelievably_ excited with knowledge extraction and question answering demos on PDFs. Why? Because i've built similar systems for over a decade and know how difficult it is to build on top of messy archival data. Now, with very little code, i'm getting SOTA results. If you didn't have experience with this, you'd probably thing "Huh, that's not impressive, XYZ does this already". But it's the moving of the baseline that's what's really impressive. === AI hype-beasts aside of course - the breathless pontificating of "influencers" and former crypto bros is cringe.
- CharlesW 3y ago
  Hey, would you mind sharing the tools and other resources you’ve found helpful? I’m really interested in trying this but am not sure where to start.
  - celestialcheese 3y ago
    This tutorial is a good overview of the rough systems behind most of this "Chat your data" application you're seeing now. https://www.pinecone.io/learn/langchain-retrieval-augmentation/ https://www.pinecone.io/learn/langchain-retrieval-augmentati... Langchain / Llama indexes are both toolboxes that abstract away a lot of the plumbing for doing this kind of thing, and Pinecone is one of dozens of vector databases. Personally, i'd try out langchain and chromadb and go through some of the examples langchain has in their docs, then be prepared to completely abandon langchain and just work with the LLM APIs directly. Start with openai, get on the waitlist for GPT-4 tokens, and also get on Anthropics Claude waitlist for the 100k-1.3. It's _very_ good for knowledge retrieval. Langchain tries to do too much in extracting away the prompts, and the prompts are really what matter in getting interesting stuff out of your own data. Use langchain, llamaindex pieces but build from scratch for most things as your tinkering. It's really not hard if you have a background in programming, and it's _so_ much fun. You'll feel like you have superpowers once you get a scraping interface hooked into an LLM. All of a sudden you can automate some really complex pipelines very quickly
  - ukuina 3y ago
    Here's a packaged solution! https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT
- surgical_fire 3y ago
  I am kind of in the same boat as you. I sit in a weird spot of thinking the current trend of AI is really impressive, but also thinking people are massively over-hyping it (with a smaller counter wave of some people really undermining it). Having tangled with natural language processing and transformation in the past (always with dismal results), I can say it's one of the most annoying problems to tackle in computation, because natural languages have the horrible tentency of being very irregular (i.e.: they are a fucking mess). ChatGPT capabilities to parse and generate fluent language never ceases to amaze me. At the same time I don't think it's going to take over the world. It's more like a game-changer productivity tool (with all the upheaval that comes along when those appear) than the birth of Skynet.
  - Freebytes 3y ago
    I had thought about a translation system that sought to translate into a different, more coherent language, pass that into an AI model, get a response in that language, and then translate the results back into English. Early OpenAI responses did not seem very impressive to me; however, that was before ChatGPT came along and blew me away. And GPT-4 gives even better responses! I am not sure if the intermediate language solution is necessary or viable at this point because you would need a powerful LLM to perform the translation. But, there might be value in symbolic logic being part of the machine learning vernacular as part of the early tokenization and training process.
- gatkinso 3y ago
  fair, mostly I'm unimpressed with visual outputs - AI making 'art' etc. PDF extraction is extremely cool and useful and the results are fantastic. Fully agree re influencers. Seems endemic these days.
moonchrome 3y ago
I think it shows how much low impact content we consume constantly - where the quality outside of superficial appearance doesn't really matter. Images with obvious flaws, text with factual/logical mistakes - as long as it looks right on the first glance - it's passable in a lot of places. Just made me more aware of how bulshit is the norm and not the exception.
deleted 3y ago
[deleted]
burnished 3y ago
People are impressed because its black fucking magic my friend.
- Jtsummers 3y ago
  Pretty sure it's not magic, black fucking or otherwise, just really complex math, large data sets, and very fast computers.
  - esafak 3y ago
    Magic is that which can not be explained. Could you explain GPT-4's results if you saw them a few years ago?
    - Jtsummers 3y ago
      Probably, I mean I first studied ANNs over two decades ago and had conversations with people about them prior to starting college in the 90s who had developed solutions with them in the 80s (obviously severely computationally constrained in those days compared to today). So still not magic. To be very blunt: If you believe that GPT and the like are magic, then you're not thinking clearly. You're blinded by the results (which are impressive).
      - esafak 3y ago
        ANNs in the 90s were nothing like this. They were not even like this a few years ago. And neither were HMMs. There is an emergent human quality to them because they have approached our abilities; the comparison is tenable, whereas before it was not.
        Jtsummers 3y ago
        Scale and topology. That's the difference between ANNs of the 90s and 00s and today. They're still based on the same fundamental principles and doing (modulo scale) the same fundamental things: classification, prediction, generation. It is not magic, it has never been magic.
bmc7505 3y ago
I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI safety. Personally I find it difficult to believe they are willfully participating in such conartistry, and makes me question their views on other subjects. Although I suppose intelligence and gullibility can coexist and some forms of delusion are better pitied than scorned.
- sebzim4500 3y ago
  Why is your prior belief that AI safety is a cult so strong that even multiple people you would rather assume that multiple people who you previously respected are now delusional than to consider that there might be arguments worth considering?
  - bmc7505 3y ago
    It’s a LARP at best and a scam at worst. I shared some of my thoughts on the matter in a prior thread: https://news.ycombinator.com/item?id=35145189#35147288 https://news.ycombinator.com/item?id=35145189#35147288
    - sebzim4500 3y ago
      I don't understand the relevance of the prior thread, except that you made the same claim with similarly scant evidence. What exactly is it that makes you so confident?
      - bmc7505 3y ago
        The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. Furthermore, I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks, even if they could harness galactic amounts of energy. Physical Turing machines have bottlenecks that upper-bound the power of oracles.
mrbungie 3y ago
Have you seen how the outputs of image generation tools evolved during mere 2-3 years? It went from the stuff of nightmares to actually pasable and consumable images pretty fast.
- gatkinso 3y ago
  It's certainly gotten better since deep dream et al
fzeroracer 3y ago
I've seen a lot of people wow'd and impressed by the AI, acting as if it's sentient etc whom would turn around in a second and argue that animals are flesh robots powered purely by survival instincts. A lot of this is because I think it sort of wedges a knife into an area of our brain which makes us think it's similar to us. Look at this output, it's able to write about Shakespeare or summarize Beowulf or talk about these topics in a way I can understand. But ultimately it's an affirmative mirror; it will respond exactly as you prompt it. It cannot disagree with you or ask 'why' or synthesize the world beyond what it's told to do. And it's incredibly hard to get some people to understand this. Even harder when you have companies pushing this because it's trendy even as we see the issues of private data being leaked or the frays in the data sets appearing.
yreg 3y ago
I'm very impressed about both the rapid pace of progress in the area and also the current capabilities of the models. I love them, it brings me joy. I like how suddenly the image generation state of the art made unexpected and significant progress. One can witness something like that only time to time. What I don't get at all is that there are people like you, who are frowning upon and somehow disgusted by people like me.

celestialcheese 3y ago

Simple things that don't seem impressive to you, but are impressive to others, may be because you haven't experienced the first hand difficulty of doing that thing pre-transformers/GPT. For example, I get _unbelievably_ excited with knowledge extraction and question answering demos on PDFs. Why? Because i've built similar systems for over a decade and know how difficult it is to build on top of messy archival data. Now, with very little code, i'm getting SOTA results. If you didn't have experience with this, you'd probably thing "Huh, that's not impressive, XYZ does this already". But it's the moving of the baseline that's what's really impressive. === AI hype-beasts aside of course - the breathless pontificating of "influencers" and former crypto bros is cringe.

CharlesW 3y ago
Hey, would you mind sharing the tools and other resources you’ve found helpful? I’m really interested in trying this but am not sure where to start.
- celestialcheese 3y ago
  This tutorial is a good overview of the rough systems behind most of this "Chat your data" application you're seeing now. https://www.pinecone.io/learn/langchain-retrieval-augmentation/ https://www.pinecone.io/learn/langchain-retrieval-augmentati... Langchain / Llama indexes are both toolboxes that abstract away a lot of the plumbing for doing this kind of thing, and Pinecone is one of dozens of vector databases. Personally, i'd try out langchain and chromadb and go through some of the examples langchain has in their docs, then be prepared to completely abandon langchain and just work with the LLM APIs directly. Start with openai, get on the waitlist for GPT-4 tokens, and also get on Anthropics Claude waitlist for the 100k-1.3. It's _very_ good for knowledge retrieval. Langchain tries to do too much in extracting away the prompts, and the prompts are really what matter in getting interesting stuff out of your own data. Use langchain, llamaindex pieces but build from scratch for most things as your tinkering. It's really not hard if you have a background in programming, and it's _so_ much fun. You'll feel like you have superpowers once you get a scraping interface hooked into an LLM. All of a sudden you can automate some really complex pipelines very quickly
- ukuina 3y ago
  Here's a packaged solution! https://github.com/imartinez/privateGPT https://github.com/imartinez/privateGPT
surgical_fire 3y ago
I am kind of in the same boat as you. I sit in a weird spot of thinking the current trend of AI is really impressive, but also thinking people are massively over-hyping it (with a smaller counter wave of some people really undermining it). Having tangled with natural language processing and transformation in the past (always with dismal results), I can say it's one of the most annoying problems to tackle in computation, because natural languages have the horrible tentency of being very irregular (i.e.: they are a fucking mess). ChatGPT capabilities to parse and generate fluent language never ceases to amaze me. At the same time I don't think it's going to take over the world. It's more like a game-changer productivity tool (with all the upheaval that comes along when those appear) than the birth of Skynet.
- Freebytes 3y ago
  I had thought about a translation system that sought to translate into a different, more coherent language, pass that into an AI model, get a response in that language, and then translate the results back into English. Early OpenAI responses did not seem very impressive to me; however, that was before ChatGPT came along and blew me away. And GPT-4 gives even better responses! I am not sure if the intermediate language solution is necessary or viable at this point because you would need a powerful LLM to perform the translation. But, there might be value in symbolic logic being part of the machine learning vernacular as part of the early tokenization and training process.
gatkinso 3y ago
fair, mostly I'm unimpressed with visual outputs - AI making 'art' etc. PDF extraction is extremely cool and useful and the results are fantastic. Fully agree re influencers. Seems endemic these days.

moonchrome 3y ago

I think it shows how much low impact content we consume constantly - where the quality outside of superficial appearance doesn't really matter. Images with obvious flaws, text with factual/logical mistakes - as long as it looks right on the first glance - it's passable in a lot of places. Just made me more aware of how bulshit is the norm and not the exception.

deleted 3y ago

[deleted]

burnished 3y ago

People are impressed because its black fucking magic my friend.

Jtsummers 3y ago
Pretty sure it's not magic, black fucking or otherwise, just really complex math, large data sets, and very fast computers.
- esafak 3y ago
  Magic is that which can not be explained. Could you explain GPT-4's results if you saw them a few years ago?
  - Jtsummers 3y ago
    Probably, I mean I first studied ANNs over two decades ago and had conversations with people about them prior to starting college in the 90s who had developed solutions with them in the 80s (obviously severely computationally constrained in those days compared to today). So still not magic. To be very blunt: If you believe that GPT and the like are magic, then you're not thinking clearly. You're blinded by the results (which are impressive).
    - esafak 3y ago
      ANNs in the 90s were nothing like this. They were not even like this a few years ago. And neither were HMMs. There is an emergent human quality to them because they have approached our abilities; the comparison is tenable, whereas before it was not.
      - Jtsummers 3y ago
        Scale and topology. That's the difference between ANNs of the 90s and 00s and today. They're still based on the same fundamental principles and doing (modulo scale) the same fundamental things: classification, prediction, generation. It is not magic, it has never been magic.

bmc7505 3y ago

I am disappointed with increasing frequency to learn how certain individuals whom I previously believed to be serious scholars fall for the cargo cult of AI safety. Personally I find it difficult to believe they are willfully participating in such conartistry, and makes me question their views on other subjects. Although I suppose intelligence and gullibility can coexist and some forms of delusion are better pitied than scorned.

sebzim4500 3y ago
Why is your prior belief that AI safety is a cult so strong that even multiple people you would rather assume that multiple people who you previously respected are now delusional than to consider that there might be arguments worth considering?
- bmc7505 3y ago
  It’s a LARP at best and a scam at worst. I shared some of my thoughts on the matter in a prior thread: https://news.ycombinator.com/item?id=35145189#35147288 https://news.ycombinator.com/item?id=35145189#35147288
  - sebzim4500 3y ago
    I don't understand the relevance of the prior thread, except that you made the same claim with similarly scant evidence. What exactly is it that makes you so confident?
    - bmc7505 3y ago
      The problems that AI will manifest are the result of human ambition and failings, no different as any other technology that empowers individuals. Yes, individuals and organizations will misuse AI for immoral purposes, but the popular belief that AI is inherently antihumanist launders accountability by pretending to remove human agency from the equation. How we use or misuse AI technology is entirely on us. Furthermore, I would argue there are strong complexity-theoretic bottlenecks to computational processes which limit the expressive power of neural networks, even if they could harness galactic amounts of energy. Physical Turing machines have bottlenecks that upper-bound the power of oracles.

mrbungie 3y ago

Have you seen how the outputs of image generation tools evolved during mere 2-3 years? It went from the stuff of nightmares to actually pasable and consumable images pretty fast.

gatkinso 3y ago
It's certainly gotten better since deep dream et al

fzeroracer 3y ago

I've seen a lot of people wow'd and impressed by the AI, acting as if it's sentient etc whom would turn around in a second and argue that animals are flesh robots powered purely by survival instincts. A lot of this is because I think it sort of wedges a knife into an area of our brain which makes us think it's similar to us. Look at this output, it's able to write about Shakespeare or summarize Beowulf or talk about these topics in a way I can understand. But ultimately it's an affirmative mirror; it will respond exactly as you prompt it. It cannot disagree with you or ask 'why' or synthesize the world beyond what it's told to do. And it's incredibly hard to get some people to understand this. Even harder when you have companies pushing this because it's trendy even as we see the issues of private data being leaked or the frays in the data sets appearing.

yreg 3y ago

I'm very impressed about both the rapid pace of progress in the area and also the current capabilities of the models. I love them, it brings me joy. I like how suddenly the image generation state of the art made unexpected and significant progress. One can witness something like that only time to time. What I don't get at all is that there are people like you, who are frowning upon and somehow disgusted by people like me.