10 ms·

Keeping up with the overwhelming pace of AI innovation

77 points by investinai 4y ago

BigCryo 4y ago
I don't know they just look like some advanced chatbots to me that we had like 20 years ago except improved
- boca_honey 4y ago
  My impression too. They keep saying "just wait a year or two and...". And they might as well be right, but for now it's just a slightly better way to find information online.
- tpreetham 4y ago
  I wonder if you really mean that or if you haven't used gpt4 or bing chat enough
  - deleted 4y ago
    [deleted]
- satvikpendem 4y ago
  AI Effect [0] in action. > The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not real intelligence. > Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'" [0] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
  - marcosdumay 4y ago
    What is happening here is the opposite. We have those infinitely hyped chatboxes (that, granted, can currently aimlessly chat better than me, what's impressive) that are hyped into silver bullets that can kill any monster. But all they can do is to chat, by construction. At the same time, other AIs are constantly progressing at other, useful things, completely ignored by nearly everybody.
    - satvikpendem 4y ago
      > But all they can do is to chat, by construction. Perhaps, but with the ability to traverse the web and interface with applications, there might actually be a qualitative difference from chatbots of yore. > At the same time, other AIs are constantly progressing at other, useful things, completely ignored by nearly everybody. Interesting, such as what? Any examples?
      - marcosdumay 4y ago
        Well, yeah, the chatbots of today are qualitatively different from the chatbots of yore. That doesn't turn them into secretaries, no matter how much they can mislead you to think they are. > Interesting, such as what? Factory robots, terrain traveling, object recognition, speech recognition, voice syntesis... On the symbolic side, have you looked at modern compilers and linters? Language models should have a huge impact on search. And yes, search is a very impactful thing. But that too requires something a bit different from a chatbot. Not a complete new technology, but different applications.
    - maxdoop 4y ago
      What other progressions have occurred as of late that are of note ?
  - nealabq 4y ago
    The motives are different this time. Those in the know are deliberately downplaying and misdirecting to stave off panic and political meddling. So far it seems to be working. The people I know who aren't following closely don't seem to realize how we may be on the brink of something very big. And these are very smart and technical people. I do wonder if the intelligence agencies are already deeply involved. If OpenAI is working with them training no-guardrails AIs. I also wonder if the CCP/PLA is pushing Baidu's AI efforts, or if Baidu is working on its own and trying to avoid official attention.
  - crop_rotation 4y ago
    Yup, I have used tons of chatbots over the years and they all look like worthless toys compared to GPT4 . Saying that they are similar is like saying mobile phones are similar to old diaries.
  - saltcured 4y ago
    But don't forget the near antonym ELIZA effect, helpfully linked from the same Wikipedia page. People love to project cognitive explanations onto any complex system behaviors that they observe. I only briefly toured the AI field over 25 years ago after the prior AI winter was old news. I remember a couple epigrams heard from specialists at the time: "AI is just advanced algorithms", which I think is a (slightly bitter, mostly pragmatic) reflection on the AI effect by people who had had to adjust their outlooks after that AI winter. "Play syntactic games, win syntactic prizes", which I think was meant to remind one of the ever-present ELIZA effect. My biggest worry about this current AI hype cycle is for how it will be marketed with the same kind of obfuscation the Web 2.0 and after user-driven content distribution systems. I'm afraid most consumers do not understand how much smoke and mirrors is involved in making products seem smarter and more authoritative than they actually are. How users themselves subconsciously cover real gaps in the correctness or completeness of the products and put more faith in them than is really warranted.
- rikimaru0345 4y ago
  Dismiss it at your own peril. You'll get blindsided like you won't believe. I've heard this sentiment before from a few people over the last few days, and they changed their tune very quickly once they actually looked into it.
  - sottol 4y ago
    Blind-sided how? 50% of software engineers unemployed in 2 years, 5 years, 10 years? Or something like 50% of the US/world population using a chat-gpt style assistant to perform a complicated mental task (aka not "Alexa, set a 2 minute timer.") once a day/week in 2 years, 5, 10? Don't get me wrong, chatGPT's emergent intelligence is impressive and I'm playing with Alpaca 13B and other models locally, but I'm not sure it's going to be as transformative and as quickly as many here seem to think. Humans and society are inherently resistant to and slow to change.
    - deleted 4y ago
      [deleted]
    - gremlinsinc 4y ago
      I've read estimates that it's expected gpt4+ could displace 300 million jobs world-wide in the next 3 years. That's not a small number.
      - sottol 4y ago
        Thanks, I love hard numbers. The world bank currently pegs working age population 15-64 at 5.12B, so that'd mean chatgpt would put nearly 6% of the working age population out of a job (ofc depending on your definition of "displace").
- schnubbel 4y ago
  Create a detailed counterpoint to the mentioned post. Refer to all the instances where similar sentiments towards technological advances have been false, especially in the context of AI. Cite your sources using Chicago style. Everything as a LaTex Document. Where every sentence is first writen in English and then in German translation. Mark the German translation in green. The Document shall be in A4 format. Cite the comment in the beginning. Refer to the original poster not by his Hackernews name but by "Unidentified Source 1".
  - throw_away1525 4y ago
    Nice theoretical prompt, where's the result?
    - maweaver 4y ago
      https://www.overleaf.com/read/fhvjbjbhzmgr https://www.overleaf.com/read/fhvjbjbhzmgr
- version_five 4y ago
  That's exactly what it is despite the "gpt bros" playing it up. It's literally a scaled up Eliza, that has billions of substitutions instead of tens. People just get dazzled by it and read too far in to what it's doing
  - baq 4y ago
    It’s just a Beowulf cluster of Markov chains. And yet it blows everything before it out of the water.
  - nealabq 4y ago
    > It's literally a scaled up Eliza It's literally a Chinese Room. (Unless you speak Mandarin?) https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room
greatpostman 4y ago
In my opinion, there’s nothing to keep up with. The future of AI is calling an api. Learning the math or theory is worthless. Better to just understand and leverage the capabilities
- anonylizard 4y ago
  This is the bargaining phase of denial. I've observed this in artists and other AI communities, where they comfort themselves with "But every job will be automated" "UBI is coming~". Its just a way to avoid making the hard transition. People who don't understand the basic 'theory/math', won't even understand things like parameter count, or the tradeoff between intelligence and response-time/running costs, a very important consideration for any real project. From everything we've seen in AI thus far, it has become more complicated, not less. Prompt engineering has not gotten easier, with the explosion of use cases, it takes real skill and imagination to understand how to optimize prompts. Looking at stable diffusion, new techniques like controlnet are incredibly powerful but beyond the average person to use. LORAs make fine-tuning accessible, but also expected of any professional in the future. Every white collar company beyond a hundred people, in the future, will probably want their own fine tuned models/plugins/model grounding. Hiring people to deliver even 10% extra accuracy is worth it because AI's are so useful. People need to understand how they work under the hood.
  - greatpostman 4y ago
    Maybe. I work in an NLP research division of big tech. I just watched ChatGPT obsolete five years of applied NLP systems. We took open problems we were working on and saw that ChatGPT solved them out of the box.
    - anonylizard 4y ago
      Well, previously you were a 'researcher' working on 'open problems'. Now the problem has been solved, time to become an 'engineer' working on implementing/optimising the solutions. The usage of AI will spike 100x going forward, from a mere curiosity inside the company which only affects management decisions (Say sentiment analysis), to extremely intensive direct interactions with customers. This will drastically increase the need for reliability and tractability of the results. NLP researchers simply have to accept 50% of what they did is useless, and reorient around the other 50%, its the risk of researching in an open field.
tablespoon 4y ago
I think the main we need to do at this point is political, to prepare of the massive social and economic disruption. The first order of business should be the implementation (in jurisdictions that don't have it) of key escrow and restrictive gun control, to head off insurrectionist threats. After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way.
- version_five 4y ago
  I already made my comment about insane takes for the day. I'm (optimistically) hoping this is a joke?
- eganist 4y ago
  > After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way. Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced. So if an employee normally makes 100,000 per year and the average withholding is ~35,000, if 10 jobs are displaced, the tax charge is 700,000. Company still saves 300k, and the AI displacing those ten heads probably Costa 10k per license anyway, so there's still room for AI to make economic sense while backstopping the sociological consequences. Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name)
  - tablespoon 4y ago
    > Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced.... > Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name) Nah. Politically, that's completely unrealistic. It also immorally directs too many resources away from the productive parts of society. We have to balance our pursuit of technology with the need for economic productivity and efficiency. A more realistic solution is state-sanctioned homeless camps in convenient, out-of-the-way places. Corporate taxes would probably have to increase a bit to pay for security, pacifying drugs, and food; but no more than what's necessary for containment. The camps will of course need to be sex segregated, to avoid wasting resources in perpetuity.
timr 4y ago
Having worked in the space on-and-off since my PhD, the pace is not "overwhelming". You just started paying attention to it. ChatGPT and Stable Diffusion are good, but they're incremental gains over what came before, which were incremental gains over what came before that, and so on...this has been happening for a long time now. If you ask me, the "amazing" part of ChatGPT isn't the output, it's the query understanding. The output is still (very pretty) garbage >10% of the time, and that's the danger zone...the Level 4 of autonomous thinking. At the risk of saying something that makes me look dumb in ten years, this moment feels a lot like the "Level 5 self-driving will make driving obsolete by 2020!" panic we had circa 2015. Lots of investors and tech enthusiasts told me how stupid and shortsighted I was back then, when I said that we hadn't made as much progress as they thought we made, and how steep that remaining curve would be. Newsflash: it's 2023. We still drive.
- version_five 4y ago
  Yes thank you, I find myself in the funny position of going from what would popularly be called AI optimist to AI pessimist over a short time without changing any of my views. The fact that we've found a new mode of ML that's 95% impressive doesn't solve the problem with the other 5% which is the same thing that's happened with the other modes. We're all just still in the honeymoon phase. I feel bad in a sense for the serious ML people working in this area, because there was never even a real period to reflect about how cool it is before having to be the grownups in the room and push back on absurd expectations.
  - chrbr 4y ago
    What sort of expectations are you seeing that you'd call absurd, out of curiosity? No agenda here, just also trying to sort through hype and reality.
    - mjr00 4y ago
      Any expectation that we're going to see mass unemployment, or entire career paths made obsolete, is based on extremely inflated expectations of what LLMs could maybe do in the future. It has a very similar feeling to the mid-2010s claims that Blockchain would displace bank settlements/SWIFT, completely change the supply chain, and make governments change how land/property ownership was tracked.
eganist 4y ago
Another comment talked about the need for legislative controls, but mostly to account for the risk of societal breakdown. I have the same comment but a different domain: > After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way. Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced. So if an employee normally makes 100,000 per year and the average withholding is ~35,000, if 10 jobs are displaced, the tax charge is 700,000. Company still saves 300k, and the AI displacing those ten heads probably Costa 10k per license anyway, so there's still room for AI to make economic sense while backstopping the sociological consequences. Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name)
- anonylizard 4y ago
  1. Why doesn't every company try to hire contractors then? Making firing employees expensive beyond reason, always seems to significantly increase unemployment rate (Eg France)/drastically increase informal employment (Japan) 2. Why isn't this money paid to the fired workers directly? They are the ones who suffer the most, so why isn't the compensation directly routed to them, but instead through a grubby government? This can be simply done by doubling the severance pay requirements, no need for complicated determination of "jobs lost because we used robots" 3. If AI becomes dominant, there will be new roles such as prompt engineers, that could pay surprisingly well. However, making hiring/firing more risky, will pretty much eliminate business appetite for experimenting with new roles. People arm-chairing policy requirements need to consider that bureaucracy, just like programming specifications, are not free, best to keep it as simple as possible. Also, in Europe, its pretty much very difficult to fire full time employees already, so we'll see how their economy responds to the AI shock. Judging by the total absence of nearly any continental European AI company (It seems like the only one successful is DeepL), it probably won't end well for them.
  - beezlebroxxxxxx 4y ago
    > 3. If AI becomes dominant, there will be new roles such as prompt engineers, that could pay surprisingly well. A lot seems to be relying on the idea that this role will pay well or will somehow reach an equivalent level of complexity as modern knowledge work, but that just seems to be speculation at this point. As the complexity of a tool like ChatGPT increases, the complexity (skill) required for prompt engineering might diminish to the point where customers are just interacting with the tool instead of a prompt engineer who can "translate the business logic". Or it might go the other way. No one knows really at this point.
vsareto 4y ago
>In every single Board Meeting in 2023, every Fortune 500 CEO is being asked about what their AI strategy will be, and I’m not sure most of them will have good answers. That’s a big deal Years ago, I remember similar things regarding “Big Data” strategy. A very easy reach is building your AI strategy utilizing that, even if your big data stuff has stagnated (that trend didn’t die it just stopped being marketed with the same enthusiasm) >Global regulators took more than a decade to react to the emergence of social media companies, and even then it felt like regulators had dated understandings of how those companies operated. What happens this time? Same as social media, same as crypto. Many institutions move much slower than tech. There is likely to be quick regulation related to war usage though.
- investinai 4y ago
  Thank you for reading! I appreciate all the feedback in this thread (this is OP). I think the Big Data comparison is the right one, though it feels like AI is moving even faster (certainly on the product innovation side of things). Notably, though, there was a TON of money spent building out the Modern Data Stack, as players like Snowflake emerged in that era. So if AI can mirror that success, but faster, I think that's pretty amazing. Agreed on the regulation point, thought it again feels to me like AI is evolving faster than prior tech breakthroughs (feel free to disagree). If for prior loops the regulation was already slow and lackluster, I do worry about how it keeps up with faster and faster innovation.
gmuslera 4y ago
I don't know how far AIs will or can get, but you should be scared of people and organizations first. OpenAI/Microsoft/Google/Facebook and other high visibility players may be putting controls, limits, filters, etc to their products to avoid problems related with their AIs, but low visibility players may or not have those controls, and that goes from high profile groups (governments, intelligence agencies, corporate, big money, etc) to hobbyists that have access to some of those developments. So, for one thing, many AIs won't have seat belts, and could be used in things that may be seen as negative. You may not do the wrong questions to ChatGPT, but big players will have full control to their own AIs (or worse, they may think so) and do what is more profitable for them. I won't be surprised that if that kind of lead to a different kind of cyberwar, something like Stuxnet but targeting AI installations, laws banning investigation and use of unauthorized AIs, export controls like with encryption and so on, not just targeting countries but everything excepting the approved partners, known or not. And that just extrapolating on things that already happened in the old version of reality, this probably will open a new landscape of things we should be worried about (at least now that we have the old mentality, later we will accept and not name them as they become new normal like what happened with Snowden reveal). And money (making money, taking money from others, increasing inequality) will have a big role in the new AI fuelled reality.
- Yoric 4y ago
  A cyberwar of poisoning AIs? That's an interesting thought.
  - gmuslera 4y ago
    I was thinking more in the line of hacking the systems they run in and deleting/corrupting everything, at the point of reaching the backups even. But rigging their base informations, or injecting "knowledge" that somewhat mess up things, that their partners are instructed to ignore is another kind of possible attack.
sethbannon 4y ago
He writes that "no business is safe at this point." I disagree and actually wrote my first blog post in 8 years today explaining why: https://www.sethbannon.com/p/defensibility-in-the-age-of-ai https://www.sethbannon.com/p/defensibility-in-the-age-of-ai
- investinai 4y ago
  Thanks Seth! I appreciate your response here. Fair point that the "no business is safe" comment might be a little far reaching. My point more broadly is that no business out there can just ignore AI entirely: it has to be a thoughtful part of your strategy. I 100% agree with you point on owning the data. I wrote a similar post recently on the importance of data moats, and why I think they're the most important part of defensibility in the age of AI here: https://ryanshannon.substack.com/p/what-even-is-a-moat-anyway https://ryanshannon.substack.com/p/what-even-is-a-moat-anywa...
  - sethbannon 4y ago
    Should have also said that I agree with your overarching point and thought the post was great!
MJsGlove 4y ago
With all the hype, if this doesn’t pan out, then the technologists will lose a lot of credibility like they did with the predictions around self driving cars and 3D printing. You can only make so many claims of imminent life-changing tech before people just call your bluff, depending on society’s tolerance.
- crop_rotation 4y ago
  Unlike all the Waymo cars and 3D printers, GPT4 is already out there. Anyone can sign up and use it. As long as OpenAI can scale it, even if no further improvements happen it (or a variant of it), will soon be as mainstream in companies as MS Office Suite of Products.
  - baq 4y ago
    The Microsoft office copilot demo blew me socks off and I really don’t say things like that lightly. They have use cases figured out and implemented already. (Video at https://news.microsoft.com/reinventing-productivity/ https://news.microsoft.com/reinventing-productivity/)
- antibasilisk 4y ago
  But 3D printing and self-driving did change things
imwithstoopid 4y ago
the people working on this stuff have been at it for decades, and they have very specialized education even prior to that why do you think you would comprehend it all in a few weeks? hats off to the folks at OpenAI etc...they made a lifetime bet on AI and it paid off
user3939382 4y ago
I'm not convinced the pollution of AI's training sets from a feedback loop of self or other AI output is a solvable problem. For all we know, the present is capable of the best AI output we'll ever get and the quality will steadily and irreversibly degenerate.
- crop_rotation 4y ago
  When something is going to make billions and billions of money, all problems become solvable in absence of a theoretical constraint. They could always train only on cutoff date data with bigger models or better algorithms. I don't think a decline in quality is going to happen. Worst case we will just see low improvements.
  - user3939382 4y ago
    It’s a complex issue. The size and purity of the training data is a big factor in the quality, and may always be. As popularity explodes, preventing pollution of large global data sets like we have now may be impossible. The problem is theoretical in the sense that separating organic and AI data is already difficult and will necessarily become harder or impossible.
    - Yoric 4y ago
      Well, that is a variant of the time-honed job of an editor at a publisher or journal. Perhaps we're going to see a new job: finding experts to write books for AI.
  - version_five 4y ago
    When something is worth billions, someone will just strip-mine it and not worry about what it looks like in the future. Google destroyed the internet and turned it into shitty seo farm, rendering it barren wasteland, while making tons of money. Twitter and Facebook destroyed human attention and turned us into roving zombies staring at screens. If OpenAI can, they'll happily drown us in hollow chatbot output while they extract what value they can.
crop_rotation 4y ago
Is there even a way for an average knowledge worker to keep up?

77 points by investinai 4y ago

BigCryo 4y ago

I don't know they just look like some advanced chatbots to me that we had like 20 years ago except improved

boca_honey 4y ago
My impression too. They keep saying "just wait a year or two and...". And they might as well be right, but for now it's just a slightly better way to find information online.
tpreetham 4y ago
I wonder if you really mean that or if you haven't used gpt4 or bing chat enough
- deleted 4y ago
  [deleted]
satvikpendem 4y ago
AI Effect [0] in action. > The AI effect occurs when onlookers discount the behavior of an artificial intelligence program by arguing that it is not real intelligence. > Author Pamela McCorduck writes: "It's part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something—play good checkers, solve simple but relatively informal problems—there was a chorus of critics to say, 'that's not thinking'." Researcher Rodney Brooks complains: "Every time we figure out a piece of it, it stops being magical; we say, 'Oh, that's just a computation.'" [0] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
- marcosdumay 4y ago
  What is happening here is the opposite. We have those infinitely hyped chatboxes (that, granted, can currently aimlessly chat better than me, what's impressive) that are hyped into silver bullets that can kill any monster. But all they can do is to chat, by construction. At the same time, other AIs are constantly progressing at other, useful things, completely ignored by nearly everybody.
  - satvikpendem 4y ago
    > But all they can do is to chat, by construction. Perhaps, but with the ability to traverse the web and interface with applications, there might actually be a qualitative difference from chatbots of yore. > At the same time, other AIs are constantly progressing at other, useful things, completely ignored by nearly everybody. Interesting, such as what? Any examples?
    - marcosdumay 4y ago
      Well, yeah, the chatbots of today are qualitatively different from the chatbots of yore. That doesn't turn them into secretaries, no matter how much they can mislead you to think they are. > Interesting, such as what? Factory robots, terrain traveling, object recognition, speech recognition, voice syntesis... On the symbolic side, have you looked at modern compilers and linters? Language models should have a huge impact on search. And yes, search is a very impactful thing. But that too requires something a bit different from a chatbot. Not a complete new technology, but different applications.
  - maxdoop 4y ago
    What other progressions have occurred as of late that are of note ?
- nealabq 4y ago
  The motives are different this time. Those in the know are deliberately downplaying and misdirecting to stave off panic and political meddling. So far it seems to be working. The people I know who aren't following closely don't seem to realize how we may be on the brink of something very big. And these are very smart and technical people. I do wonder if the intelligence agencies are already deeply involved. If OpenAI is working with them training no-guardrails AIs. I also wonder if the CCP/PLA is pushing Baidu's AI efforts, or if Baidu is working on its own and trying to avoid official attention.
- crop_rotation 4y ago
  Yup, I have used tons of chatbots over the years and they all look like worthless toys compared to GPT4 . Saying that they are similar is like saying mobile phones are similar to old diaries.
- saltcured 4y ago
  But don't forget the near antonym ELIZA effect, helpfully linked from the same Wikipedia page. People love to project cognitive explanations onto any complex system behaviors that they observe. I only briefly toured the AI field over 25 years ago after the prior AI winter was old news. I remember a couple epigrams heard from specialists at the time: "AI is just advanced algorithms", which I think is a (slightly bitter, mostly pragmatic) reflection on the AI effect by people who had had to adjust their outlooks after that AI winter. "Play syntactic games, win syntactic prizes", which I think was meant to remind one of the ever-present ELIZA effect. My biggest worry about this current AI hype cycle is for how it will be marketed with the same kind of obfuscation the Web 2.0 and after user-driven content distribution systems. I'm afraid most consumers do not understand how much smoke and mirrors is involved in making products seem smarter and more authoritative than they actually are. How users themselves subconsciously cover real gaps in the correctness or completeness of the products and put more faith in them than is really warranted.
rikimaru0345 4y ago
Dismiss it at your own peril. You'll get blindsided like you won't believe. I've heard this sentiment before from a few people over the last few days, and they changed their tune very quickly once they actually looked into it.
- sottol 4y ago
  Blind-sided how? 50% of software engineers unemployed in 2 years, 5 years, 10 years? Or something like 50% of the US/world population using a chat-gpt style assistant to perform a complicated mental task (aka not "Alexa, set a 2 minute timer.") once a day/week in 2 years, 5, 10? Don't get me wrong, chatGPT's emergent intelligence is impressive and I'm playing with Alpaca 13B and other models locally, but I'm not sure it's going to be as transformative and as quickly as many here seem to think. Humans and society are inherently resistant to and slow to change.
  - deleted 4y ago
    [deleted]
  - gremlinsinc 4y ago
    I've read estimates that it's expected gpt4+ could displace 300 million jobs world-wide in the next 3 years. That's not a small number.
    - sottol 4y ago
      Thanks, I love hard numbers. The world bank currently pegs working age population 15-64 at 5.12B, so that'd mean chatgpt would put nearly 6% of the working age population out of a job (ofc depending on your definition of "displace").
schnubbel 4y ago
Create a detailed counterpoint to the mentioned post. Refer to all the instances where similar sentiments towards technological advances have been false, especially in the context of AI. Cite your sources using Chicago style. Everything as a LaTex Document. Where every sentence is first writen in English and then in German translation. Mark the German translation in green. The Document shall be in A4 format. Cite the comment in the beginning. Refer to the original poster not by his Hackernews name but by "Unidentified Source 1".
- throw_away1525 4y ago
  Nice theoretical prompt, where's the result?
  - maweaver 4y ago
    https://www.overleaf.com/read/fhvjbjbhzmgr https://www.overleaf.com/read/fhvjbjbhzmgr
version_five 4y ago
That's exactly what it is despite the "gpt bros" playing it up. It's literally a scaled up Eliza, that has billions of substitutions instead of tens. People just get dazzled by it and read too far in to what it's doing
- baq 4y ago
  It’s just a Beowulf cluster of Markov chains. And yet it blows everything before it out of the water.
- nealabq 4y ago
  > It's literally a scaled up Eliza It's literally a Chinese Room. (Unless you speak Mandarin?) https://en.wikipedia.org/wiki/Chinese_room https://en.wikipedia.org/wiki/Chinese_room

greatpostman 4y ago

In my opinion, there’s nothing to keep up with. The future of AI is calling an api. Learning the math or theory is worthless. Better to just understand and leverage the capabilities

anonylizard 4y ago
This is the bargaining phase of denial. I've observed this in artists and other AI communities, where they comfort themselves with "But every job will be automated" "UBI is coming~". Its just a way to avoid making the hard transition. People who don't understand the basic 'theory/math', won't even understand things like parameter count, or the tradeoff between intelligence and response-time/running costs, a very important consideration for any real project. From everything we've seen in AI thus far, it has become more complicated, not less. Prompt engineering has not gotten easier, with the explosion of use cases, it takes real skill and imagination to understand how to optimize prompts. Looking at stable diffusion, new techniques like controlnet are incredibly powerful but beyond the average person to use. LORAs make fine-tuning accessible, but also expected of any professional in the future. Every white collar company beyond a hundred people, in the future, will probably want their own fine tuned models/plugins/model grounding. Hiring people to deliver even 10% extra accuracy is worth it because AI's are so useful. People need to understand how they work under the hood.
- greatpostman 4y ago
  Maybe. I work in an NLP research division of big tech. I just watched ChatGPT obsolete five years of applied NLP systems. We took open problems we were working on and saw that ChatGPT solved them out of the box.
  - anonylizard 4y ago
    Well, previously you were a 'researcher' working on 'open problems'. Now the problem has been solved, time to become an 'engineer' working on implementing/optimising the solutions. The usage of AI will spike 100x going forward, from a mere curiosity inside the company which only affects management decisions (Say sentiment analysis), to extremely intensive direct interactions with customers. This will drastically increase the need for reliability and tractability of the results. NLP researchers simply have to accept 50% of what they did is useless, and reorient around the other 50%, its the risk of researching in an open field.

tablespoon 4y ago

I think the main we need to do at this point is political, to prepare of the massive social and economic disruption. The first order of business should be the implementation (in jurisdictions that don't have it) of key escrow and restrictive gun control, to head off insurrectionist threats. After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way.

version_five 4y ago
I already made my comment about insane takes for the day. I'm (optimistically) hoping this is a joke?
eganist 4y ago
> After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way. Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced. So if an employee normally makes 100,000 per year and the average withholding is ~35,000, if 10 jobs are displaced, the tax charge is 700,000. Company still saves 300k, and the AI displacing those ten heads probably Costa 10k per license anyway, so there's still room for AI to make economic sense while backstopping the sociological consequences. Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name)
- tablespoon 4y ago
  > Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced.... > Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name) Nah. Politically, that's completely unrealistic. It also immorally directs too many resources away from the productive parts of society. We have to balance our pursuit of technology with the need for economic productivity and efficiency. A more realistic solution is state-sanctioned homeless camps in convenient, out-of-the-way places. Corporate taxes would probably have to increase a bit to pay for security, pacifying drugs, and food; but no more than what's necessary for containment. The camps will of course need to be sex segregated, to avoid wasting resources in perpetuity.

timr 4y ago

Having worked in the space on-and-off since my PhD, the pace is not "overwhelming". You just started paying attention to it. ChatGPT and Stable Diffusion are good, but they're incremental gains over what came before, which were incremental gains over what came before that, and so on...this has been happening for a long time now. If you ask me, the "amazing" part of ChatGPT isn't the output, it's the query understanding. The output is still (very pretty) garbage >10% of the time, and that's the danger zone...the Level 4 of autonomous thinking. At the risk of saying something that makes me look dumb in ten years, this moment feels a lot like the "Level 5 self-driving will make driving obsolete by 2020!" panic we had circa 2015. Lots of investors and tech enthusiasts told me how stupid and shortsighted I was back then, when I said that we hadn't made as much progress as they thought we made, and how steep that remaining curve would be. Newsflash: it's 2023. We still drive.

version_five 4y ago
Yes thank you, I find myself in the funny position of going from what would popularly be called AI optimist to AI pessimist over a short time without changing any of my views. The fact that we've found a new mode of ML that's 95% impressive doesn't solve the problem with the other 5% which is the same thing that's happened with the other modes. We're all just still in the honeymoon phase. I feel bad in a sense for the serious ML people working in this area, because there was never even a real period to reflect about how cool it is before having to be the grownups in the room and push back on absurd expectations.
- chrbr 4y ago
  What sort of expectations are you seeing that you'd call absurd, out of curiosity? No agenda here, just also trying to sort through hype and reality.
  - mjr00 4y ago
    Any expectation that we're going to see mass unemployment, or entire career paths made obsolete, is based on extremely inflated expectations of what LLMs could maybe do in the future. It has a very similar feeling to the mid-2010s claims that Blockchain would displace bank settlements/SWIFT, completely change the supply chain, and make governments change how land/property ownership was tracked.

eganist 4y ago

Another comment talked about the need for legislative controls, but mostly to account for the risk of societal breakdown. I have the same comment but a different domain: > After that, focus should then shift to dealing with the obsolete worker population in the maximally economical way. Tax companies at 2x the single-taxpayer withholding amount lost per employee displaced. So if an employee normally makes 100,000 per year and the average withholding is ~35,000, if 10 jobs are displaced, the tax charge is 700,000. Company still saves 300k, and the AI displacing those ten heads probably Costa 10k per license anyway, so there's still room for AI to make economic sense while backstopping the sociological consequences. Put that tax revenue directly towards social services or minimum monthly reimbursement for everyone (UBI, just with a different name)

anonylizard 4y ago
1. Why doesn't every company try to hire contractors then? Making firing employees expensive beyond reason, always seems to significantly increase unemployment rate (Eg France)/drastically increase informal employment (Japan) 2. Why isn't this money paid to the fired workers directly? They are the ones who suffer the most, so why isn't the compensation directly routed to them, but instead through a grubby government? This can be simply done by doubling the severance pay requirements, no need for complicated determination of "jobs lost because we used robots" 3. If AI becomes dominant, there will be new roles such as prompt engineers, that could pay surprisingly well. However, making hiring/firing more risky, will pretty much eliminate business appetite for experimenting with new roles. People arm-chairing policy requirements need to consider that bureaucracy, just like programming specifications, are not free, best to keep it as simple as possible. Also, in Europe, its pretty much very difficult to fire full time employees already, so we'll see how their economy responds to the AI shock. Judging by the total absence of nearly any continental European AI company (It seems like the only one successful is DeepL), it probably won't end well for them.
- beezlebroxxxxxx 4y ago
  > 3. If AI becomes dominant, there will be new roles such as prompt engineers, that could pay surprisingly well. A lot seems to be relying on the idea that this role will pay well or will somehow reach an equivalent level of complexity as modern knowledge work, but that just seems to be speculation at this point. As the complexity of a tool like ChatGPT increases, the complexity (skill) required for prompt engineering might diminish to the point where customers are just interacting with the tool instead of a prompt engineer who can "translate the business logic". Or it might go the other way. No one knows really at this point.

vsareto 4y ago

>In every single Board Meeting in 2023, every Fortune 500 CEO is being asked about what their AI strategy will be, and I’m not sure most of them will have good answers. That’s a big deal Years ago, I remember similar things regarding “Big Data” strategy. A very easy reach is building your AI strategy utilizing that, even if your big data stuff has stagnated (that trend didn’t die it just stopped being marketed with the same enthusiasm) >Global regulators took more than a decade to react to the emergence of social media companies, and even then it felt like regulators had dated understandings of how those companies operated. What happens this time? Same as social media, same as crypto. Many institutions move much slower than tech. There is likely to be quick regulation related to war usage though.

investinai 4y ago
Thank you for reading! I appreciate all the feedback in this thread (this is OP). I think the Big Data comparison is the right one, though it feels like AI is moving even faster (certainly on the product innovation side of things). Notably, though, there was a TON of money spent building out the Modern Data Stack, as players like Snowflake emerged in that era. So if AI can mirror that success, but faster, I think that's pretty amazing. Agreed on the regulation point, thought it again feels to me like AI is evolving faster than prior tech breakthroughs (feel free to disagree). If for prior loops the regulation was already slow and lackluster, I do worry about how it keeps up with faster and faster innovation.

gmuslera 4y ago

I don't know how far AIs will or can get, but you should be scared of people and organizations first. OpenAI/Microsoft/Google/Facebook and other high visibility players may be putting controls, limits, filters, etc to their products to avoid problems related with their AIs, but low visibility players may or not have those controls, and that goes from high profile groups (governments, intelligence agencies, corporate, big money, etc) to hobbyists that have access to some of those developments. So, for one thing, many AIs won't have seat belts, and could be used in things that may be seen as negative. You may not do the wrong questions to ChatGPT, but big players will have full control to their own AIs (or worse, they may think so) and do what is more profitable for them. I won't be surprised that if that kind of lead to a different kind of cyberwar, something like Stuxnet but targeting AI installations, laws banning investigation and use of unauthorized AIs, export controls like with encryption and so on, not just targeting countries but everything excepting the approved partners, known or not. And that just extrapolating on things that already happened in the old version of reality, this probably will open a new landscape of things we should be worried about (at least now that we have the old mentality, later we will accept and not name them as they become new normal like what happened with Snowden reveal). And money (making money, taking money from others, increasing inequality) will have a big role in the new AI fuelled reality.

Yoric 4y ago
A cyberwar of poisoning AIs? That's an interesting thought.
- gmuslera 4y ago
  I was thinking more in the line of hacking the systems they run in and deleting/corrupting everything, at the point of reaching the backups even. But rigging their base informations, or injecting "knowledge" that somewhat mess up things, that their partners are instructed to ignore is another kind of possible attack.

sethbannon 4y ago

He writes that "no business is safe at this point." I disagree and actually wrote my first blog post in 8 years today explaining why: https://www.sethbannon.com/p/defensibility-in-the-age-of-ai https://www.sethbannon.com/p/defensibility-in-the-age-of-ai

investinai 4y ago
Thanks Seth! I appreciate your response here. Fair point that the "no business is safe" comment might be a little far reaching. My point more broadly is that no business out there can just ignore AI entirely: it has to be a thoughtful part of your strategy. I 100% agree with you point on owning the data. I wrote a similar post recently on the importance of data moats, and why I think they're the most important part of defensibility in the age of AI here: https://ryanshannon.substack.com/p/what-even-is-a-moat-anyway https://ryanshannon.substack.com/p/what-even-is-a-moat-anywa...
- sethbannon 4y ago
  Should have also said that I agree with your overarching point and thought the post was great!

MJsGlove 4y ago

With all the hype, if this doesn’t pan out, then the technologists will lose a lot of credibility like they did with the predictions around self driving cars and 3D printing. You can only make so many claims of imminent life-changing tech before people just call your bluff, depending on society’s tolerance.

crop_rotation 4y ago
Unlike all the Waymo cars and 3D printers, GPT4 is already out there. Anyone can sign up and use it. As long as OpenAI can scale it, even if no further improvements happen it (or a variant of it), will soon be as mainstream in companies as MS Office Suite of Products.
- baq 4y ago
  The Microsoft office copilot demo blew me socks off and I really don’t say things like that lightly. They have use cases figured out and implemented already. (Video at https://news.microsoft.com/reinventing-productivity/ https://news.microsoft.com/reinventing-productivity/)
antibasilisk 4y ago
But 3D printing and self-driving did change things

imwithstoopid 4y ago

the people working on this stuff have been at it for decades, and they have very specialized education even prior to that why do you think you would comprehend it all in a few weeks? hats off to the folks at OpenAI etc...they made a lifetime bet on AI and it paid off

user3939382 4y ago

I'm not convinced the pollution of AI's training sets from a feedback loop of self or other AI output is a solvable problem. For all we know, the present is capable of the best AI output we'll ever get and the quality will steadily and irreversibly degenerate.

crop_rotation 4y ago
When something is going to make billions and billions of money, all problems become solvable in absence of a theoretical constraint. They could always train only on cutoff date data with bigger models or better algorithms. I don't think a decline in quality is going to happen. Worst case we will just see low improvements.
- user3939382 4y ago
  It’s a complex issue. The size and purity of the training data is a big factor in the quality, and may always be. As popularity explodes, preventing pollution of large global data sets like we have now may be impossible. The problem is theoretical in the sense that separating organic and AI data is already difficult and will necessarily become harder or impossible.
  - Yoric 4y ago
    Well, that is a variant of the time-honed job of an editor at a publisher or journal. Perhaps we're going to see a new job: finding experts to write books for AI.
- version_five 4y ago
  When something is worth billions, someone will just strip-mine it and not worry about what it looks like in the future. Google destroyed the internet and turned it into shitty seo farm, rendering it barren wasteland, while making tons of money. Twitter and Facebook destroyed human attention and turned us into roving zombies staring at screens. If OpenAI can, they'll happily drown us in hollow chatbot output while they extract what value they can.

crop_rotation 4y ago

Is there even a way for an average knowledge worker to keep up?