8 ms·
> Is nobody in these very rich guys' spheres pushing back on their thought process? Yes, frequently and loudly. When Altman was collecting the award at Cambri
by ben_w 2y ago
> Is nobody in these very rich guys' spheres pushing back on their thought process?
Yes, frequently and loudly.
When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors.
> So far we are multiple years in with much investment and little return, and no obvious large-scale product-market fit, much less a superintelligence.
I just got back from an Indian restaurant in the middle of Berlin, and the table next to me I overheard a daughter talking to her mother about ChatGPT and KI (Künstliche Intelligenz, the German for AI).
The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public.
What's not obvious is how to monetise it. Old meme parroted around was "has no moat", which IMO is like saying Microsoft has no moat for spreadsheets: sure, anyone can make the core tech, and sure we don't know who is Microsoft vs StarOffice vs ClarisWorks vs Google Docs, but there's more than zero moat. From what I've seen, if OpenAI didn't develop new products, they'd be making enough to be profitable, but it's a Red Queen race to remain worth paying for.
As for "much less a superintelligence": even the current models meet every definition of "very smart" I had while growing up, despite their errors. As an adult, I'd still call them book-smart if not abstractly smart. Students or recent graduates, but not wise enough to know their limits and be cautious.
For current standards of what intelligence means, we'd better hope we don't get ASI in the next decade or two, because if and when that happens then "humans need not apply" — and by extension, foundational assumptions of economics may just stop holding true.
- this_user 2y ago> When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors. He always does that to give himself cover, but he has clearly shown that his words mean very little in this regard. He always dodges criticism. He used to talk about the importance of him being accountable to the OpenAI board and them being able to fire him if necessary when people were questioning the dangers of having one person have this much control over something as big as bleeding edge AI. He also used to mention how he had no direct financial interests in the company since he had no equity. Then the board did fire him. What happened next? He came back, the board is gone, he now openly has complete control over OpenAI, and they have given him a potentially huge equity package. I really don't think Sam Altman is particularly trustworthy. He will say whatever he needs to say to get what he wants.
- kulahan 2y agoWasn't he fired for questionable reasons? I thought everyone wanted him back, and that's why he was able to return. It was, as I remember, just the board that wanted him out. I imagine if he was doing something truly nefarious, opinions might have been different, but I have no idea what kind of cult of personality he has at that company, so I might be wrong here.
- samvher 2y agoDefine everyone. I was delighted when they fired him. I don't believe he has humanity's best interest at heart.
- ben_w 2y ago745 of 770 employees responded on a weekend to call for his return, threatened mass resignations if the board did not resign; among the signatories was board member Sutskever, who defected from the board and publicly apologized for his participation in the board's previous actions.
- fwip 2y ago745 of 700?
- ben_w 2y agoWhoops, typo for 770, edited to correct. Thanks!
- samvher 2y agoI followed the drama. The point I was (somewhat unsuccessfully) trying to make was that while, sure, there were groups who wanted him back (mainly the groups with vested financial interests and associated leverage), my sense was that the way it played out was not necessarily in line with wider humanity's best interest, i.e. as would have been hoped based on OpenAI's publicly stated goals.
- 23B1 2y ago> and he actively empathised with the protestors. https://www.simplypsychology.org/narcissistic-mirroring.html https://www.simplypsychology.org/narcissistic-mirroring.html
- ben_w 2y agoEven if so, that's still showing awareness of what his critics are critical of. Now, can you make a falsifiable prediction: what would a narcissist do that a normal person would not, such that you can tell if he's engaging in a narcissist process rather that what your own link also says is a perfectly normal and healthy behaviour?
- tivert 2y ago> Even if so, that's still showing awareness of what his critics are critical of. Mere awareness is really that meaningful. >>> and he actively empathised with the protestors. >> https://www.simplypsychology.org/narcissistic-mirroring.html https://www.simplypsychology.org/narcissistic-mirroring.html > Now, can you make a falsifiable prediction: what would a narcissist do that a normal person would not, such that you can tell if he's engaging in a narcissist process rather that what your own link also says is a perfectly normal and healthy behaviour? He doesn't have to, because I think the point was to raise doubts about your interpretation. But if you're looking for more evidence, there are all the stories (from many different people) about Altman being dishonest and manipulative and being very effective at it. That's the context that a lot of people are going use to interpret your claim that Altman "actively empathised with the protestors."
- xdavidliu 2y agonit: isn't really that meaningful
- manquer 2y ago> The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public. Hardly the evidence of PMF. There is always something new in the zeitgeist, that every one is talking about, some more so than others . 2 years before it was VR, few years before that NFTs and blockchain everything, before that it was self driving cars before that personal voice assistants like Siri and so on . - self driving has not transformed us into minority report and despite how far it has come it cannot in next 30 years be ubiquitous, even if the l5 magic tech exists today in every new car sold it will take 15 years for current cars to lifecycle. - Crypto has not replaced fiat currency , even in most generous reading you can see it as store of value like gold or whatever useless baubles people assign arbitrary value to, but has no traction for 3 out of other 4 key functions of money . - VR is not transformative to every day life and is 5 fundamental breakthroughs away. - Voice assistants are useless setting alarms and selecting music 10 years in. There has been meaningful and measurable in each of these fields, but none of them have met the high bar of world transforming . AI is aiming for much higher bar of singularity and consciousness. Just in every hype cycles we are in peak of inflated expectations, we will reach a plateau of productivity where it is will be useful in specific areas (as it already is) and people will move on to the next fad.
- bee_rider 2y agoConsciousness is a stupid and unreasonable goal, it is basically impossible to confirm that a machine isn’t just faking it really well. Singularity is at least definable…Although I think it is not really the bar required to be really impactful. If we get an AI system that can do the work of like 60% of hardcore knowledge workers, 80% of office workers, and 95% of CEOs/politicians and other pablumists, it could be really change how the economy works without actually being a singularity.
- deleted 2y ago[deleted]
- lupire 2y agoThose numbers are approximately zero, because demand will immediately 10x as soon as AI makes that affordable, as computers have always done. So, profitable (if the AI can cost less than a human for the work), but not changing how the economy works
- naming_the_user 2y ago> For current standards of what intelligence means, we'd better hope we don't get ASI in the next decade or two, because if and when that happens then "humans need not apply" — and by extension, foundational assumptions of economics may just stop holding true. I'm not sure that we need superintelligence for that to be the case - it may depend on whether you include physical ability in the definition of intelligence. At the point that we have an AI that's capable of every task that say a 110 IQ human is, including manipulating objects in the physical world, then basically everyone is unemployed unless they're cheaper than the AI.
- ben_w 2y agoWhile I would certainly expect a radical change to economics even from a middling IQ AI — or indeed a low IQ, as I have previously used the example of IQ 85 because that's 15.9% of the population that would become permanently unable to be economically useful — I don't think it's quite as you say. Increasing IQ scores seem to allow increasingly difficult tasks to be performed competently — not just the same tasks faster, and also not just "increasingly difficult" in the big-O-notation sense, but it seems like below certain IQ thresholds (or above them but with certain pathologies), some thoughts just aren't "thinkable" even with unbounded time. While this might simply be an illusion that breaks with computers because silicon outpaces synapses by literally the degree to which jogging outpaces continental drift, I don't see strong evidence at this time for the idea that this is an illusion. We may get that evidence in a very short window, but I don't see it yet. Therefore, in the absence of full brain uploads, I suspect that higher IQ people may well be able to perform useful work even as lower IQ people are outclassed by AI. If we do get full brain uploads, then it's the other way around, as a few super-geniuses will get their brains scanned but say it takes a billion dollars a year to run the sim in real time, then Moore's and Koomey's laws will take n years to lower that to $10 million dollars a year, 2n years to lower it to a $100k a year, and 3n years to lower it to $1k/year.
- naming_the_user 2y agoI completely agree with everything that you're saying here - my use of the term "basically everyone" was lazy in that I'm implying that at the 110 IQ level the majority (approx 70%) of people are economically obsolete outside of niche areas (e.g. care/butler style work in which people desire the "human touch" for emotional reasons). I think that far below the 70% level we've already broken economics. I can't see a society functioning in which most people actually _know_ that they can't break out of whatever box they're currently in - I think that things like UBI are a distraction in that they don't account for things like status jockeying that I think we're pretty hardwired to do.
- jkaptur 2y agoCan you expand on your spreadsheet analogy? I think Joel Spolsky explained the main Office moat well here: https://www.joelonsoftware.com/2008/02/19/why-are-the-microsoft-office-file-formats-so-complicated-and-some-workarounds/ https://www.joelonsoftware.com/2008/02/19/why-are-the-micros... > ... it might take you weeks to change your page layout algorithm to accommodate it. If you don’t, customers will open their Word files in your clone and all the pages will be messed up. Basically, people who use Office have extremely specific expectations. (I've seen people try a single keyboard shortcut, see that it doesn't work in a web-based application, and declare that whole thing "doesn't work".) Reimplementing all that stuff is really time consuming. There's also a strong network effect - if your company uses Office, you'll probably use it too. On the other hand, people don't have extremely specific expectations for LLMs because 1) they're fairly new and 2) they're almost always nondeterministic anyway. They don't care so much about using the same one as everyone they know or work with, because there's no network aspect of the product. I don't think the moats are similar.
- pj_mukh 2y ago"Basically, people who use Office have extremely specific expectations." Interesting point, but to OP's point- This wasn't true when Office was first introduced and Office still created a domineering market share. In fact, I'd argue these moat-by-idiosyncracy features are a result of that market share. There is nothing stopping OpenAI from developing their own over time.
- bee_rider 2y agoDoes office actually have a moat? I thought the kids liked Google docs nowadays. (No opinion as to which is actually better, the actual thing people should do is write LaTeX in vim. You can even share documents! Just have everybody attach to the same tmux session and take turns).
- ben_w 2y agoIf you're writing a spreadsheet in LaTeX, I suspect something has gone very wrong somewhere along the line. Spreadsheets are as much a calculation environment as they are a table of figures, and if you're technical enough to be writing docs in LaTeX you should be doing the calculations in a more rigorous environment where you don't copy the formulas between cells.
- lyu07282 2y ago> foundational assumptions of economics may just stop holding true Those assumptions are already failing billions of people, some people might still be benefiting from those "assumptions of economics" so they don't see the magnitude of the problem. But just as the billions who suffer now have no power, so will you have no power once those assumptions fail for you too.
- sangnoir 2y ago> ...like saying Microsoft has no moat for spreadsheets Which would be very inaccurate as network-effects are Excel's (and Word's) moat. Excel being bundled with Office and Windows helped, but it beat Lotus-123 by being a superior product at a time the computing landscape was changing. OpenAI has no such advantage yet: a text-based API is about as commoditized as a technology can get and OpenAI is furiously launching interfaces with lower interoperability (where one can't replace GPT-4o with Claude 3.5 via a drop-down)
- sumedh 2y ago> OpenAI has no such advantage yet: a text-based API is about as commoditized as a technology It has branding, for most people AI is ChatGpt. Once you reach critical mass, getting people to switch becomes difficult if your product is good enough and most people are happy.
- tsimionescu 2y agoBranding is a very weak moat outside fashion industry and similar places - outside visible status symbols. It's a relatively strong moat for clothes, watches, even things like the iPhone. But for things you do alone at home, this quickly goes away. Uber is a strong brand, everyone associated non-taxi taxis with Uber. When Bolt came around in much of Europe, and offered the same service with better wait times (they were paying drivers more, so lots of drivers switched), people moved over to Bolt in droves.
- sebastiennight 2y ago> Branding is a very weak moat outside fashion industry and similar places - outside visible status symbols. Have you tried getting someone to switch from Chrome to Firefox? UI is basically the same, the product is more performant (faster load, easy to disable all ads, etc). But the moat is how resistant a normal user is to switch from the thing they've used without having to think about it, to the new thing. I can definitely see the ChatGPT app becoming as "sticky" as Chrome.
- jrflowers 2y ago> protesters dropped in on the after-award public talk I’m going to guess that GP does not consider random protestors to be in Sam Altman’s ‘sphere’ > The product market fit is fantastic. This is true insomuch that you define “product market fit” as “somebody mentioning it in an Indian restaurant in Berlin” > every definition of "very smart" Every definition you say? https://bsky.app/profile/edzitron.com/post/3lclo77koj22y https://bsky.app/profile/edzitron.com/post/3lclo77koj22y
- amluto 2y agoThat’s an odd way to define “smart”. The current crop of LLMs have absolutely no ability to query the raw input or output. I can look at the words I’m typing and think about them. Transformers don’t even get the raw characters, and they also don’t have any feedback from later layers into the queries generated at earlier layers. So the fact that they have any ability at all to reflect on their own input is IMO both impressive and somewhat surprising. I occasionally wonder whether one could get better results by allowing later layers to query against the K/V data from earlier layers. This might make inference, and possibly even training, of models that don’t fit in a single node’s memory rather more complicated.
- ben_w 2y ago> Every definition you say? When I was growing up, "very smart" people had a law degree or a medical degree or understood quantum physics and/or relativity (but all three would be silly), or spoke perhaps three or four languages (but thirty would be a joke). They might perhaps also be above average at games of skill like chess, or perhaps could improvise poetry in some specific style (but every style would be a bit of a Mary Sue). I may not trust ChatGPT's medical diagnosis or legal analysis, but it does pass those exams; I may not expect much novel insights about unifying QM and GR, but it seems to "know" the topics well enough for people to worry it may lead to cheating in those subjects and most others besides. And the chess score may not be the absolute best, but it's way above the average human player. And it seems entirely adequate in quite a lot of languages (though not native), and most styles of poetry (again, not what anyone would call the Nobel committee for, but way above what most people can dream of). > https://bsky.app/profile/edzitron.com/post/3lclo77koj22y https://bsky.app/profile/edzitron.com/post/3lclo77koj22y We also didn't have spellcheck running all the time when I was growing up, and when we did there was quite a lot of red, but even that misses things like "I'm applying for collage" and "what lovely wether*" which were also real mistakes made by actual humans. * spelled like that it means "castrated ram"
- 015a 2y ago> he actively empathised with the protestors. I have significant doubt that Sam is capable of empathy, period. It seems like what he's capable of is an extremely convincing caricature of it which he has practiced for many years.
- disgruntledphd2 2y agoI mean, one could say the same about basically every human (and indeed, some AI systems).
- BadHumans 2y ago> When Altman was collecting the award at Cambridge the other year, protesters dropped in on the after-award public talk/Q&A session, and he actively empathised with the protestors. I've known many CEO who will stare you right in the eye and say they hear your concerns then immediately disregard you and continue with business as usual. Being a psychopath is part of the job description.
- n_ary 2y ago> I overheard a daughter talking to her mother about ChatGPT and KI (Künstliche Intelligenz, the German for AI). Given the fact that how much ALL of our media beating that drum since GPT-3, I am hoping that my cat will start talking about it without understanding what it is. Even Draghi’s report has immense hidden lobbying dedicated to KI.
- shitloadofbooks 2y agoIn my opinion the difference is that a recent graduate knows to say “I don’t know” to questions they’re not sure on, whereas LLMs will extremely confidently and convincingly lie to your face and tell you dangerous nonsense.
- ben_w 2y agoMy experience is that intellectual humility is a variable, not a universal. Seen some students very willing to recognise their weaknesses, others who are hamstrung by their hubris. (And not just students, the worst code I've seen in my career generally came from those most certain they're right). https://phys.org/news/2023-12-teens-dont-acknowledge-fact-eager.html https://phys.org/news/2023-12-teens-dont-acknowledge-fact-ea... And yes, this is a problem with some LLMs that are trained to always have an answer rather than to acknowledge their uncertainty.
- foldr 2y ago> The product market fit is fantastic. > [...] > What's not obvious is how to monetise it. This is an interesting new use of 'product market fit'. I would have thought that in the absence of a path to monetisation, there is no market. Or we could talk about the 'market' for selling $10 bills priced at $5.
- ben_w 2y agoI phrased that badly, so that's fair. If you follow the rest of that paragraph, my meaning may be clearer: there's a market for what they do right now — the OpenAI financial reports that have been bandied around say that if they were only offering this as a paid service, not also developing new models, they'd be making a profit — but it's not obvious how this will change going forward. The first spreadsheets were easy to market: here's a solution to your problem, give us money and you can have this solution. But as time goes on, you need more than the original solution, because that's too easy to replicate. ChatGPT was easy to replicate, and has been replicated. The "needing more" to stay in the game is what I meant by the Red Queen race. What exists now is fine, it's a good fit, but the question of the long-term is still open.
- moooo99 2y ago> The product market fit is fantastic. This isn't the first time I've heard random strangers discussing it in public. People were constantly discussing crypto in public, it even regularly made it into Tagesschau. Yet it remains an (influential) niche product. People discussing a topic in public is not a good proxy for product market fit. It does not imply those people are using said product, nor does it imply people would be willing to pay for said product. > even the current models meet every definition of "very smart" I had while growing up, despite their errors. I recently tried to read a grocery store receipt into GPT4o and asked it what the KG price for the steak position was (it was listed as €/kg right next to the position name). It came up with some elaborate math, but not with the correct answer. So yeah. These models might be able to confidentially answer questions, but I found them to be (partially) incorrect more often than not.