9 ms·
Grok has gotten to the top of one benchmark: https://x.com/lmarena_ai/status/1891706264800936307 https://x.com/lmarena_ai/status/1891706264800936307 It's been
by rendang 2y ago
Grok has gotten to the top of one benchmark:
https://x.com/lmarena_ai/status/1891706264800936307 https://x.com/lmarena_ai/status/1891706264800936307
It's been said before but it is great news for consumers that there's so much competition in the LLM space. If it's hard for any one player to get daylight between them & the 2nd best alternative, hopefully that means one monopolistic firm isn't going to be sucking up all the value created by these things
- aprilthird2021 2y agoI think it's already clear that these are going to be commoditized and the free / open source versions will be good enough to capture enough of the value that the remaining players will not be Facebook-level monopolies on the space
- rendang 2y agoApparently it isn't clear to the investors valuing OpenAI at >300B. Possibly they're betting that the ecosystem & integrations around their models will generate a certain amount of lock-in or otherwise make the difference in a close-to-even field
- hmottestad 2y agoFunnily enough a lot of the open source world has landed on an API that is basically a copy of OpenAI. So if you develop against OpenAI it’s almost a slot in solution to switch to an open source solution.
- pzo 2y agoand on top of that you have solutions like openrouter.ai where you can route inference easily with a combobox
- riffraff 2y agoInvestors thought someone renting office space was going to revolutionize the world and valued their company 50B.
- jeswin 2y agoI don't think it's automatically a bad idea. Offices require a lot of support, networking, security, maintenance, certifications etc. There are efficiency gains in scaling. In addition, Wework is useful for companies which hire employees in different cities.
- snowwrestler 2y agoLots of things are good ideas but investing is about price vs value. Good ideas can be overpriced as easily as bad ideas.
- apples_oranges 2y agoWell, now their job is to keep up the illusion until they have cashed out or offloaded the investment to somebody else.
- aprilthird2021 2y agoThis reminds me of a comedy sketch where a guy is interviewing for a job at a startup, finally gets to the last round and meets the founder, and he tells him the whole thing is an illusion for investors
- fragmede 2y agothe bet is if they can produce ai that can replace a level of generic office worker. a bot that you can add to slack and give tasks to do.
- bigbones 2y agothe IP rights holders have yet to bare their teeth. I don't think the outcome you suggest is clear at all, in fact I think if anything entirely the opposite is the most probable outcome. I've lost count of the number of technology epochs that at the time were either silently or explicitly dependent on ignoring the warez aspects while being blinded by the possibilities, Internet video, music and film all went through this phase. GPTs are just a new medium, and by the end of it royalties will in all likelihood still end up being paid to roughly the same set of folk as before I quite like the idea of a future where the AI job holocaust largely never happened because license costs ate up most of the innovation benefit. It's just the kind of regressive greed that keeps the world ticking along and wouldn't be surprised if we ended up with something very close to this
- beeflet 2y agoGood historical comparison, but I doubt it this time because there is plausible deniability that a model wasn't trained on a given piece of data. Also, the pool of public domain data is always increasing, so the AI will eventually win in any case, even if we have to wait 100 years
- bigbones 2y agoAs I recall it, there was a time when copyright infringement on YouTube was so prolific that the rightsholders essentially forced creation of the first watermarking system that worked at massive scale. I do wonder if any corners of research are currently studying the attribution problem with the specific lens of licensing as its motivation
- beeflet 2y agoYeah that was the old Viacom vs Youtube days. Here is a great video if you have half an hour to spare: https://www.youtube.com/watch?v=qV2h_KGno9w https://www.youtube.com/watch?v=qV2h_KGno9w . Pretty funny court case where it turns out viacom was violating their OWN copyright... set a massive precedent. But one thing this reminds me of is the idea of a "trap street", something mapmakers used to do was put in false locations on their maps to prove that other mapmakers were copying them: https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street . I figure you could do something similarly adversarial with AI to pollute the public training data on the internet. IDK like adversarial attacks on image classifiers https://www.youtube.com/watch?v=AOZw1tgD8dA https://www.youtube.com/watch?v=AOZw1tgD8dA . With an LLM you could try to make them into a manchurian candidate.
- bobxmax 2y agoThe vast majority of people couldn't care less about open source
- jonlucc 2y agoIf you're paying $200/month for something I can do with open source software and $10/month of compute, why wouldn't I offer you the service for $100/month? And then someone offer it for $50? Not everyone has to know about, understand, or use open source solutions for it to open the field.
- bobxmax 2y agoIf you can do a $200/mo service for $10/mo, the closed source will reduce their prices to $15/mo and beat you This is just a weird dichotomy you're introducing. Open source will introduce price pressure as any competition will - that doesn't mean you won't have a monopoly.
- dauhak 2y agoIf you have virtually no pricing power and have to drop your $200/mo to $15/mo that's a big deal if your $300bn valuation is implying that not happening, which is what OP's point is about Idk what you mean by saying this doesn't preclude a monopoly - having your pricing power eroded by competition is kinda one of the key features of what a monopolistic market isn't
- bobxmax 2y agoNot at all. Monopolies don't imply an anti-rigid price curve. In fact, monopolies almost never have that. A monopoly means a company has enough leverage to corner and disproportionately own the market. This is entirely possible (and usually the case) even with significant pricing pressure.
- gopher_space 2y agoI think you're both missing a bigger picture. How many of these services can now be replicated in-house by a single developer? Which part of the service actually costs money once that dev deconstructs the process? Feels like I won't be paying for anything that isn't real-time. And that any time delay I can introduce in my process will come with massive savings. Picture hiding the loading of loot info behind a treasure chest opening animation in a game, except that time difference means you can pull all the work in-house. Openrouter.ai seems like a step in the right direction but I'd want to do all their calculations myself as well as factor in local/existing gear in a way they don't.
- thefourthchime 2y agoExactly. I use GPT4o for nearly everything, and occasionally, I'll need o1. For 95% of what I do, it's already good enough.
- ddxv 2y agoYep, seeing Grok come out I'm just so glad there are free alternatives that aren't behind paywalls.
- qingcharles 2y agoI've spent the last hour testing it and I'm blown away. And this is coming from a very hardcore user of OpenAI/Claude products on a daily basis. It passed every goofy test I have for writing articles which involves trying to surface arcane obscure details. (it certainly means however they are scraping the Web they are doing a good job here) It made the database code I wrote over the last week with o3/o1/GPT4o/Claude3.5 look like a joke. It fills me with rage over who owns this thing. Even if people tank Tesla's car business and run Twitter into the ground, I think our new Galactic Edgelord is going to win his first trillion on xAI and Teslabots anyway. btw: it tried to charge me $40/mo for this thing: https://imgur.com/a/QXslgBo https://imgur.com/a/QXslgBo
- palata 2y ago[flagged]
- RobinL 2y agoApologies for possibly stupid question but where can you use it right now? Just on 'direct chat' on https://lmarena.ai/ https://lmarena.ai/ or is there a better alternative? Or do you have early access?
- qingcharles 2y agoI was using it on grok.com, logged in via a Twitter account. But I notice it just got added to the Grok tab on Twitter a moment ago. Also, the "Deep Search" button was not available when it first went live, so I'm retesting everything again with this feature enabled, which I assume is a reasoning version of the same model.
- giancarlostoro 2y agoOne neat feature is you can use Grok on any tweet, its helped me find context to obscure tweets many times over, very quickly.
- int_19h 2y agoYou need an X Premium Plus subscription on Twitter.
- Hamuko 2y ago>It's been said before but it is great news for consumers that there's so much competition in the LLM space. Is it? Because it seems like a bunch of megacorps pirating every single copyrighted work available in digital format, spending an enormous amount of electricity (that is probably not 100% clean) to churn through them, and the end result we have a bunch of parrots that may or may not produce accurate results so that spammers can more effectively fill the Internet with crap.
- thephyber 2y ago[flagged]
- croes 2y agoWho cares about benchmarks? These things still cost me time because of hallucinations.
- cheema33 2y agoHumans hallucinate as well. Benchmarks count.
- croes 2y agoBut with less power consumption.
- golergka 2y agoI'm willing to bet $100 that a human consumes at least 10x times energy than a latest LLama (picking the open source model so that it can be easily verified) to produce two pages of text. All of this "AI is destroying the environment and consuming too much power" is about total consumption, both training and inference. Inference itself is cheap and green.
- croes 2y agoYeah, too bad it’s about quality not quantity.
- idiotsecant 2y agoThose goalposts just keep sliding
- brulard 2y agoWhat percentage of humans produce high-quality text?
- gkbrk 2y agoLess than half, and that's being generous.
- crocowhile 2y agoIt's not good news when this competition comes at cost of a gigantic over inflated bubble, in which all the big players keep on sucking billions from investors without even having a business model. This hype will burst sooner than later and will trigger yet another global recession. This is untenable.
- qgin 2y agoThe premise is that this ultimately replaces all intellectual and physical labor for the rest of time. It’s possible it becomes commoditized as soon as it exists, but in terms of investment dollars it’s either worth as much as you can spend or nothing at all.
- pjc50 2y ago> ultimately replaces all intellectual and physical labor for the rest of time Sounds incredibly valuable, but in reality collapses into Butlerian Jihad fairly quickly when you have 90% unemployment. Edit: if the claims are true, then this will be far more destabilizing than social media. What do elections mean when the AI-guided political parties are putting out AI press releases for the AI press summaries, which voters have read out to them through their AI? What happens when >50% of the voters ask the AI who they should vote for? You end up with an AI dictatorship where the levers of discontent are fake.
- conartist6 2y agoYeah at some point it seems inevitable that if machines do all the work that creates real "value" and people have no comparable value, in a very practical sense we will all be slaves to machines
- thrance 2y agoLike in Dune, we won't be slaves to the machines, but to the people owning the machines.
- LightBug1 2y ago[flagged]
- wyclif 2y ago[flagged]
- theshackleford 2y ago[flagged]
- LightBug1 2y ago[flagged]
- Refusing23 2y agobenchmarks dont show the quality or 'correctness' of the response though.
- deleted 2y ago[deleted]
- SecretDreams 2y agoProbably bad news for the vendors, though. I genuinely struggle to see how most of these LLM companies are going to monetize and profit off their efforts with LLMs already in commodity territory. Government contracts can only flow for so long?
- k__ 2y agoYes, I couldn't have imagined that in the end, the AI wrappers are where the money is.
- tpm 2y agoWhat if the money isn't there either? What if this AI thing lowers costs of everything it touches without generating meaningful financial returns itself?
- ineedasername 2y agoLowering costs is pretty valuable. People will pay for that. Everyone will pay for that. It may be that margins go razor thin but outside of running your own instance locally (which is increasingly viable for mid quality & requirements on modest HW) people will pay. I’m not surrounded by early adopter types at all abd there’s still a small but growing chunk paying $20/mo right now.
- tpm 2y ago> It may be that margins go razor thin That's what I mean. One example are PV panels, they are making energy production cheaper, so their producers should be good investment right? No they go bust all the time because prices are falling and margins are thin even if the volume grows. Of course the economies of scale here are different but still.
- cuuupid 2y agoGovernment contracts are so big a few of them can sustain a F500 company; for AI, many CDAO contracts are 50-500MM$. If they do a big SI project with it, could be 1-2B$. Money is also guaranteed over 5 years and if the program doesn't get shuttered, the contract will renew at that point (or go to recompete). That being said it's my understanding that these companies don't have many huge contracts at all -- you can audit this in like 10 minutes on FPDS. Companies need a LOT of capital, time, and expertise to break into the industry and just compliance audit timelines are 1-4 years right now, so this could definitely change in the next couple years.