8 ms·
A path to O1 open source
- jeanlucas 2y agoLooks like China will lead the next generation of open source tech.
- curious_cat_163 2y agoAnd be a lot more efficient (e.g. DeepSeek-V3) [1] about it... [1] https://arxiv.org/pdf/2412.19437v1 https://arxiv.org/pdf/2412.19437v1
- enos_feedler 2y agoYep, including HarmonyOS NEXT.
- basilgohar 2y agoIt is not open source. "HarmonyOS NEXT (Chinese: 鸿蒙星河版; pinyin: Hóngméng Xīnghébǎn) is a proprietary distributed operating system and a major iteration of HarmonyOS, developed by Huawei to support only HarmonyOS native apps." [0] https://en.wikipedia.org/wiki/HarmonyOS_NEXT https://en.wikipedia.org/wiki/HarmonyOS_NEXT
- mdaniel 2y agoHaven't you heard? Open Source now means whatever businesses want it to mean so long as it gets eyeballs or free labor :-(
- skissane 2y agoHarmonyOS NEXT is based on an open source core, OpenHarmony [0], with proprietary additions. So, not hugely dissimilar from iOS (lots of bits of which are open source, most significantly the core of its XNU kernel) and Android (considering that the proprietary Google Mobile Services is de facto a mandatory component) [0] https://en.wikipedia.org/wiki/OpenHarmony https://en.wikipedia.org/wiki/OpenHarmony
- lern_too_spel 2y agoThe Taco Bell kiosk and your exercise bike don't need the "mandatory" GMS.
- enos_feedler 2y agoI am speculating it will become open source. or at least supplant Android's role in the global smartphone ecosystem.
- elorant 2y agoHow is China training models without access to cutting edge GPUs?
- kristjansson 2y agoPretty easily, it turns out?
- pixelesque 2y agoThey're being more efficient about it by the looks of things, rather than brute-forcing things... https://x.com/karpathy/status/1872362712958906460 https://x.com/karpathy/status/1872362712958906460
- ryao 2y agoThey have access to cutting edge GPUs via rentals: https://www.msn.com/en-us/money/markets/bytedance-plans-to-sidestep-us-sanctions-by-renting-nvidia-gpus-in-the-cloud-report-says-it-has-set-aside-7-billion-budget/ar-AA1wLF5A https://www.msn.com/en-us/money/markets/bytedance-plans-to-s...
- rajamaka 2y agoThe same way drug users access illegal drugs.
- arthurcolle 2y agovery carefully
- HarHarVeryFunny 2y ago- using non-cutting edge GPUs (just more of them) - creating more efficient models such as MoE based DeepSeek - getting their hands on cutting edge GPUs all the same I think it was Dylan Patel (from semianalysis) on Dwarkesh that mentioned one scam is for a Chinese source to arrange for a SOTA NVidia cluster to be bought/installed in some non-embargoed country, then dismantled and shipped to China.
- jpcookie 2y ago[flagged]
- fuddle 2y agoIt's ironic that they are attempting to open source a model from "OpenAI".
- deleted 2y ago[deleted]
- mikkom 2y ago"Open"ai really should change their name
- deleted 2y ago[deleted]
- FanaHOVA 2y agoI wish HN would stop devolving into Reddit. This comment is the same boring "joke" that has been repeated 100 times on every platform, and keeps being posted for karma. It adds nothing to the conversation.
- veggieroll 2y ago> Please don't post comments saying that HN is turning into Reddit. It's a semi-noob illusion, as old as the hills. [0] But also, if you want people to stop mocking "Open" AI, then maybe they should stop being such a mockable caricature of themselves. [0]: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
- FanaHOVA 2y agoIf you joined 8 months ago it might be hard to recognize. I've been on HN for more than a decade and the quality of discourse has drastically lowered in quality especially in the last 3-4 years. This is a problem with the broader web, not just HN. Tech / startups is now a mainstream topic that attracts a lot of people who are not really in the weeds and are just able to write surface level comments. Regarding the name, open is just a word. Apple doesn't sell apples. The company never promised to open source every model, only to make them accessible to the public, so you're arguing semantics that lead to no improvement in the technical conversation.
- newyankee 2y agoI guess now the strategy of OpenAI would be to keep the small edge all the time, integrate it with businesses fast & possibly kickstart new businesses by supporting them and trying to be synonymous with the best in AI (may be with Deepmind). I cannot think of any other moat, unless somehow they have a lot of proprietary and useful data (like in company) that others cannot replicate
- visarga 2y agoBut that edge will become more and more expensive, while the competition will cover more and more of the task space and make it less profitable for OAI
- jbegley 2y agolmao at “learining” being misspelled in the first sentence
- saikia81 2y agoand later on again
- mdaniel 2y agoIn these times, how else does one expect to advurtise that theeir text was not geeenerated by an LLM?
- Oarch 2y agoExcessive profanity could be a fun way to prove human authors!
- yencabulator 2y agoCUSSTCHA Curse-Using Social Scoring Turing test to tell Computers and Humans Apart
- mdaniel 2y agoOh, fuckin' A, I love this shit!
- jcims 2y agoThat's hilarious, and sorry, I can't help myself. Curse-Using Social Scoring Turing Assessment to Tell Computers and Humans Apart CUSSATCHA
- BlueTemplar 2y agoBless you.
- asddubs 2y agoor just pepper in some erotic pictures of copyrighted characters
- xoxosc 2y agoIs it me or the first link on the site http://ability.openai http://ability.openai is broken? Is openai a tld now?
- mdaniel 2y agowhois says no, and it seems there's a close one of https://tld-list.com/tld/open https://tld-list.com/tld/open that's owned by (strangely enough) American Express Travel. It could be yet another tpyo of foo.open.ai which would work today, no global TLD required (I mean, they have damn near unlimited money, just buy out whoever owns it now)
- rrr_oh_man 2y agoThat seems, ironically, to be an AI proofreader mistake. The correct URL is https://cdn.openai.com/o1-system-card-20241205.pdf https://cdn.openai.com/o1-system-card-20241205.pdf , at least according to https://arxiv.org/html/2412.14135v1 https://arxiv.org/html/2412.14135v1 (which contains typos that OP's submissions doesn’t).
- benatkin 2y agoThey should correct the name to closedai before they get the TLD, since there's an administrative cost to these TLDs. All of them are here though. No .openai. https://www.iana.org/domains/root/db https://www.iana.org/domains/root/db .open is among the worst. :/ > Purchasing a .open domain name isn't available to the general public. This particular extension is owned by American Express and currently isn't for sale or open to registration, limiting its use to only selected entities associated with American Express. It's primarily designed to serve the interests of the corporation and its customers' claims or needs. https://tld-list.com/tld/open https://tld-list.com/tld/open - Who is able to buy a .open domain name?
- zb3 2y ago.open is as open as OpenAI so I guess it's a good fit :)
- 0xml 2y agoThis is a feature of arxiv that automatically converts text looks like a link into "this http url". The submitter missed the space after the "." in "...strong reasoning ability. OpenAI has claimed...". https://info.arxiv.org/help/prep.html#abstract-required https://info.arxiv.org/help/prep.html#abstract-required
- chefandy 2y agoIt seems to me that for the most capable and useful models, openness almost exclusively benefits businesses, or maybe academic organizations with money for serious hardware. I know what I can run on my 4090 at home but the results pale in comparison to the commercial services. I see why people consider these matters important from a theoretical standpoint but from a practical standpoint it doesn’t seem particularly consequential. I self-host a few FOSS server applications that are primarily sold as SaaS subscriptions, and folks are often very critical of those businesses benefitting from the “open source” label because they’re often seemingly deliberately difficult to self-host. This seems to be an order of magnitude less open than that. Is there some use case for people with reasonable hardware that I’m just not aware of?
- lukeschlather 2y agoI look at this as being for the reasonable hardware of the future. This is starting to look like actual AGI, and I don't think actual AGI is going to run on a 4090. But an H100 starts to sound like a mass-market product even with the $50k price tag if it actually can run an AGI.
- ripped_britches 2y agoYou can get cheaper hosting of open weights from a commercial provider than you can closed weights from the same company. So even if you’re not hosting yourself, openness is a major factor for price competitiveness.
- Oras 2y agoFirst line in the abstract > OpenAI o1 represents a significant milestone in Artificial Inteiligence, Inteiligence Safe to say OpenAI has nothing to worry about
- yencabulator 2y agoIt's amazingly bad. > the main techinique behinds o1 is the reinforcement learining.
- deleted 2y ago[deleted]
- arresin 2y agoI prefer (maybe intentional) spelling mistakes to ai generated drone and verbosity.
- d-lisp 2y agoI cannot help with the fact that it sounds like a bad strategy to claim this is a good reason "not to worry" about something. If I were "OpenAI" I would rather read the content than evaluate the form of such articles to know if I should "worry" or not. It seems like the most Inteiligent method.
- HarHarVeryFunny 2y agoSounds about right. How long until we see DeepSeek-o1 ?
- elashri 2y agoThere is DeepSeek-R1 where they have R1-lite preview version available for testing on their chat website.
- HarHarVeryFunny 2y agoHave they released anything on how it works other than "test time compute"? I wonder how similar it is to what's being proposed on this roadmap, that sounds close to what I imagine OpenAI are doing. I guess we'll see when they open source it.
- mig39 2y agoIs this a joke? Why are there so many spelling mistakes? "Inteiligence" "challanging" “learining” What is going on?
- melvinmelih 2y ago> Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Bo Wang, Shimin Li, Yunhua Zhou, Qipeng Guo, Xuanjing Huang, Xipeng Qiu I'm guessing English isn't their first language.
- mig39 2y ago"Hey, ChatGPT, please correct any typos or spelling mistakes in this abstract."
- mdaniel 2y agoModern times: a bunch of matrices of floating point numbers can chat with a human but also: no one knows what the red squiggle marks mean in any textarea anymore because they no longer use them to write anything
- greenchair 2y agoand too lazy to run spell check
- rrr_oh_man 2y agoI don’t think anyone who upvoted this has read more than one sentence of this paper.
- ricardobeat 2y agoIt’s interesting even if not true or correct. You could also choose to enrich the discussion by elaborating on why you think this is worthless instead.
- nickvec 2y agoI have a hard time giving worth to a paper whose first sentence fails to spell intelligence correctly.
- pizza 2y agoIn a mathematical conversation, someone suggested to Grothendieck that they should consider a particular prime number. “You mean an actual number?” Grothendieck asked. The other person replied, yes, an actual prime number. Grothendieck suggested, “All right, take 57.” Might I suggest Postel's Law?
- deleted 2y ago[deleted]
- melvinmelih 2y agoThis paper has been available for a few weeks, and I wrote an article [1] exploring how to apply its inner workings to the design of multi-agent systems. If you can design "reasoning" at the model level, you can also design "reasoning" in larger, more complex systems using the same principles. https://melvintercan.com/p/lessons-from-reasoning-designing https://melvintercan.com/p/lessons-from-reasoning-designing
- punnerud 2y agoNot one word in the article about using embeddings for the reasoning, a bit strange?
- mmaunder 2y agoNo. O1 doesn’t do RAG.
- mrayycombi 2y agoFrom the first few paragraphs it doesn't pass the sniff test for me. "Now AI has made everything more complex!" "AI is embedded in everything we do"... Sounds like marketing gibberish and obfuscation, combined with self promotion. That's just my read at first sniff.
- xvector 2y agoThis is absolutely a worthless fluff paper
- behnamoh 2y agoflagged it. more people should flag this kinda stuff.
- pizza 2y ago"Doesn't pass my sniff test" is not the purpose of the flag button. Furthermore, it passes my personal sniff test: hundreds of people upvoting it while the top comment is saying it's worthless. Usually the real alpha is in the comments under such things.
- mrayycombi 2y agoI didn't flag it, I flamed it. Seems like it stunk enough for others to flag it. Lol.
- gone35 2y agoI disagree. I found the review useful.
- mtkd 2y agoThere was a Berman video on it earlier today that summarises it https://www.youtube.com/watch?v=-haWhgmUheA https://www.youtube.com/watch?v=-haWhgmUheA Detail starts about 7mins in
- halayli 2y ago> has claimed that the main techinique behinds o1 is the reinforcement learining. Typos in the first sentence of the paper doesn't give confidence that I am about to read something worthwhile.
- DiscourseFan 2y agoAt least you know a person wrote it
- lgessler 2y agoI think this is both a harmful and irrational attitude. Why focus on some trivial mechanical errors and disparage the authors for it instead of the thing that is much more important, i.e., the substance of the work? And in dismissing work for such trivial reasons, you risk ignoring things you might have otherwise found interesting. In an ideal world would second-language speakers of English proofread assiduously? Of course, yes. But time is finite, and in cases like this, so long as a threshold of comprehensibility is cleared, I always give the benefit of the doubt to the authors and surmise that they spent their limited resources focusing on what's more important. (I'd have a much different opinion if this were marketing copy instead of a research paper, of course.)
- HarHarVeryFunny 2y agoWell, it not exactly a research paper, more an overview of the problem and suggested techniques, but it'd still be interesting to hear some criticism based on the content rather than the (admittedly odd) omission to run it through a spell checker. I do wonder why it was written in English, apparently targeting a western audience. Two of the authors are from "Shanghai AI Labs" rather than students, so one might hope it had at least been proofread and passed some sort of muster.
- dTal 2y ago>in dismissing work for such trivial reasons, you risk ignoring things you might have otherwise found interesting Not dismissing work for trivially avoidable mistakes risks wasting your precious, limited lifespan investing effort into nonsense. These signals are useful and important. If they couldn't be bothered to proofread, what else couldn't they be bothered to do? >spent their limited resources focusing on what's more important Showing that you give a crap is important, and it takes seconds to run through a spell checker.
- kcorbitt 2y agoLots of folks working on open-source reasoning models trained with reinforcement learning right now. The best one atm appears to be Alibaba's 32B-parameter QwQ: https://qwenlm.github.io/blog/qwq-32b-preview/ https://qwenlm.github.io/blog/qwq-32b-preview/ I also recently wrote a blog explaining how reinforcement fine-tuning works, which is likely at least part of the pipeline used to train o1: https://openpipe.ai/blog/openai-rft https://openpipe.ai/blog/openai-rft
- HappMacDonald 2y agoI don't know if I would call it "the best one" when it has "How many r in strawberry" as one of its example questions and when tried it arrives at the answer "two".
- currymj 2y agomany people are dismissing this paper because it has errors in spelling and grammar. this is a terrible heuristic for evaluating AI papers. If you use it, you will miss a lot of good work by very strong researchers with below-average English writing skills. I have not read this paper carefully so claim nothing one way or the other about its quality. It superficially seems like a pleasant and timely survey although a little flag-planty.