9 ms·
Yi: Open Foundation Models by 01.AI
- FinNerd 3y ago[flagged]
- kleton 3y agohttps://huggingface.co/01-ai/Yi-34B https://huggingface.co/01-ai/Yi-34B
- rafram 3y agoYi models are available for download [1], so no, this isn’t a wrapper around GPT-4. [1]: https://huggingface.co/01-ai/Yi-34B https://huggingface.co/01-ai/Yi-34B
- FinNerd 3y agoIn theory couldn’t you download the application but then it just makes API calls to OpenAI? I am trying to use HuggingFace and suspect I will get similar answers to GPT-4. Can someone who knows how to download it ask for example which country Justin Trudeau is the president of?
- rafram 3y agoI’m sorry, but you really need to read up on the fundamentals of what a model is, how weights work, and so on before making comments like this. No, the model weights are not secretly making calls to the OpenAI API.
- d3m0t3p 3y agoI really like the fact that he is saying nonsense and his about is :”C-suite exec at fintech” I don’t doubt that hahaha
- endofreach 3y agoI don't think this is an appropriate comment on HN. He was asking a question. Not fully grasping a technical topic (that most people don't really understand), shouldn't be mocked.
- zeograd 3y agoIt is not an application, it's a model. It claims to be llama-compatible, so you can reuse the same ecosystem, like ollama for inference. It will not run random code on your behalf and call OpenAI API like you fear.
- deleted 3y ago[deleted]
- spencerchubb 3y agoAn AI model takes a specific input and gives a specific output. In the case of a language model, it takes in text and outputs text. They can not just execute arbitary code When you download from huggingface, you are downloading the parameters of the model. The parameters are basically just a bunch of numbers, and they had to find the parameters by training the model
- maccam912 3y agoSince you haven't had a response to the second part, here it is: PreviewJSON Justin Trudeau is not the president of any country. He is, however, the Prime Minister of Canada, which he has been since November 4, 2015. The prime minister is the head of government in Canada's federal parliamentary democracy and constitutional monarchy.
- maccam912 3y agoPreview Justin Trudeau is not the president of any country. He is, however, the Prime Minister of Canada, which he has been since November 4, 2015. The prime minister is the head of government in Canada's federal parliamentary democracy and constitutional monarchy.
- Takennickname 3y agoDid you read anything other than the title? (Even though the title itself gives it away)
- helsinkiandrew 3y agoThe Github repository, gives a better introduction/howto: https://github.com/01-ai/yi https://github.com/01-ai/yi > Yi-34B-Chat model landed in second place (following GPT-4 Turbo), outperforming other LLMs (such as GPT-4, Mixtral, Claude) on the AlpacaEval Leaderboard (based on data available up to January 2024). > Yi-34B model ranked first among all existing open-source models (such as Falcon-180B, Llama-70B, Claude) in both English and Chinese on various benchmarks, including Hugging Face Open LLM Leaderboard (pre-trained) and C-Eval (based on data available up to November 2023).
- WhitneyLand 3y agoIt’s been ~1 year since gpt-4 was released. It’s hard to guess how long before any flavor of an “open” model will consensus match what was released in 2023 let alone potentially exceed it. A big part of the race seems like it will depend on how high gpt-5 can raise the bar. If it’s only incrementally things may converge quickly.
- c0n5pir4cy 3y agoThe Yi models were actually released back in early November 2023. So there isn't as big a gap in time as it seems. I'm not sure why there is such a big gap between the release of the models and the publication of the paper. EDIT: Okay, this appears to be a new set of models with the same name, based on the same models from November but now with multimodal capabilities.
- oersted 3y agoLooking at the leaderboard shows a clearer picture: https://tatsu-lab.github.io/alpaca_eval/ https://tatsu-lab.github.io/alpaca_eval/ - GPT-4-Turbo: 50.00% - Snorkel (current 2nd, Mistral 7B fine-tune): 34.86% - Yi 34B Chat (current 6th): 29.66% - GPT-4: 23.58% Thoughts: - Just saying that it came 2nd is quite misleading, the difference in score is significant. - Not sure what's up with this benchmark, I've never seen GPT-4-Turbo vs GPT-4 performing so differently. - The Snorkel model is impressive with just 7B parameters. The Yi authors claim that their success is based on good training data cleaning. This seems to be key at least for this benchmark. Snorkel has also always been all about that, using programmatic methods to generate lots of quality training data.
- jes5199 3y ago01, like from the Animatrix ?
- bearjaws 3y agoSeeing models like this work so well gives me hope that mobile first LLMs for things like better voice to text and typing prediction will not just 'work' in 2-3 years but actually not kill your battery too.
- m3kw9 3y agoIt it kills battery it won’t be part of OS, and if it is on iOS it won’t be allowed to be in the AppStore, or it will be gated by API/hardware gating. For Andoird, they’ll just allow it and your batter will last 30min after a few questions
- coffeebeqn 3y agoIf it’s fast enough to be useful it would also not physically be able to use that much power. Your phone CPU and GPU have a maximum W they can pull at any one time and if this runs for a few seconds then the maximum it can use is that. If it maxes out all cores and memory for 30 minutes then it won’t really work for anything
- evilduck 3y agoMLC Chat is already on the App Store and allowed to be used. I haven't used Yi with it, but a quantized Mistral or Llama runs quite well on an iPhone 15. See https://llm.mlc.ai https://llm.mlc.ai. "Apple GPT" is also rumored to be coming too. It is processor and therefore battery intensive but it already won't kill your battery inside of 30 minutes. Obviously it will be worse for resource usage than an app if it's always kept running by some OS level process and set as the processing layer for every trivial thing but it seems like cheaper input handling could decide to promote some input up to being evaluated by an LLM or not.
- barronli 3y agoiOS already has app with LLM running locally on iPhone: https://apps.apple.com/gb/app/mlc-chat/id6448482937 https://apps.apple.com/gb/app/mlc-chat/id6448482937 I've also tried a few Android LLM apps, all running more than 30min. Current LLM models are not running constantly on the phones to drain your battery. They just run when responding to a prompt. It by no means consumes more battery than a heavy game.
- orra 3y agoThe repo source code is Apache 2.0 licensed, but the weights are not. Just in case anybody else is excited then misled by their tagline "Building the Next Generation of Open-Source and Bilingual LLMs".
- est31 3y agoMore reading on the weight license: https://news.ycombinator.com/item?id=38159862 https://news.ycombinator.com/item?id=38159862
- mmastrac 3y agoThe model license, excerpts: https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMEN... 1) Your use of the Yi Series Models must comply with the Laws and Regulations as well as applicable legal requirements of other countries/regions, and respect social ethics and moral standards, including but not limited to, not using the Yi Series Models for purposes prohibited by Laws and Regulations as well as applicable legal requirements of other countries/regions, such as harming national security, promoting terrorism, extremism, inciting ethnic or racial hatred, discrimination, violence, or pornography, and spreading false harmful information. 2) You shall not, for military or unlawful purposes or in ways not allowed by Laws and Regulations as well as applicable legal requirements of other countries/regions, a) use, copy or Distribute the Yi Series Models, or b) create complete or partial Derivatives of the Yi Series Models. “Laws and Regulations” refers to the laws and administrative regulations of the mainland of the People's Republic of China (for the purposes of this Agreement only, excluding Hong Kong, Macau, and Taiwan).
- heroprotagonist 3y agoNot really very open. Though it makes me wonder what the model might say about Tiananmen Square, Uyghurs, reeducation camps, or any other thing you're not really supposed to talk about in China. Is there a benchmark for bias in model outputs? No doubt China has one, somewhere, except it's not skewed towards prevention.
- echelon 3y ago
- jacobn 3y ago“we attribute the performance of Yi models primarily to its data quality resulting from our data-engineering efforts” Data work is rarely sexy, but (almost) always useful. Did they release the corpus?
- gwern 3y agoThey did not, in part because it would reveal the data-filtering routines (particularly the political censorship - Chinese LLM papers sometimes mention the ban list but never reveal it), and also in part because it might reveal things they'd rather keep secret. For example, Bytedance has already been caught using the OA API to generate data for their models because they are having such a hard time catching up to OA - and evading bans for doing that, and also instructing employees on how to lie & cover it up: https://www.theverge.com/2023/12/15/24003151/bytedance-china-openai-microsoft-competitor-llm https://www.theverge.com/2023/12/15/24003151/bytedance-china... Do you think that a small Chinese startup like 01.AI, which by their own admission had to "bet the farm" to buy enough GPUs to train the Yi models at all https://www.bloomberg.com/news/articles/2023-11-05/kai-fu-lee-s-open-source-01-ai-bests-llama-2-according-to-hugging-face https://www.bloomberg.com/news/articles/2023-11-05/kai-fu-le... , and which were completely silent about cloning the American LLaMA architecture until people analyzed the released checkpoints and noticed it looked awfully familiar, is going to be above such tactics...? In this economic/geopolitical context? Especially when everyone seems to be doing it, not just Bytedance?* (01.AI claims that, the architecture aside, they didn't simply further train LLaMA models but trained from scratch. You can decide for yourself how much you are willing to believe this.) I wouldn't bet a lot of money on it, and that's why I don't expect to see any large comprehensive data releases from 01.AI for the Yi models. * This is one of my theories for why so many disparate models by so many different groups all seem to weirdly converge on the same failure modes like 'write a non-rhyming poem', and why GPT-3.5, and then GPT-4, seemed to be oddly difficult to surpass, as if there were some magnetic force which made reaching near 3.5/4 quality easy for 'independent' models, but then surpassing somehow difficult. Everyone is lying or mistaken about 3.5/4 data getting into their corpus, and the sugar-rush of imitation learning fools you into thinking you're making a lot of progress, even when your overall approach sucks. (As Andrej Karpathy notes, neural nets want to work, and so even if you have serious bugs in your code, they will still work pretty well - and simply permanently fall short of their true potential. Cautionary recent example: https://twitter.com/karpathy/status/1765473722985771335 https://twitter.com/karpathy/status/1765473722985771335 )
- deleted 3y ago[deleted]
- dannyw 3y agoCan play around with the model here: https://replicate.com/01-ai/yi-34b-chat https://replicate.com/01-ai/yi-34b-chat Very slow, but this is unquantized and there's probably a lot of demand.
- mg 3y agoHmm.. it fails for my favorite test prompt: https://www.gnod.com/search/ai#q=Two%20cars%20have%20a%20100%20mile%20race.%20Car%20A%20drives%2010%20miles%20per%20hour.%20Car%20B%20drives%205%20miles%20per%20hour%2C%20but%20gets%20a%2010%20hour%20headstart.%20Who%20wins%3F https://www.gnod.com/search/ai#q=Two%20cars%20have%20a%20100... I gave it 3 tries and each time, Yi picked one of the cars as the winner. I've been watching for many months now, how LLMs got better and better at solving it. Many still struggle with it, but the top ones nowadays mostly get it right.
- appplication 3y agoOn one hand, I don’t really understand why anyone would expect an LLM to solve logic puzzles. The only way it can do so is not through reasoning, but by having been trained on a structurally similar puzzle. On the other hand, it does feel fun that the top ones appear to solve it, and I understand why it feels cool to have a computer that appears to be capable of solving these puzzles. But really, I think this is just specificity in training. There is no theoretical or empirical basis for LLMs having any reasoning capability. The only reason it can solve it is because side the creators of these top models specifically trained the models on problems like this to give the appearance of intelligence.
- mg 3y agoThere might be no reasoning in a single pass which outputs a single token. But in the loop where the output of the LLM repeatedly gets fed back into its input, reasoning is clearly happening: The LLMs lay out how to go about figuring out the answer, do a series of calculation steps and then come up with an answer. If you add "Please answer in just one short sentence." to the prompt, even the top ones get it wrong.
- visarga 3y agoReasoning is also an iterative process. Besides scaling in response length, the model can also get multiple feedbacks from outside to correct itself.
- 3y ago
- theptip 3y ago“01.ai” is not a very auspicious name; 01 was the first AI nation, that eventually waged war with humanity and then enslaved them in The Matrix.
- riku_iki 3y ago> that eventually waged war with humanity I think humanity waged war on 01
- ben_w 3y agoOne thing I assumed when watching the Animatrix, but never had confirmed, was that the name "01" was chosen because it sounds a bit like "Zion".
- acjohnson55 3y agoI, for one, welcome our digital overlords.
- d-z-m 3y agothere's also a new Yi model, Yi-9B[0]. [0]: https://huggingface.co/01-ai/Yi-9B https://huggingface.co/01-ai/Yi-9B
- GaggiX 3y agoYi-34B is the LLM used by LLaVA-1.6 (also known as LLaVA-NeXT) and it's by far the best open source large multimodal models, demo: https://llava.hliu.cc/ https://llava.hliu.cc/
- gyre 3y agoPotentially interesting on the alignment front: In my experience the yi-6b model running on ollama is more likely to refuse politically sensitive queries (relating to Tiananmen Square, Peng Shuai’s disappearance, etc) when asked in Chinese, and more likely to provide information when asked in English. I wonder if this difference falls out naturally from available training data, is a deliberate internationalization choice, or is just noise from the queries I happened to run.
- Havoc 3y agoCould also be both. Training data organically creating the difference but with an additional layer of specific alignment on top too
- mattstir 3y agoI noticed similar behaviour in an older model (Skywork 13B) a few months back. When asked in Chinese, it would politely say that nothing of note occurred when responding to queries about Tiananmen Square, etc. In English, it would usually respond truthfully. It was deliberate in the case of Skywork, based on their model card (https://huggingface.co/Skywork/Skywork-13B-base https://huggingface.co/Skywork/Skywork-13B-base): > We have developed a data cleaning pipeline with great care to effectively clean and filter low-quality data and eliminate harmful information from text data. I'd imagine it's likely similar for Yi.
- BoorishBears 3y agoHuge jump to go from that line in the model card to it being intentional from the model's creators. China censors those events. They pre-trained with a specific focus on Chinese text, and integrated more native Chinese text than most models do. Doesn't require any additional filtering on their behalf to have the model reflect that, and if anything the fact that they're mentioned in english implies the opposite of your hypothesis. If they were going to filter Tiananmen Square, the lift to filter it in English would not be any higher.
- arijun 3y agoI wonder if you could use the multilingual capabilities to workaround it's own censorship? I.e. what would happen if you asked it to translate the query to English, asked it in English, and then asked it to translate back to Chinese.
- zone411 3y agoYi 34B Chat has not done well on my new NYT Connections benchmark and it's only in the 22nd place on the LMSYS Elo-based leaderboard (151 Elo below GPT 4 Turbo). It's doing better in Chinese. When it comes to models with open-sourced weights, Qwen 72B is clearly stronger.
- Yenrabbit 3y agoOoh I also use connections as a benchmark! It tends to favour things with 'chain of thought' style reasoning in the training mix somewhere since directly producing the answer is hard. Do you have public code you could share?
- nbbaier 3y agoI'd also love to see how you're doing this benchmarking!
- gpjanik 3y agoI understand that all these new models are an attempt to catch up with GPT-4, but frankly speaking, in the current shape and form, they're almost entirely useless. I frantically tried anything available on Groq to improve performance of my GPT-4 based chatbot - it's incomparably bad - and the more of them I see, the more I believe OpenAI has fundamentally no competition at all at the moment. No exception with the above, also pretty bad (IMHO worse than GPT-3.5).
- yumraj 3y agoGiven that this is a Chinese model, I’m genuinely curious if researchers have been evaluating risk that these models could be used for soft propaganda or similar purpose? As others have reported, English and Chinese queries return different replies on topics that are not kosher in China. What’s the risk that such models could be used for nefarious purposes by providing propaganda/biased/incorrect/… responses that on a cursory glance seem factual.
- ithkuil 3y agoAt the very least models will exhibit the bias present in the underlying training text and on top of that there will be a bias imposed by those wanting to correct the unwanted bias present in the underlying training text, possibly swinging the pendulum too far in the other side. I have the feeling you're asking something more specific, something more of a direct interference coming from politics and not just the natural "point of view" about various topics that are present in the chinese training corpora that is understandably different from western corpora. Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet?
- yumraj 3y ago> I have the feeling you're asking something more specific... > Do you have anything specific in mind about something that you expect the Chinese government to feed as propaganda that is not already widely being sculpted into the chinese text corpora available on the internet? I don't have anything specific, and it doesn't have to be different from "chinese text corpora available on the internet", it's just that these models can become yet another channel of distribution for the "chinese text corpora available on the internet" especially if they are unknowingly/naively picked up and used as the foundation by others to build their offerings.
- maxglute 3y agoPRC will definitely weaponize this for mass foreign propaganda, which up until now PRC has been thoroughly deficient in, despite all the reees of 50cents on western net. The reality pre-LLM is PRC propaganda on western social media platforms has been very limited in scale for the simple reason that they are not wasting valuable English fluency to shit post on western platforms enmass. Low 100s-1000s of accounts, most of which target diasphora in spambot efforts, frequently in Chinese. Now that LLM has made it cheap to spam passable English/foreign languages, I'd expect increased volumes of PRC propaganda on western social media where anonymous posting is asymmetrically easier. But then again, they don't need a PRC LLM for that, plenthy of US bad posting on western platforms from international audiences and US herself.
- hdhdhsjsbdh 3y agoIn the past year or so, arxiv has become more of an advertising platform than a scientific resource. The title of this “paper” makes that quite clear: their company URL is right there!