10 ms·
Update on Llama adoption
- nikolayasdf123 2y agoLLAVA is pretty great
- simonw 2y agoDo you know of any good paid API providers offering LLAVA? I want to experiment with it out a bunch more without having to host it locally myself.
- nikolayasdf123 2y agonope. I am self-hosting. support is pretty good actually. llama.cpp supports it (v1.6 too; and in openai API server as well). ollama supports it. open-web-ui chat too. using it now on desktop (I am in China, so no OpenAI here) and in cloud cluster on project.
- xyc 2y agoCloudflare has it https://developers.cloudflare.com/workers-ai/models/llava-1.5-7b-hf/ https://developers.cloudflare.com/workers-ai/models/llava-1.... Locally it's actually quite easy to setup. I've made an app https://recurse.chat/ https://recurse.chat/ which supports Llava 1.6. It takes a zero-config approach so you can just start chatting and the app downloads the model for you.
- xyc 2y agoJust realized I read your blog about Llava llamafile which got me interested in local AI and made the app :) What's your reservation about running it locally?
- bfirsh 2y agohttps://replicate.com/yorickvp/llava-13b https://replicate.com/yorickvp/llava-13b :)
- nuz 2y agoWhat are you using it for? Curious if there's any interesting purposes I haven't thought of
- josefresco 2y agoIt is! Just downloaded it the other day and while far from perfect it's pretty neat. I uploaded a Gene Wilder/Charlie in the Chocolate Factory meme and it incorrectly told me that it was Johnny Depp. Close I guess! I run LLAVA and llama (among other models) using https://ollama.com https://ollama.com As a "web builder" I do think these tools will be very useful for accessibility (eventually), specifically generating descriptive alt tags for images.
- dakial1 2y ago...says the owner of it. Now seriously, by Llama being "sort of" open source, it does not seem to be something someone can fork and develop/evolve it without Meta, right? If one day Meta comes and says "we are closing Llama and evolving it in a proprietary mode from now on" would this Llama indie scene continue to exist? If this is the case wouldn't this be considered a dumping strategy by Meta, to cut revenue streams from other platforms (Gemini/OpenAI/Anthropic) and contain their growth?
- RicoElectrico 2y agoThe models can be fine-tuned which is good enough.
- mupuff1234 2y agoNot good enough to be considered open source.
- bunderbunder 2y agoRealistically the problem here might just be that the concept of open source doesn't really fit machine learning models very well, and we should stop trying to force it. Sharing the end product, but not the tools and resources used to produce it, is how open source has always worked. If I develop software for a commercial operating system using a commercial toolchain, and distribute the source code under GPL, we would call that software open source. Others who get the code don't automatically get the ability to develop it themselves, but that's kind of beside the point. I don't have the rights to publicly redistribute those tools, anyway; the only part I can put under an open source license is the part for which I have copyright. Training data for a LLM like Llama works similarly when it comes to copyright law. They don't own copyright and/or redistribution rights for all of it, so they can't make it open, even if they want to. If that seems unsatisfying, that's because it is. Unfortunately, though, I don't think the Free Software community is going to get very far by continuing to try to fight today's openness and digital sovereignty battles using tactics and doctrine that were developed in the 20th century.
- asdfksa 2y agoNice marketing that is written for investors. Let us translate: > By making our Llama models openly available we’ve seen a vibrant and diverse AI ecosystem come to life [...] They all use the same model and the same transformer algorithm. The model has an EULA, you need to apply for downloading it, the training data set and the training software are closed. > Open source promotes a more competitive ecosystem that’s good for consumers, good for companies (including Meta), and ultimately good for the world. So the "competitive" system means that everyone uses LLama and PyTorch. > In addition to Amazon Web Services (AWS) and Microsoft’s Azure, we’ve partnered with Databricks, Dell, Google Cloud, Groq, NVIDIA, IBM watsonx, Scale AI, Snowflake, and others to better help developers unlock the full potential of our models. Sounds really open.
- nightski 2y agoFar more open than the competition. I'll take it.
- koolala 2y agoDon't let a gift be a curse.
- ru552 2y ago>They all use the same model and the same transformer algorithm. The model has an EULA, you need to apply for downloading it, the training data set and the training software are closed. Everything in that sentence is false except the training data part. >So the "competitive" system means that everyone uses LLama and PyTorch. This sentence shows you don't understand the LLM landscape and it's also false. >Sounds really open Correct. They partner with practically every vendor available for inference, which, isn't even needed if you run their models locally. Meta has done a lot of wrong things over the years. How they are approaching LLMs is not one of them.
- freilanzer 2y ago> Everything in that sentence is false except the training data part. You do need to apply on Huggingface to download the model. > This sentence shows you don't understand the LLM landscape and it's also false. PyTorch definitely is the most used ML framework.
- mkesper 2y agoLlama isn't open source at all. Stop using that phrase for your product featuring even an EULA.
- ekianjo 2y agoopen source is so ambiguous its a useless expression at this stage. At least FOSS is less problematic.
- kzrdude 2y agohttps://opensource.org/osd https://opensource.org/osd
- ekianjo 2y agoso what? nobody cares about such definitions since you can see the expression is used to mean everything
- benterix 2y agoA few decades ago an organization was founded specifically to address statements such as this one. That's why some early Microsoft attempts at competing with OS had to be called "shared source", not "open source".
- deleted 2y ago[deleted]
- CamperBob2 2y agoUnfortunately they didn't do a competent job at addressing it, which would have involved trademarking the phrase. As a result, "Open Source" means whatever you, I, Meta, or anyone else wants it to mean.
- riedel 2y agoThis a hard stance if you talk just about the code (not the model weights). The llama community licence is a bit weird and probably not an OSI compliant licence, but close. Regarding weights this is different, but to me it is actually difficult to understand still now to aplly copyright law here. Having said that one nicht und erstand why certain stupid looking clauses went into the code licence. If we do not understand copyright of model weights and do not have court rulings in the use oft training data und er different copyright regimes (US and EU), I would not care too much. We are still in the Wild West.
- deleted 2y ago[deleted]
- codingwagie 2y agoProbably will get flagged, but I get so annoyed by the cynical takes on Meta and their open source strategy. Meta is the only company releasing true open source (React, pytorch, graphql) and now LLama. This company has done more for software development than any other in the last decade. And now they are burning down the competition in AI, making it accessible to all. Meta software engineering compensation strategy pushed up the high end of developer compensation by almost twice. Enough with the weird cynicism on their licensing policy.
- wilsonnb3 2y ago> Meta is the only company releasing true open source What? There are so many open source projects from huge companies these days. VSCode, .NET, typescript from MS Angular, flutter, kubernetes, go, android, chromium from Google
- SushiHippie 2y agoThe llama models use a non open source license [0]. Yes it is still better than not being able to access the weights at all, but calling these weights open source is not correct. [0] https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/main/LICENSE https://huggingface.co/meta-llama/Meta-Llama-3.1-8B/blob/mai...
- codingwagie 2y agoDude they spent billions on the model and then just open sourced it
- martindevans 2y agoNo, they spent billions on a model and released the weights, and that's fantastic! It's not not open source though.
- lrrna 2y ago[flagged]
- 2y ago
- blackeyeblitzar 2y agoHere we go again with the co opting of open source and the marketing open washing. Llama isn’t open source. Sharing weights is like sharing a compiled program. Without visibility into the training data, curation / moderation decision, the training code, etc Llama could be doing anything and we wouldn’t know. Also open source means the license used should be something standard not proprietary, without restrictions on how you can use it.
- talldayo 2y ago> Sharing weights is like sharing a compiled program. Not at all. They're only similar in the sense that both are a build artifact. > Without visibility into the training data, curation / moderation decision, the training code, etc Llama could be doing anything and we wouldn’t know. "could be doing anything" is quite the tortured phrase, there. For one, model training is not deterministic and having the full training data would not yield a byte-perfect Llama retrain. For two, the released models are not turing-complete or filled with viruses; you can open the weights yourself and confirm they're static and harmless. For three, training code exists for all 3 Llama models and the reason nobody uses them is because it's prohibitively expensive to reproduce and has zero positive potential compared to finetuning what we have already. > Also open source means the license used should be something standard not proprietary, without restrictions on how you can use it. There are very much restrictions on redistribution for nearly every single Open Source license. Permissive licensing may not mean what you think it means.
- thor-rodrigues 2y agoI think that focusing primarily on the discussion of what is or isn't open source software makes us miss an interesting point here, that Llama enables users to have a similar performance to frontier models in your own systems, without having to send data to third-party sources. My company is building an application for an university client, regarding the examination of research data written in "human language" (mostly notes and docs). Due the high confidentiality of the subjects, as often they deal with non-patented information, we couldn't risk using frontier models, as it could break the novelty of the invention, therefore losing patentability. Now with Llama3.1, we can simply run these models locally, on systems that is not even connected to the internet. LLMs are mostly good in examining massive amount of research papers and information, at least for the application we are aiming at, saving thousands of hours of tiresome (and very boring) human labour. I am trying to endorse Meta or Zuckerberg or anything like that, but at least in this aspect, I think Llama being "open-source" is a very good aspect.
- jstummbillig 2y agoTo me it's fairly interesting how relatively little money it takes meta to pose a risk to other models makers businesses, who are dependent on having to run the model after they created it (because that is how they make money) while meta does not even have to deal with the cost attached to providing inference infra, at all, to pose that risk.
- koolala 2y ago"the first frontier-level open source AI" They are never going to stop saying this or show us the actual source data. Imagine if they did... Do they even entertain the idea? Can they really not imagine Open Source AI being possible because of all the personal data they train on?
- deleted 2y ago[deleted]
- Spivak 2y agoSource code, yes. Source data, probably never unless the US government gets real cool with a lot of things real quickly.
- pj_mukh 2y agoHas anyone heard about any effect Meta has said would happen if Californias SB 1047 passes[1]? Looking forward to continued updates and releases of Llama (and SAM!) from Meta. [1] https://www.theverge.com/2024/8/28/24229068/california-sb-1047-ai-safety-bill-passed-state-assembly-governor-newsom-signature https://www.theverge.com/2024/8/28/24229068/california-sb-10...
- ksajadi 2y agoThis is a good one for everything related to SB1047 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120 https://pca.st/episode/44b41e28-5772-41c4-bcd7-5d7aa48d5120
- pama 2y agoYoshua Bengio is a very respected scientist with well deserved reputation, but this discussion is upsetting… “academia now trains much smaller models… 10^26 FLOPs is 10 to the 26 floating point operations per second.. yes.. how big is that compared to GPT-4? It is much bigger than all the existing ones…” (flops has a different meaning: there is no per second in the law; one single H100 from last year performs 1e15 FLOPs per second; llama3.1 was close to the 1e26 limit this year, and the total training FLOPS of other models are not published; research could change once compute is even cheaper but state laws move at glacial speeds…). It is disheartening so see the damage capacity in the hands of a couple of paranoic people who perhaps read the wrong scifi and had lots of power to influence others. If California passes this law, in a few years the world economy will be very different.
- EasyMark 2y agoI think companies will simply quit doing business in California. They are killing the golden goose with all these regulations, just like they poison their own real estate markets allowing NIMBYs to dictate regulations that keep housing prices high and allow petty criminals to run rampant while cities like San Francisco continue to diminish and suffer