5 ms·
Run LLaMA and Alpaca on your computer
- nickthegreek 4y agoFor some reason my install does not have the model selector at the top. Real shame, thats the feature I wanted!
- toastal 4y agoLinked library has a DCMA takedown. Host this stuff outside of US corporate control.
- sliken 4y agoOr just post a magnet link to a torrent.
- knodi123 4y agoDownloading alpaca weights actually does use a torrent now! But it seems a little.... leechy... to use a fire-and-forget CLI torrent client to grab a dozens of gigabytes file and then quit.
- capableweb 4y agoBest would be to let the seeding happen for a while, until it at least reaches 0.5 ratio or something like that. But downloading over torrent is still better than over HTTP, it'll still seed the pieces (unless the downloader turned that off fully, that'd be an asshole move) as it download the missing pieces.
- shaky-carrousel 4y agoIf the model weights are able to be copyrighted, then they are also able to be a derivative work of something copyrighted, as the model is based on copyrighted works. So either the weights are infringing copyright, which means they don't belong to Meta, or the weights can't be copyrighted. Either way, that DMCA is worthless.
- demoride 4y agoYour philosophical argument is interesting, but what the op was saying was one of the linked repos used in this repo is inaccessible due to DMCA: https://github.com/shawwn/llama-dl https://github.com/shawwn/llama-dl So while what you say may be true the DMCA seems to have worth for these orgs because they can get code removed by the host, who is uninterested in litigating, and the repo owner likely is even less capable of litigating the DMCA. Unfortunately as a tool of fear and legal gridlock DMCA has shown itself to be very useful to those with ill intent.
- halfeatenscone 4y agoI don't think this follows. Calling it a derivative work already feels like a stretch, but even granting that framing, the use is clearly transformative and therefore likely to be considered fair use in the US. I actually wrote a Wikipedia article on the intersection of copyright law and deep learning models the other day (https://en.wikipedia.org/wiki/Artificial_intelligence_and_copyright https://en.wikipedia.org/wiki/Artificial_intelligence_and_co...). I was hoping to include a section on the copyrightability of model weights, but was sadly able to find 0 coverage in reliable sources.
- nl 4y agoThis should include https://arxiv.org/abs/2303.15715 https://arxiv.org/abs/2303.15715
- kmeisthax 4y agoSo, while the poster you replied to has the wrong reasoning, they got to the right conclusion. Mostly. Let's start with a non-AI example: compilers. If I have the source code for Linux, I can compile it, but I don't own the kernel binaries I made. This is because the compiler is a purely mechanical process, not a tool of human creativity. Copyright in the US attaches to creativity from human authors. So the source code would be the creative work, not the binaries. We don't normally talk about this because ownership over the source code still flows through to the binaries. Your permission to copy that Linux binary is downstream of Linus having granted you permission to do so under the GPL. If you had instead copied, say, the NT kernel, you would be infringing the copyright on the NT kernel source code by distributing binaries of it. So now let's go to AI land. You've collected a bunch of training data and dumped it into a linear algebra blender. That's like compiling source code: the ML trainer program adds no creativity or authorship, so you haven't gained any ownership over the data. Remember: this training data is scraped off the Internet from other people's work. Fair use merely makes it non-infringing to do this, it does not mean you own the result. There are two avenues by which Meta could still get US copyright over the language model: - They could make a model with their own training data that they made, and use their ownership over the training data to get ownership over the model. - They could assert ownership over the compilation of training data. Compilation ownership is kind of weird. Basically, in the US, you can make a compilation of other people's work and own solely that. Like, say, a "Top 10 Songs I Like" playlist[0]. But even then the creativity and authorship rules still apply. These models are not being trained by having humans manually select specific works that would do well in the model. They scrape the Internet and train on everything[1]. In fact, they usually don't even use their own scrapes; they use Common Crawl, LAION-5B, and/or The Pile. Whether or not any of this is right would require someone to actually share LLaMA, get sued by Facebook, and then assert this legal theory. And hope that Facebook does not assert any other legal claims, such as misappropriation of trade secrets, which might actually stick. [0] Or in a particularly egregious example, someone copyrighting their Magic: The Gathering deck in protest of this nonsense. [1] Stable Diffusion at least uses an "aesthetics score", but AFAIK that's generated by an AI so also not copyrightable.
- seydor 4y agoI hope we will see a revival of native desktop development with AI. There is really no reason for bloated JS containers, we can ask AI to rewrite it in C++.
- suby 4y agoI hope so too. Plus, in my experience with using GPT4 to generate code, it's more effective at writing C++ than it is at writing Python. Something about annotating the types helps it with getting things right.
- hbn 4y ago> Something about annotating the types helps it with getting things right. Same rule holds true for human programmers!
- brucethemoose2 4y agoOne might call C++ bloat. Why not C, or maybe even a stricter C subset more amenable to compiler optimization.
- dist-epoch 4y agoThere will be a single app in the future: a generative AI app which will synthesize pixels on the screen depending on what you are doing.
- knodi123 4y agoThat will be absurdly wasteful of power. My calculator runs using the power of ambient light in a comfortably lit room. There are some things that an LLM is well-suited for, but c'mon. Unless electricity and hardware becomes free, I just don't see your prediction coming true.
- nexuist 4y agoI don't think they mean the LLM will literally be controlling the pixels, it will just be writing the high level code that eventually controls the pixels.
- henry_viii 4y agoPrevious discussion: https://news.ycombinator.com/item?id=35127020 https://news.ycombinator.com/item?id=35127020
- nunobrito 4y agoNice, thanks for making this available.
- Dwedit 4y agollama.cpp is simpler to get working than this is.
- mastazi 4y agoHow so? how is this: npx dalai llama 7B npx dalai serve harder than: # build this repo git clone https://github.com/ggerganov/llama.cpp cd llama.cpp make #For Windows and CMake, use the following command instead: cd <path_to_llama_folder> mkdir build cd build cmake .. cmake --build . --config Release # obtain the original LLaMA model weights and place them in ./models ls ./models 65B 30B 13B 7B tokenizer_checklist.chk tokenizer.model # install Python dependencies python3 -m pip install torch numpy sentencepiece # convert the 7B model to ggml FP16 format python3 convert-pth-to-ggml.py models/7B/ 1 # quantize the model to 4-bits (using method 2 = q4_0) ./quantize ./models/7B/ggml-model-f16.bin ./models/7B/ggml-model-q4_0.bin 2 # run the inference ./main -m ./models/7B/ggml-model-q4_0.bin -n 128
- hrpnk 4y agodalai does a lot of magic in the background. I had to run the package from source to debug the error messages. When run in on a Mac with zsh, it still runs bash under the hood to setup the python venv. Took me an hour to debug that as bash did not have poetry and pyenv configured to use <3.11 python since I never use bash. Yet, sentencepiece does not have a 3.11 wheel available, thus running dalai fails. Today, just 2d after running dalai successfully, it refuses to start and just hangs in there. Unclear why.
- MintPaw 4y agollama.cpp and alpaca.cpp both ship binary releases. Dalai does not seem to.
- vorticalbox 4y agofor one the script after cloning llama.cpp repo then calls `exit()` which then drops me out of my fish shell.
- 4y ago
- ConanRus 4y ago[dead]
- kevviiinn 4y agoHas anyone been able to get this to download the 65B model? It keeps failing for me but 30B downloads just fine
- tmaly 4y agoThe windows install says: When installing Visual Studio, make sure to check the 3 options as highlighted below: Python development Node.js development Desktop development with C++ What if you already have visual studio installed and you did not check all of these?
- muricula 4y agoIt's been a little while but iirc, you can rerun the visual studio installer and check those boxes and it will update your install.