3 ms·

seriously? meta llm's are on huggingface and you can download and run them on your machine along with the weights

theptip 2y ago
Open source != open weights != open training set. Of course, GP is ignoring that Meta is far more open than its direct competitors (MSFT, GOOG, OpenAI).
- throwthrowuknow 2y ago
  I agree that open source or just “open” is the wrong term here although it’s hard to apply it even to a project where the training data and everything else are shared. “Free” or “libre” might be closer since the end product is free to use and modify. It’s still fuzzy though, are weights code or are they an API? Oddly enough they seem to be the embodiment of the interchangeability of code and data perhaps due to neural networks being closer to mathematics than software normally is.
  - fragmede 2y ago
    it's not fuzzy. weights are binary blobs. we know what binaries are, source code gets compiled into binaries. photoshop is a binary. firmware are binary blobs. that llama3.ggml outputs text when using a wrapper program vs photoshop launches a program directly doesn't make anything fuzzy.
    - throwthrowuknow 2y ago
      Not really, those weights are the same ones that are there during training getting modified by backprop. You can take them and continue training the model in order to produce different behaviour. You can also modify them directly like with LoRA or steering. You can also make bigger modifications like expanding the layers or removing nodes. It’s not a frozen binary, those numbers aren’t obfuscated or optimized into some special purpose form that prevents or makes modification difficult.
      - fragmede 2y ago
        And I can edit binaries with binedit. Reverse engineering is a lost art, but reversing binaries with Ida pro and patching binaries is a time honored skill. It's a shame it's usually used to patch out license checks but there have been some truely impressive reverse engineering on video game ROMs for which the source is not available so we're left with modifying the binary. or as we're looking at it, the model.
        fragmede 2y ago
        https://github.com/WerWolv/ImHex https://github.com/WerWolv/ImHex (https://news.ycombinator.com/item?id=40828493 https://news.ycombinator.com/item?id=40828493) is a hex editor specially made for people that do exactly that kind of binary editing.
        throwthrowuknow 2y ago
        But you’re not reversing anything with weights. They aren’t altered at all. There’s no secret human readable code, just training/test data and hyperparameters. Even if they did give you access to the exact data set they used there’s not much chance you could do anything with it without millions of dollars worth of compute time and even then they’ve already trained the model on that data so the best you could do is start from scratch and see if you could get a better result in which case your time is better spent on building your own data set and RLHF process. Like what value do you think you’re going to get anyways?
PunchTornado 2y ago
just putting weights on hug is not enough for a model to be open. it is like sharing the binaries of a software. they should put their entire code on github and show how it was trained, architecture and everything else. also, the most important thing about a model is not it's architecture, but the dataset it was trained on. that is the "it" in a model.
- jncfhnb 2y ago
  The preferred form for modifying an ML model is the weights.
- stale2002 2y ago
  I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks. Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down, or what? Because anyone who actually supports open "anything" would understand this, instead of pedantically arguing about the definition of open source and by suggesting things that both aren't possible and would make the release of these models much less likely.
  - throwthrowuknow 2y ago
    Perhaps a good solution would be to form a trust responsible for storing and distributing training data. It could be held in common by all contributors and allow anyone to access high quality curated data. It would need legal allowances and exemption from copyright which would be difficult to work out but perhaps some mechanism could be created where copyright holders could derive benefit from not seeking to have their works removed, not necessarily monetary. This would at least get data collection out of the shadows and allow companies and researchers to attest that they are using the data only for training and not distributing it otherwise. That could go a long way toward clarifying issues of whether or not a book or article was in a dataset, how often it appeared, how many epochs it was used in, whether it only appeared as referenced or quoted by other text, etc. With that kind of disclosure it would be possible to write tests to ensure that the models won’t simply regurgitate the material without attribution or unprompted and publish those tests to assuage fears.

Open source != open weights != open training set. Of course, GP is ignoring that Meta is far more open than its direct competitors (MSFT, GOOG, OpenAI).

throwthrowuknow 2y ago
I agree that open source or just “open” is the wrong term here although it’s hard to apply it even to a project where the training data and everything else are shared. “Free” or “libre” might be closer since the end product is free to use and modify. It’s still fuzzy though, are weights code or are they an API? Oddly enough they seem to be the embodiment of the interchangeability of code and data perhaps due to neural networks being closer to mathematics than software normally is.
- fragmede 2y ago
  it's not fuzzy. weights are binary blobs. we know what binaries are, source code gets compiled into binaries. photoshop is a binary. firmware are binary blobs. that llama3.ggml outputs text when using a wrapper program vs photoshop launches a program directly doesn't make anything fuzzy.
  - throwthrowuknow 2y ago
    Not really, those weights are the same ones that are there during training getting modified by backprop. You can take them and continue training the model in order to produce different behaviour. You can also modify them directly like with LoRA or steering. You can also make bigger modifications like expanding the layers or removing nodes. It’s not a frozen binary, those numbers aren’t obfuscated or optimized into some special purpose form that prevents or makes modification difficult.
    - fragmede 2y ago
      And I can edit binaries with binedit. Reverse engineering is a lost art, but reversing binaries with Ida pro and patching binaries is a time honored skill. It's a shame it's usually used to patch out license checks but there have been some truely impressive reverse engineering on video game ROMs for which the source is not available so we're left with modifying the binary. or as we're looking at it, the model.
      - fragmede 2y ago
        https://github.com/WerWolv/ImHex https://github.com/WerWolv/ImHex (https://news.ycombinator.com/item?id=40828493 https://news.ycombinator.com/item?id=40828493) is a hex editor specially made for people that do exactly that kind of binary editing.
      - throwthrowuknow 2y ago
        But you’re not reversing anything with weights. They aren’t altered at all. There’s no secret human readable code, just training/test data and hyperparameters. Even if they did give you access to the exact data set they used there’s not much chance you could do anything with it without millions of dollars worth of compute time and even then they’ve already trained the model on that data so the best you could do is start from scratch and see if you could get a better result in which case your time is better spent on building your own data set and RLHF process. Like what value do you think you’re going to get anyways?

PunchTornado 2y ago

just putting weights on hug is not enough for a model to be open. it is like sharing the binaries of a software. they should put their entire code on github and show how it was trained, architecture and everything else. also, the most important thing about a model is not it's architecture, but the dataset it was trained on. that is the "it" in a model.

jncfhnb 2y ago
The preferred form for modifying an ML model is the weights.
stale2002 2y ago
I am not sure why people keep pretending like this is possible or keep asking it to happen. You know that distributing all the data would be illegal copyright infringement, right? Also, its not like they could distribute the (petabytes?) worth of data to every rando who asks. Do you just not want open weight models to exist, and you are using a fake call for it to be more open, so as to get this stuff all shut down, or what? Because anyone who actually supports open "anything" would understand this, instead of pedantically arguing about the definition of open source and by suggesting things that both aren't possible and would make the release of these models much less likely.
- throwthrowuknow 2y ago
  Perhaps a good solution would be to form a trust responsible for storing and distributing training data. It could be held in common by all contributors and allow anyone to access high quality curated data. It would need legal allowances and exemption from copyright which would be difficult to work out but perhaps some mechanism could be created where copyright holders could derive benefit from not seeking to have their works removed, not necessarily monetary. This would at least get data collection out of the shadows and allow companies and researchers to attest that they are using the data only for training and not distributing it otherwise. That could go a long way toward clarifying issues of whether or not a book or article was in a dataset, how often it appeared, how many epochs it was used in, whether it only appeared as referenced or quoted by other text, etc. With that kind of disclosure it would be possible to write tests to ensure that the models won’t simply regurgitate the material without attribution or unprompted and publish those tests to assuage fears.