4 ms·

There's nothing special in "AI". Open AI is just like all open source/free software: Publicly available complete training data, public weights, public trainin

There's nothing special in "AI". Open AI is just like all open source/free software: Publicly available complete training data, public weights, public training source code so that the weights can be replicated exactly, public inference code so that the weights can be used. All of this under reasonable free software licenses (i.e. FSF/OSI-approved). Conceptually, the training data should be considered part of the source code. The weights are provided for practical purposes, because they are difficult to "compile".

mnahkies 3y ago
I agree with this, but being pedantic is it actually possible to replicate weights exactly? My intuition would be that training is non-deterministic / reproducible, though you should be able to achieve equivalent results given the same inputs
- chesterk 3y ago
  It's possible to make training pipelines reproducible and deterministic using random seeds. There's support for this in PyTorch, Jax, etc. And it can be useful to do so for debugging. You can make it configurable with a flag
  - pixl97 3y ago
    How many Peta calculations before errors start accruing? Training large models involves a stupid huge amount of compute.
    - glandium 3y ago
      There's also the question of parallelization making the accruance of errors less deterministic, too.
to11mtm 3y ago
Does the ability to reproduce their weights include the 'fixes' to hide laundered GPL code?