4 ms·

Here is a tiny excerpt try to get dvc to work just so I could get the training weights for deployment ... remember I don't develop much w Python... $ dvc p

Here is a tiny excerpt try to get dvc to work just so I could get the training weights for deployment ... remember I don't develop much w Python... $ dvc pull Command 'dvc' not found, but can be installed with: sudo snap install dvc $ sudo snap install dvc error: This revision of snap "dvc" was published using classic confinement and thus may perform arbitrary system changes outside of the security sandbox that snaps are usually confined to, which may put your system at risk. If you understand and want to proceed repeat the command including --classic. ok I get dvc installed somehow -- don't remember. Time to get the weights... $ python3 -m dvc pull ERROR: unexpected error - Forbidden: An error occurred (403) when calling the HeadObject operation: Forbidden Having any troubles? Hit us up at https://dvc.org/support, we are always happy to help! Finally I just have my colleague manually copy the weights. This kind of thing went for hours.

pdntspa 3y ago
Researchers are notorious for writing bad code What even is dvc edit: also- i'd avoid snap and just use your regular package manager.
- waynecochran 3y ago
  I think dvc is like git for large binary files. You need someway to manage your NN weights -- what are other methods?
  - pdntspa 3y ago
    git lfs is what everyone is using, HF in particular
efiop 3y ago
Hey, DVC maintainer here. Thanks for giving DVC a try! There are a few ways to install dvc, see https://dvc.org/doc/install/linux https://dvc.org/doc/install/linux With snap, you need to use `--classic` flag, as noted in https://dvc.org/doc/install/linux#install-with-snap https://dvc.org/doc/install/linux#install-with-snap Unfortunately that's just how snap works for us there :( Regarding the pull error, it simply looks like you don't have some credentials set up. See https://dvc.org/doc/user-guide/data-management/remote-storage/amazon-s3#amazon-s3-and-compatible-servers https://dvc.org/doc/user-guide/data-management/remote-storag... Still, the error could be better, so that's on us. Feel free to ping us in discord (see invite link in https://dvc.org/support https://dvc.org/support). I'm @ruslan there. We'll be happy to help.
- waynecochran 3y ago
  Thanks… i know my colleague uses it a lot. I generally use his models and don’t do much ML development yet. At some point I need to properly learn all of this. It seems ML tools are only for developers not for those who simply want to deploy and use the resulting NN.

pdntspa 3y ago

Researchers are notorious for writing bad code What even is dvc edit: also- i'd avoid snap and just use your regular package manager.

waynecochran 3y ago
I think dvc is like git for large binary files. You need someway to manage your NN weights -- what are other methods?
- pdntspa 3y ago
  git lfs is what everyone is using, HF in particular

efiop 3y ago

Hey, DVC maintainer here. Thanks for giving DVC a try! There are a few ways to install dvc, see https://dvc.org/doc/install/linux https://dvc.org/doc/install/linux With snap, you need to use `--classic` flag, as noted in https://dvc.org/doc/install/linux#install-with-snap https://dvc.org/doc/install/linux#install-with-snap Unfortunately that's just how snap works for us there :( Regarding the pull error, it simply looks like you don't have some credentials set up. See https://dvc.org/doc/user-guide/data-management/remote-storage/amazon-s3#amazon-s3-and-compatible-servers https://dvc.org/doc/user-guide/data-management/remote-storag... Still, the error could be better, so that's on us. Feel free to ping us in discord (see invite link in https://dvc.org/support https://dvc.org/support). I'm @ruslan there. We'll be happy to help.

waynecochran 3y ago
Thanks… i know my colleague uses it a lot. I generally use his models and don’t do much ML development yet. At some point I need to properly learn all of this. It seems ML tools are only for developers not for those who simply want to deploy and use the resulting NN.