9 ms·

Would you mind sharing this script? I am thinking of investing in the 2K workstation, but this sounds like a much more efficient way to go.

lettergram 10y ago
I tried to get it released before, but was shut down by the "open source office". So I can't give you the exact script. However, h2o has a script that launches a cluster, that's very similar: https://github.com/ledell/h2oEnsemble-benchmarks/blob/master/ec2/h2o-cluster-launch-instances.py https://github.com/ledell/h2oEnsemble-benchmarks/blob/master... You can also do stuff like following in python to execute some code: os.system("scp script_demo.py " + sys.argv[1] + ":/home/ubuntu/") os.system("ssh " + sys.argv[1] + " sudo chmod 777 /home/ubuntu/script_demo.py") os.system("ssh " + sys.argv[1] + " './script_demo.py'") os.system("scp " + sys.argv[1] + ":/home/ubuntu/output/* ./output/") Then it will execute, after it executes - shut it down.
- viraptor 10y ago
  Even if it may not matter in this case specifically, this is a terrible example. "sudo chmod 777" in public code is basically "I don't know what I'm doing, but go on, do the same thing yourself" :( You don't need sudo, because it's your file. You need just "chmod 755", not "777". And you don't need chmod in the first place - just run "python script_demo.py".
  - oh_sigh 10y ago
    Yes. On the other hand, there isn't a strong overlap with machine learning experts and security experts(I'm sure there will be in coming years though, as security experts start using NNs to detect anomalous/dangerous behavior).
    - viraptor 10y ago
      I know what you mean, but I disagree with "security experts". You don't need to be a security expert, just know the basics of how privileges work on your system. And if people don't, we just need to keep calling it out, because otherwise code like this ends up in production one day, where it actually matters. Simpler bash equivalent for the record: scp script_demo.py "$1" ssh "$1" python script_demo.py scp "$1":output/* ./output/
      - anantzoid 10y ago
        Exactly. If you're a ML person, you must be working with Linux in most cases. And in such case, you need to be familiar with the types of permissions, cause that's relevant even during the installation of various libraries, doing ssh etc.
  - danieltillett 10y ago
    These things come about because people learn the simple way to solve all permission problems is just do everything as su (or root) and chmod everything to 777. The problem is it does solve all problems (well sweep them under the rug) - I guess when the only tool you have is a hammer everything looks like a nail.
  - empath75 10y ago
    If you're launching an AWS instance that doesn't have any ports open and is going to terminate in a few hours, who cares?
    - viraptor 10y ago
      As I said: "Even if it may not matter in this case". But what about the next time? (poster obviously doesn't understand the code) What about people who read this comment and follow the advice because it works? (happens all the time with SO) What about when your script becomes the standard deployment method for the company? (happens everywhere) What about people who don't know about file privileges either and blindly copy what they see here? There's lots of things we do that you could say "who cares?" about. Single letter variables. Comments. Const-correctness. Unnecessary N^2 algorithms. Usually it turns out that either you or someone you work with cares a few months/years afterwards. So just learn to do it correctly the first time. Especially if the correct way takes less time than the "magic fix".
  - lettergram 10y ago
    Very true it doesn't need a 777. It's also not my script, just copied and pasted some code that fit what I was doing in mine. Basically, it follows the same format, specific implementation varys. Also, you are being rather pedantic here. If people can figure out how to take what I copied & pasted and turn it into a script they can probably know chmod 777 isn't great. That being said, and why I think this is ridiculous, is that you are assuming this matters. Going into the weeds here, to play along: The script is immediately being ran, on temporary and very recently launched ec2 instance, probably with the use of a pem and that presumably can even be part of AWS security group that only allows your IP, and is shut down following it's execution. I can't picture this being a security vulnerability at all. Calling it a terrible example is relative - I wrote this copy and pasted on a cell phone trying to help someone. Honestly, didn't even see the "sudo chmod 777", just pasted away.
- ClassyJacket 10y ago
  >but was shut down by the "open source office" Can you expand on that? What happened?
  - lettergram 10y ago
    Basically, I couldn't get authorization to publish internal code. In this case, it was a combination of who was going to maintain it and ensuring nothing internal was leaked.

lettergram 10y ago

I tried to get it released before, but was shut down by the "open source office". So I can't give you the exact script. However, h2o has a script that launches a cluster, that's very similar: https://github.com/ledell/h2oEnsemble-benchmarks/blob/master/ec2/h2o-cluster-launch-instances.py https://github.com/ledell/h2oEnsemble-benchmarks/blob/master... You can also do stuff like following in python to execute some code: os.system("scp script_demo.py " + sys.argv[1] + ":/home/ubuntu/") os.system("ssh " + sys.argv[1] + " sudo chmod 777 /home/ubuntu/script_demo.py") os.system("ssh " + sys.argv[1] + " './script_demo.py'") os.system("scp " + sys.argv[1] + ":/home/ubuntu/output/* ./output/") Then it will execute, after it executes - shut it down.

viraptor 10y ago
Even if it may not matter in this case specifically, this is a terrible example. "sudo chmod 777" in public code is basically "I don't know what I'm doing, but go on, do the same thing yourself" :( You don't need sudo, because it's your file. You need just "chmod 755", not "777". And you don't need chmod in the first place - just run "python script_demo.py".
- oh_sigh 10y ago
  Yes. On the other hand, there isn't a strong overlap with machine learning experts and security experts(I'm sure there will be in coming years though, as security experts start using NNs to detect anomalous/dangerous behavior).
  - viraptor 10y ago
    I know what you mean, but I disagree with "security experts". You don't need to be a security expert, just know the basics of how privileges work on your system. And if people don't, we just need to keep calling it out, because otherwise code like this ends up in production one day, where it actually matters. Simpler bash equivalent for the record: scp script_demo.py "$1" ssh "$1" python script_demo.py scp "$1":output/* ./output/
    - anantzoid 10y ago
      Exactly. If you're a ML person, you must be working with Linux in most cases. And in such case, you need to be familiar with the types of permissions, cause that's relevant even during the installation of various libraries, doing ssh etc.
- danieltillett 10y ago
  These things come about because people learn the simple way to solve all permission problems is just do everything as su (or root) and chmod everything to 777. The problem is it does solve all problems (well sweep them under the rug) - I guess when the only tool you have is a hammer everything looks like a nail.
- empath75 10y ago
  If you're launching an AWS instance that doesn't have any ports open and is going to terminate in a few hours, who cares?
  - viraptor 10y ago
    As I said: "Even if it may not matter in this case". But what about the next time? (poster obviously doesn't understand the code) What about people who read this comment and follow the advice because it works? (happens all the time with SO) What about when your script becomes the standard deployment method for the company? (happens everywhere) What about people who don't know about file privileges either and blindly copy what they see here? There's lots of things we do that you could say "who cares?" about. Single letter variables. Comments. Const-correctness. Unnecessary N^2 algorithms. Usually it turns out that either you or someone you work with cares a few months/years afterwards. So just learn to do it correctly the first time. Especially if the correct way takes less time than the "magic fix".
- lettergram 10y ago
  Very true it doesn't need a 777. It's also not my script, just copied and pasted some code that fit what I was doing in mine. Basically, it follows the same format, specific implementation varys. Also, you are being rather pedantic here. If people can figure out how to take what I copied & pasted and turn it into a script they can probably know chmod 777 isn't great. That being said, and why I think this is ridiculous, is that you are assuming this matters. Going into the weeds here, to play along: The script is immediately being ran, on temporary and very recently launched ec2 instance, probably with the use of a pem and that presumably can even be part of AWS security group that only allows your IP, and is shut down following it's execution. I can't picture this being a security vulnerability at all. Calling it a terrible example is relative - I wrote this copy and pasted on a cell phone trying to help someone. Honestly, didn't even see the "sudo chmod 777", just pasted away.
ClassyJacket 10y ago
>but was shut down by the "open source office" Can you expand on that? What happened?
- lettergram 10y ago
  Basically, I couldn't get authorization to publish internal code. In this case, it was a combination of who was going to maintain it and ensuring nothing internal was leaked.