8 ms·

Uploading files without auth layer - is asking for trouble IMHO. Change without audit trail will encourage wrong doers. But I get the idea, this is an example f

Uploading files without auth layer - is asking for trouble IMHO. Change without audit trail will encourage wrong doers. But I get the idea, this is an example for a file upload in a simple way using Curl or other tools.

ju-st 6y ago
The website is already blocked because of "Malicious Sources/Malnets" in the firewall of the company I work at.
- raverbashing 6y ago
  "This is why we can't have nice things..." sigh
southerntofu 6y ago
> Uploading files without auth layer - is asking for trouble IMHO. If you make it super user-friendly and advertise it as the next Megaupload, sure. But if you keep a small audience of good-faith users it's not asking for problems. If you can teach me to make my file upload as hacker-friendly as this service while implementing auth, i'd be glad. Here the entire point is you don't need further configuration/credentials for example to upload log/config from a server.
- the_arun 6y ago
  1. How are you blocking someone from uploading millions of files programmatically? 2. How do you know someone is from small audience of good-faith? 3. What if a file has virus and corrupt all the files on your end? If you don't need auth, there are few measures you can take on your end: 1. TTL - Make these files temporary - They will be erased after x hours. Eg. x=1 2. Throttle - Limit number of uploads from a given IP/machine or control by uploads per sec 3. May be adding a malware scanner?
  - xvector 6y ago
    4. Block IPs known for abuse. This includes VPNs, Tor exit nodes, and more.
- ericbarrett 6y ago
  This is a one-way road, though. The minute a bad actor finds out about your ungated image-hosting service, it's over, and you'll have a hell of a mess to clean up. If you're lucky it'll just be somebody trying to sell penis pills. If you're not, you'll have federal investigators knocking.
- opan 6y ago
  chunk.io is a similar thing, but with auth. https://chunk.io/ https://chunk.io/
- codegeek 6y ago
  But it is not a matter of "if". It is a matter of "when". Bad actor(s) will find out at some point if it is even remotely popular and then it's game over.
- derefr 6y ago
  This is incompatible with using curl as your client, but one “hacker-friendly way to do auth” is to use Github’s public SSH keys API. You can stand up an (SCP/SFTP-subprotocol-only) SSH server, and tell the user to log in with their GitHub username + GitHub SSH key. Then configure your SSH server to call[1] a check on GitHub’s API to map the provided username to the GitHub user’s set of public SSH keys. From there, the server treats that list exactly as if it were the user’s ~/.ssh/authorized_keys file. [1] As it happens, I wrote an OpenSSHD plugin for exactly this: https://github.com/tsutsu/github-auth3 https://github.com/tsutsu/github-auth3 Following that, you can configure PAM to continue the auth process however you like, policy-wise: let any GitHub user in; only let GitHub users in from a specific GitHub org; keep an LDAP directory of GitHub usernames such that you can attach metadata to them like “is banned” or “has used up their upload credits for the day” or “is on plan tier X”; etc. Then, to actually handle the uploads, you can 1. set up automatic local user instantiation per remote user; 2. populate /etc/skel with just the right set of limited files to allow the user to upload into one “spool” directory; 3. have an inotify-like daemon that watches for files to be closed in that directory and handles them from there (e.g. uploading them to an S3 bucket, etc.) ————— Or, alternately, you can avoid building this on top of OpenSSH, since you’re really fighting against the current by trying to virtualize everything, when OpenSSH expects to be relying on, and providing access to, a traditional POSIX environment. Instead, you can have your own SSH server daemon that provides access to a pretend environment inside the SSH-server process, and handles SCP/SFTP upload streams through a custom in-process handler, the same way a web framework handles PUT requests. I don’t know how common this is in other runtimes, but Erlang has an SSH server framework that you can use to implement exactly this. (As it happens, I’ve also written a high-level service that uses this SSH server framework to implement an alternative carrier for Erlang remote shell, where you can just SSH into the Erlang node to get a shell on it: https://github.com/tsutsu/exuvia https://github.com/tsutsu/exuvia. This app is also, AFAIK, the only public/FOSS demonstration of how to use Erlang’s SSH server library—which is kind of sad. People should play with things like this more! Make MUDs and such!)
  - mhitza 6y ago
    That's a neat idea, but I would try to simplify it a bit. Now, the details are fuzzy as I haven't set up anything like this but I know the theory behind it. You could setup a website which accepts file uploads at username:file_signature@example.com , when a request is accepted you retrieve the public ssh key for "username" and validate that the "file_signature" is valid for the file uploaded. It's a bit unintuitive to sign files with your SSH key but possible https://superuser.com/a/1616656/51984 https://superuser.com/a/1616656/51984
- pokoleo 6y ago
  Try using basic auth: curl -u username:password https:// https://
banana_giraffe 6y ago
Agreed. I use a little python script that creates a curl command to upload to S3 for cases where I don't have the AWS toolchain on a remote box. Not as easy as a single command, but at least I'm less likely to be sending files off to some random site for everyone to see.

ju-st 6y ago

The website is already blocked because of "Malicious Sources/Malnets" in the firewall of the company I work at.

raverbashing 6y ago
"This is why we can't have nice things..." sigh

southerntofu 6y ago

> Uploading files without auth layer - is asking for trouble IMHO. If you make it super user-friendly and advertise it as the next Megaupload, sure. But if you keep a small audience of good-faith users it's not asking for problems. If you can teach me to make my file upload as hacker-friendly as this service while implementing auth, i'd be glad. Here the entire point is you don't need further configuration/credentials for example to upload log/config from a server.

the_arun 6y ago
1. How are you blocking someone from uploading millions of files programmatically? 2. How do you know someone is from small audience of good-faith? 3. What if a file has virus and corrupt all the files on your end? If you don't need auth, there are few measures you can take on your end: 1. TTL - Make these files temporary - They will be erased after x hours. Eg. x=1 2. Throttle - Limit number of uploads from a given IP/machine or control by uploads per sec 3. May be adding a malware scanner?
- xvector 6y ago
  4. Block IPs known for abuse. This includes VPNs, Tor exit nodes, and more.
ericbarrett 6y ago
This is a one-way road, though. The minute a bad actor finds out about your ungated image-hosting service, it's over, and you'll have a hell of a mess to clean up. If you're lucky it'll just be somebody trying to sell penis pills. If you're not, you'll have federal investigators knocking.
opan 6y ago
chunk.io is a similar thing, but with auth. https://chunk.io/ https://chunk.io/
codegeek 6y ago
But it is not a matter of "if". It is a matter of "when". Bad actor(s) will find out at some point if it is even remotely popular and then it's game over.
derefr 6y ago
This is incompatible with using curl as your client, but one “hacker-friendly way to do auth” is to use Github’s public SSH keys API. You can stand up an (SCP/SFTP-subprotocol-only) SSH server, and tell the user to log in with their GitHub username + GitHub SSH key. Then configure your SSH server to call[1] a check on GitHub’s API to map the provided username to the GitHub user’s set of public SSH keys. From there, the server treats that list exactly as if it were the user’s ~/.ssh/authorized_keys file. [1] As it happens, I wrote an OpenSSHD plugin for exactly this: https://github.com/tsutsu/github-auth3 https://github.com/tsutsu/github-auth3 Following that, you can configure PAM to continue the auth process however you like, policy-wise: let any GitHub user in; only let GitHub users in from a specific GitHub org; keep an LDAP directory of GitHub usernames such that you can attach metadata to them like “is banned” or “has used up their upload credits for the day” or “is on plan tier X”; etc. Then, to actually handle the uploads, you can 1. set up automatic local user instantiation per remote user; 2. populate /etc/skel with just the right set of limited files to allow the user to upload into one “spool” directory; 3. have an inotify-like daemon that watches for files to be closed in that directory and handles them from there (e.g. uploading them to an S3 bucket, etc.) ————— Or, alternately, you can avoid building this on top of OpenSSH, since you’re really fighting against the current by trying to virtualize everything, when OpenSSH expects to be relying on, and providing access to, a traditional POSIX environment. Instead, you can have your own SSH server daemon that provides access to a pretend environment inside the SSH-server process, and handles SCP/SFTP upload streams through a custom in-process handler, the same way a web framework handles PUT requests. I don’t know how common this is in other runtimes, but Erlang has an SSH server framework that you can use to implement exactly this. (As it happens, I’ve also written a high-level service that uses this SSH server framework to implement an alternative carrier for Erlang remote shell, where you can just SSH into the Erlang node to get a shell on it: https://github.com/tsutsu/exuvia https://github.com/tsutsu/exuvia. This app is also, AFAIK, the only public/FOSS demonstration of how to use Erlang’s SSH server library—which is kind of sad. People should play with things like this more! Make MUDs and such!)
- mhitza 6y ago
  That's a neat idea, but I would try to simplify it a bit. Now, the details are fuzzy as I haven't set up anything like this but I know the theory behind it. You could setup a website which accepts file uploads at username:file_signature@example.com , when a request is accepted you retrieve the public ssh key for "username" and validate that the "file_signature" is valid for the file uploaded. It's a bit unintuitive to sign files with your SSH key but possible https://superuser.com/a/1616656/51984 https://superuser.com/a/1616656/51984
pokoleo 6y ago
Try using basic auth: curl -u username:password https:// https://

banana_giraffe 6y ago

Agreed. I use a little python script that creates a curl command to upload to S3 for cases where I don't have the AWS toolchain on a remote box. Not as easy as a single command, but at least I'm less likely to be sending files off to some random site for everyone to see.