4 ms·

We allow uploads with pre-singed URLs at my work too. We added a few more constraints. First, we only allow clients to upload files with extensions that are rea

by cebert 2y ago

We allow uploads with pre-singed URLs at my work too. We added a few more constraints. First, we only allow clients to upload files with extensions that are reasonable for our apps. Files uploaded to S3 are quarantined with tags until we validate the binary contents appears to match the signature for a given extension/mime-type with a Lambda. Second, we use ClamAV to scan the uploaded files. Once these checks have completed, we generate a thumbnail with a Lambda and then make the file available to users. I’m honestly surprise this isn’t a value-added capability offered by AWS S3 because it’s such a common need and undifferentiated work.

dividuum 2y ago
Is using ClamAv because of some compliance checklist that forces you to include a virus scanner in there? Otherwise I don’t see how adding a ton of additional complexity with a proven track record of CVEs would add any security at all.
- cebert 2y ago
  No compliance reason other than adding it in couldn’t hurt.
  - robertlagrant 2y ago
    How does > a ton of additional complexity with a proven track record of CVEs become: > adding it in couldn’t hurt ?
    - matja 2y ago
      By running it in an ephemeral container with no network access for each file you want to check?
Kinrany 2y ago
If S3 did this, what would that look like?
thebeardedone 2y ago
It's been a while but I would benchmark how ClamAV fares. At $(day job) we implemented scanning for file uploads and after our security team tested they noticed that it detected things fairly poorly (60% was on the high side) for very well known malicious content iirc. We ultimately scrapped it, if anyone else has any better experience I'd love to hear it.
GordonS 2y ago
Virus scanning user uploaded files always seemed like a very sensible thing to me. I've used ClamAV before in web apps, and it was fairly easy to get working - but it has a few big flaws: it's very slow, it uses a huge amount of memory, and detection rates are... honestly, pretty terrible. I've used Verisys Antivirus API for 2 recently projects, and it works great, a huge improvement over self-hosted ClamAV. There are a couple of other virus scan APIs available, including Scanii[1] which I think has been mentioned on HN before, and I Trend Micro has something too - but their pricing is secret, so I have to assume it's expensive. [0] https://www.ionxsolutions.com/products/verisys-virus-api https://www.ionxsolutions.com/products/verisys-virus-api [1] https://scanii.com https://scanii.com
- byyll 2y ago
  How are those better than ClamAV? I am not saying they aren't but you just said the detection rates of one is terrible compared to the rest.
  - GordonS 2y ago
    Things got a lot better for ClamAV after the Cisco purchase, but detection rates remain attrocious. There are some commercial signatures available, which would certainly help, but it still doesn't come close to the detection rates of commercial scanning engines. ClamAV needs lots of memory to work - we needed at least 8GB in one project I worked on (where we also used commercial signatures), 4GB was just about enough for another. The more signatures you have, the more memory it uses. ClamAV chews through a lot of CPU, and it's so very slow. For example, scanning a 25MB MSI file from my test corpus takes 65 seconds - and that's on a fairly beefy machine! I've used both of the services linked above, and both seem excellent - detection rates and performance are far superior to ClamAV, and they Just Work(TM).
arethuza 2y ago
Azure Storage seems to have an integrated AV option: https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-storage-malware-scan https://learn.microsoft.com/en-us/azure/defender-for-cloud/d... Mind you, last time I designed something using this we just used ClamAV - which is pretty easy to develop against even if it is a slight pain to manage on an ongoing basis.
LinuxBender 2y ago
I am not a proper developer so I don't have the technical details on this, but another thing I have seen done is to buy access to an API on VirusTotal [1] and submit file hashes to be compared against their database of about 70 or so virus scanners. Not foolproof, as malware can be recompiled and hidden from VT until the actual file is submitted for sandbox analysis which you probably dont want to do with potentially sensitive data so it really only catches known-knowns but apparently some companies find it useful. I utilize it on a windows machine using ProcessExplorer that can automatically submit hashes but the API volume is limited for free / anonymous access. [1] - https://docs.virustotal.com/reference/overview https://docs.virustotal.com/reference/overview
- toomuchtodo 2y ago
  CISA is offering a malware analysis service now, requires a Login.gov account to use and the user be a US citizen. Unsure how it compares to VirusTotal, it’s in our backlog to spike a poc in our IR workflow. https://www.cisa.gov/resources-tools/services/malware-next-generation-analysis https://www.cisa.gov/resources-tools/services/malware-next-g...
  - yard2010 2y ago
    I have to say this user agreement sounds like TikTok privacy policy, but my oh my! The government does its job. RESPECT.
- rpigab 2y ago
  If I was a malware distributor, I'd just throw a couple random bytes at the end each time I upload a malicious file, it'd still work and have a different checksum/hash.
  - ecaradec 2y ago
    Antivirus are really stupid tools, but not that stupid. I said that from a time where I had to work around tools flagged by antivirus. Among stupid things they do are flagging a part of an executable, some nsis plugin flagged the whole package as virus as soon as you included them. I think they probably hash files by chunks, if you have too many bad chunks then you are a virus. A few bytes at the end doesn't change that.
matusgallik008 2y ago
Thanks for the insights! I had a look at ClamAV - never heard/worked with it. This was something I was looking for. Do you also handle large files e.g. videos with lambda? Don't you struggle with Lambda limits then?
- cebert 2y ago
  Great question. We use spot EC2 instances for ClamAV scans. For thumbnails, we still use Lambdas but use AWS Media Convert for Video files to select a frame for the thumbnail.

by cebert 2y ago

dividuum 2y ago

Is using ClamAv because of some compliance checklist that forces you to include a virus scanner in there? Otherwise I don’t see how adding a ton of additional complexity with a proven track record of CVEs would add any security at all.

cebert 2y ago
No compliance reason other than adding it in couldn’t hurt.
- robertlagrant 2y ago
  How does > a ton of additional complexity with a proven track record of CVEs become: > adding it in couldn’t hurt ?
  - matja 2y ago
    By running it in an ephemeral container with no network access for each file you want to check?

Kinrany 2y ago

If S3 did this, what would that look like?

thebeardedone 2y ago

It's been a while but I would benchmark how ClamAV fares. At $(day job) we implemented scanning for file uploads and after our security team tested they noticed that it detected things fairly poorly (60% was on the high side) for very well known malicious content iirc. We ultimately scrapped it, if anyone else has any better experience I'd love to hear it.

GordonS 2y ago

Virus scanning user uploaded files always seemed like a very sensible thing to me. I've used ClamAV before in web apps, and it was fairly easy to get working - but it has a few big flaws: it's very slow, it uses a huge amount of memory, and detection rates are... honestly, pretty terrible. I've used Verisys Antivirus API for 2 recently projects, and it works great, a huge improvement over self-hosted ClamAV. There are a couple of other virus scan APIs available, including Scanii[1] which I think has been mentioned on HN before, and I Trend Micro has something too - but their pricing is secret, so I have to assume it's expensive. [0] https://www.ionxsolutions.com/products/verisys-virus-api https://www.ionxsolutions.com/products/verisys-virus-api [1] https://scanii.com https://scanii.com

byyll 2y ago
How are those better than ClamAV? I am not saying they aren't but you just said the detection rates of one is terrible compared to the rest.
- GordonS 2y ago
  Things got a lot better for ClamAV after the Cisco purchase, but detection rates remain attrocious. There are some commercial signatures available, which would certainly help, but it still doesn't come close to the detection rates of commercial scanning engines. ClamAV needs lots of memory to work - we needed at least 8GB in one project I worked on (where we also used commercial signatures), 4GB was just about enough for another. The more signatures you have, the more memory it uses. ClamAV chews through a lot of CPU, and it's so very slow. For example, scanning a 25MB MSI file from my test corpus takes 65 seconds - and that's on a fairly beefy machine! I've used both of the services linked above, and both seem excellent - detection rates and performance are far superior to ClamAV, and they Just Work(TM).

arethuza 2y ago

Azure Storage seems to have an integrated AV option: https://learn.microsoft.com/en-us/azure/defender-for-cloud/defender-for-storage-malware-scan https://learn.microsoft.com/en-us/azure/defender-for-cloud/d... Mind you, last time I designed something using this we just used ClamAV - which is pretty easy to develop against even if it is a slight pain to manage on an ongoing basis.

LinuxBender 2y ago

I am not a proper developer so I don't have the technical details on this, but another thing I have seen done is to buy access to an API on VirusTotal [1] and submit file hashes to be compared against their database of about 70 or so virus scanners. Not foolproof, as malware can be recompiled and hidden from VT until the actual file is submitted for sandbox analysis which you probably dont want to do with potentially sensitive data so it really only catches known-knowns but apparently some companies find it useful. I utilize it on a windows machine using ProcessExplorer that can automatically submit hashes but the API volume is limited for free / anonymous access. [1] - https://docs.virustotal.com/reference/overview https://docs.virustotal.com/reference/overview

toomuchtodo 2y ago
CISA is offering a malware analysis service now, requires a Login.gov account to use and the user be a US citizen. Unsure how it compares to VirusTotal, it’s in our backlog to spike a poc in our IR workflow. https://www.cisa.gov/resources-tools/services/malware-next-generation-analysis https://www.cisa.gov/resources-tools/services/malware-next-g...
- yard2010 2y ago
  I have to say this user agreement sounds like TikTok privacy policy, but my oh my! The government does its job. RESPECT.
rpigab 2y ago
If I was a malware distributor, I'd just throw a couple random bytes at the end each time I upload a malicious file, it'd still work and have a different checksum/hash.
- ecaradec 2y ago
  Antivirus are really stupid tools, but not that stupid. I said that from a time where I had to work around tools flagged by antivirus. Among stupid things they do are flagging a part of an executable, some nsis plugin flagged the whole package as virus as soon as you included them. I think they probably hash files by chunks, if you have too many bad chunks then you are a virus. A few bytes at the end doesn't change that.

matusgallik008 2y ago

Thanks for the insights! I had a look at ClamAV - never heard/worked with it. This was something I was looking for. Do you also handle large files e.g. videos with lambda? Don't you struggle with Lambda limits then?

cebert 2y ago
Great question. We use spot EC2 instances for ClamAV scans. For thumbnails, we still use Lambdas but use AWS Media Convert for Video files to select a frame for the thumbnail.