11 ms·
I looked through attacks in my access logs
- velcrovan 3y agoI make a point of running fail2ban on my servers and will even add custom jails to catch attacks specific to the types of functionality I may be exposing on the site(s) hosted on them. But it’s been a long time since I checked whether fail2ban’s other defaults are still comprehensive enough to block the most common attacks. I guess I’ll bookmark this link for when I get around to doing that.
- Palomides 3y agoif your systems have any of these easily targeted vulnerabilities exposed, fail2ban won't save you
- velcrovan 3y agofair point, the main reason I use fail2ban is to limit traffic from malicious activity rather than letting the attempts run rampant and unchecked.
- creeble 3y agoIf it makes you feel good, do it. It can also cut down on log noise a bit, for when you’re really looking for something. But in general, I’ve given up on caring about the routine “attacks” listed in all the logs. If you have good security, they don’t matter. And if you don’t, they don’t matter either.
- geraldhh 3y agocaring for pointless attacks is more work and bears more risk
- jbverschoor 3y agoIn addition, you can have nginx filters to check for simple patterns (php on a non-php site? -> instant ban). Too many 404s? -> instant ban.
- BLKNSLVR 3y agoI have recently started using my own sledgehammer-subtle approach of detecting (what I refer to as) Uninvited Activity on any port not offering a service, and straight-up banning the source IP (indefinitely at the moment) from accessing any actual service ports. Over the few months I've had it running I've needed to progressively create failsafes for IP addresses that I know are trustworthy so I don't lock myself out. I've also started tiering the importance of blocking based on different sets of ports which are being probed. I've also discovered that there's a significant amount of Uninvited Activity coming from "security" companies in their pro-active scanning of the entire IPv4 space - which I don't trust at all and ban with prejudice. It's messy and I need to update and add many explanations, but it's on Github if anyone wants a laugh: https://github.com/UninvitedActivity/UninvitedActivity https://github.com/UninvitedActivity/UninvitedActivity (I'm aware of various limitations and footguns inherent in this un-subtle approach but, as another commenter elsewhere alluded to, "it makes me feel better". I also think that a fair bit of processing volume can be taken off IDS' if a heap of "known garbage" traffic is blocked prior - it's all about tiers).
- mkoryak 3y agoSometimes I think it might be fun to setup an express server that correctly responds to one of these attacks just so I can waste someones time. But doing that would also waste my time.
- azinman2 3y agoIt’s all automated; not you’re not really wasting an actual persons time.
- pnw 3y agoIt does waste their time if your honeypot is constantly responding with legitimate looking but fake credentials. Presumably the hacker is going to try to use them? It’s the same idea used by anti-spam activists back in the day with software that would flood spam website forms with fake but realistic looking info, so the real data would be buried in the noise.
- runeb 3y agoThat part is also automated
- sumtechguy 3y agoFrom what I understand the automated part is sometimes a first pass. Just to see if you are there and have something. They will then wait some period of time and come back later with the real attack. Sort of like the war dialer from the movie wargames. Basically try everything get a list of interesting targets. Then go at them. Now some are fully automated and just try whatever exploits they are trying right then and there.
- 8organicbits 3y agoExactly. Poison the attacker's list of credentials with bogus values. This decreases their efficiency, slowing them down.
- 3y ago
- warkanlock 3y agoIf anyone is receiving these types of logs on AWS, please do yourself a favor and place AWS WAF in front of your VPC. It's not expensive and can significantly help you, saving you from many headaches in situations like this. While it might not block everything that arrives at your service, it can be a great help!
- anamexis 3y agoThis is a good suggestion, but careful with the default rulesets. We turned on AWS WAF (in our case, the motivation was just SOC 2 compliance). There were a few overzealous rules that subtly broke parts of our app. There were request body rules that did things like block requests that contained "localhost" in the request body. There was also a rule that blocked requests without a User-Agent header, which we were not previously requiring on API requests, so we broke our entire API for a few users until we figured that out.
- everfrustrated 3y agoIn my experience WAFs are not something that one should ever "just turn on". Complete due diligence is required to fully understand and realise the impact of the rules and should be tested like any software change by going through a testing phase. Ideally software teams should be fully trained and be responsible for their lifecycle.
- macNchz 3y agoYes you need to be familiar with the rulesets being applied, and prepared to closely monitor what is being blocked. Ideally I think I’d roll it out one ruleset at a time to limit the number of potential issues being introduced at once. Had a fun one after turning on the AWS WAF with some default rules–a small number of users reported they couldn’t upload new logo images anymore. Turned out some Adobe product was adding XML metadata to the image files, which the WAF picked up and blocked.
- marcosdumay 3y agoJust to add, but only testing will never work well enough for something like this. This is one of the cases where you must understand what you are doing. There's no technique for doing it mindlessnessly.
- SamuelAdams 3y agoWhat are some realistic, self hosted mitigation strategies for defending against these attacks?
- azeemba 3y agoUpdate your software frequently.
- klysm 3y agoBut not too frequently
- Tijdreiziger 3y agoI’d rather update it too frequently and potentially bork something, than not update it frequently enough and potentially get pwned.
- mfashby 3y agoUpdates as the other commenter says. Also isolation technology like docker containers, chroots, bsd jails, protections that systemd offers, or virtual machines. While not perfect, it means that the attackers must have the ability to chain exploits in order to break out of the compromised application to the rest of the host system.
- ggpsv 3y agoDocker is great but it is easy to shoot yourself on the foot if you use it conveniently but don't actually understand it. A common mistake is to publish the Docker ports unknowingly to all interfaces (e.g `5432:5432`), which makes your Docker container available to everyone. It is common to see this in Docker tutorials or pre-made Docker Compose files. Coupled with UFW, it may give you a false sense of security because Docker manages its own iptables rules.
- elashri 3y ago
- cantSpellSober 3y ago> the user agents for these attacks mention Mozlila/5.0 The linked write up on why is interesting, finds attacks for many of the same files, and recommends blocking the user agent.
- woodruffw 3y agoThe author says they aren’t a security person, so to correct a minor thing: the first examples are credential and configuration discovery, not directory traversal. The latter, to the best of my knowledge, is reserved for techniques where the attacker “escapes” the webroot or otherwise convinces the server to serve things outside of its normal directories.
- asynchronous 3y agoIt’s technically both if they’re performing an include file that wasn’t supposed to be hosted- ex. “/../../passwd/etc”
- 3abiton 3y agoThanks for the clarification!
- foofie 3y agoI also check the access logs collected by my self-hosted services, and I think there's a detail that's conspicuously absent from this analysis: the bulk of these malicious requests are made by regular people running plain old security scanners that are readily available from sites such as GitHub. These are largely unsofisticated attacks that consist of instances of one of these projects just hammering a server without paying any attention to responses or even if they have been throttled or not. Some attacks don't even target the IP and instead monitor domains and its subdomains, and periodically run the same scans from the exact same range of IP addresses. For example, on a previous job we had a recurring scan made over and over again from a single static IP address located in Turkey that our team started to refer to it as "the Turkish guy", and our incident response started featuring a preliminary step to identify weird request patterns that was basically checking if it was the Turkish guy toying with our services.
- azinman2 3y agoWhy protect the IPs of the those trying to attack you?
- bshacklett 3y agoMost of them are probably bots running without the knowledge of the IP owner. There’s little benefit to sharing those IPs with anyone other than the provider who owns them.
- BLKNSLVR 3y agoIs there a potential benefit in larger sharing of malicious-traffic IP addresses in that if more sites blocked these known-bad IP addresses then there'd be a higher chance that a victim would notice something is wrong as their online services have started blocking them? It feels to me like we're too polite, so we're letting the infected walk amongst us. It might not be their fault, but I'd be guessing it'd also be better for victim to find out sooner rather than later if they're pwned. I suppose the slippery slope / end game of this would then concentrate all intentional malicious traffic to VPNs and proxy's and Tor and the like, with them becoming useless due to being blocked. I block any IP address that's probed any of my ports they have no business probing. See https://news.ycombinator.com/item?id=39171782 https://news.ycombinator.com/item?id=39171782
- patja 3y agoFail2ban doesn't help when the attacker/abuser rotates their IP address constantly. I now look for aberrations in a few http headers they often neglect to spoof in their attempt to act like an honest human.
- boringuser2 3y agoThese attackers don't have inexhaustible resources.
- logifail 3y agoIndeed, but neither do you. Be very careful that fail2ban isn't actually exhausting your own resources faster than you believe you're exhausting the attackers' resources... https://www.google.com/search?q=fail2ban+resource+usage https://www.google.com/search?q=fail2ban+resource+usage
- quesera 3y agoI've recently logged credential stuffing attacks coming from over 100K distinct IPv4 addresses to a small backend API.
- vntok 3y agoRate-limit after x failed logins on either source IP, username and password. Just provide a realworld sidechannel escape hatch for legitimate users (ex: phone or email). Barely anyone will actually use it.
- quesera 3y agoWe had to go further. The attacks were from arbitrary IPs, and were working through a huge list of leaked username/password combinations. The attacker even went to the trouble of spoofing some custom headers in our API client. Our eventual solution was to a) attack our own auth credentials first, identify any users with leaked creds (from other services) and force a password reset for them. b) disallow users from setting common leaked passwords. c) make the auth checking request as low-cost as possible, and scalable separately from the main application. d) when an attack request is detected, bypass the relatively-expensive real cred check but return the same failure response (including timing) as a real failure. e) build a secondary requirement in the auth flow that can be transparently enabled when under high volume attack. This works, so far. It sheds the volume to the application, and has low-to-zero impact on legit users. This took a couple weeks away from feature development though!
- nothis 3y agoHyper-naive take: Couldn't nearly all of these attacks be blocked by a white-list approach, essentially hiding every file or directory from the internet except a very controlled list of paths and escaping all text sent so it can't contain code? I somehow always imagine these types of hacks to be more clever, like, I dunno, sending harmless-looking stuff that causes the program receiving it to crash and send some instructions into unprotected parts of RAM or whatever. This all looks like "echo ; /bin/cat /etc/passwd" and somehow the server just spitting it out. Is that really the state of web security?
- dartos 3y agoYou’re probably right, but consider that not every person is even aware of the security risks of running servers. Someone might be trying to play with self hosting or a contractor at a company did a bad job and accidentally exposed stuff they shouldn’t. This attacker is likely just trolling lots of IPs hoping for low hanging fruit that can be exploited with simple/well known attacks.
- Zetobal 3y agoSecurity through obscurity is like a ninja tiptoeing in a room full of laser beams; make one loud move and you'll reveal that your entire protocol hinges on no one sneezing!
- akerl_ 3y agoHow is strictly controlling exposed server resources to only URIs you’ve confirmed should be an exposed an example of “security through obscurity”.
- scherlock 3y agoYup, 99.999% are script kiddies running bots that look for unsecured servers or indicators for known exploits.
- quesera 3y ago> Couldn't nearly all of these attacks be blocked by a white-list approach, essentially hiding every file or directory from the internet except a very controlled list of paths and escaping all text sent so it can't contain code? This is basically how things work. For convenience, instead of itemizing each filename, the webserver root is a subdirectory and anything underneath is fair game. The webserver uses the OS "chroot" facility to enforce this restriction. What you are seeing is ancient exploitation strings from 30 years ago that haven't worked on any serious webserver since that time, but a) keeping the test in the attackers lib is essentially free, and b) there are some unserious webservers, typically in cheap consumer hardware. Webservers pass plain text to the app server. It is the app server/framework's responsibility to understand the source of the request body and present it to the application in a clear way, possibly escaped. But the app needs to process this and sometimes through poor coding practices, fails to respect the untrusted nature of the data. This again is more typical in historical systems and low-cost consumer products where software is not a marketing advantage.
- nocommandline 3y ago[flagged]
- quesera 3y agoFYI this repo is empty except for a README.
- nocommandline 3y agoYes. At the bottom - it says coming soon. I should probably move it to the top.
- Macha 3y agoWhy share it if there's nothing to share? It's just pointless marketing.
- CyberEldrich 3y ago[dead]
- Aachen 3y agoWhy not already publish the current state while testing? If the readme says it's currently in testing then it's still more inviting to look into than an empty repo
- nocommandline 3y agoSorry, I'm not quite clear what you mean. Do you mind explaining again?
- hk__2 3y agoNot OP but: why don’t you put the code in GitHub, even if it’s not ready?
- 1B05H1N 3y agoI work in application/product security and have managed WAFs for multi-billion dollar companies for many many years. Move DNS to Cloudflare and put a few WAF rules on your site (managed challenge if bot score less than 2 / attack score == x). I doubt you'll even pay anything, and it will resolve a lot of your problems. Just test it before moving it to production please (maybe setup a test domain). Remember, a WAF is not an end-all be all, it's more of a band-aid. If you app isn't hardened to handle attacks, no amount of advanced WAF/bot protection will save it. Message/email me if you need help.
- 93n 3y agoSelfhoster here. I use mutual TLS rules with CloudFlare's WAF to filter out everyone but my known-good callers. Works great. Since the only folks with access are my family, it was pretty easy to setup as well (everyone gets a unique cert that I can revoke if need be).
- asabla 3y agoUsually I only manage internal facing applications these days, which makes the attack surface greatly reduced compare to public ones. But since you seem to have a lot of knowledge in this area. Have you manage solutions which also includes infrastructure in Azure combined with Cloudflare? And if so, any suggestions on things people usually miss? except for the usual stuff of OWASP and what not
- CharlesW 3y agoI was unfamiliar with this, so for anyone who's in a similar position: https://blog.cloudflare.com/waf-for-everyone/ https://blog.cloudflare.com/waf-for-everyone/ The Free Managed Ruleset appears to be deployed by default, and Cloudflare keeps a changelog here: https://developers.cloudflare.com/waf/change-log https://developers.cloudflare.com/waf/change-log
- 418tpot 3y ago[flagged]
- solumunus 3y ago
- pferde 3y agoAn interesting thing that I've noticed is that some of the attackers watch the Certificate Transparency logs for newly issued certificates to get their targets. I've had several instances of a new server being up on a new IP address for over a week, with only a few random probing hits in access logs, but then, maybe an hour after I got a certificate from Let's Encrypt, it suddenly started getting hundreds of hits just like those listed in the article. After a few hours, it always dies down somewhat. The take-away is, secure your new stuff as early as possible, ideally even before the service is exposed to the Internet.
- elashri 3y agoIt is also useful to rely more on wildcard certs, as it makes it difficult to determine from CT logs the specific subdomains to attack.
- bombcar 3y agoThere's really no reason to avoid wildcard certs for your domains, unless you have so many subdomains that are managed by various business interests. I use LE wildcard certs and they're great, you can use them internally.
- couchand 3y agoIt seems like the principle of least power would apply here. There's value in restricting capability to no more than strictly necessary. Consider the risk of a compromised some-small-obscure-system.corporate.com in the presence of a mission-critical-system.corporate.com when both are issued wildcard certs. Wildcard certs are indeed a valuable tool, but there is no free lunch.
- baobun 3y agoYou'd usually put a reverse proxy exposing the services and terminating TLS with the wildcard cert. The individual services can still have individual non-wildcard internal-only certs signed by an internal CA. These don't need to touch an external CA or appear in CT logs - only the reverse proxy/proxies should ever hit these, and can be configured to trust the internal CA (only) explicitly.
- burgerquizz 3y agoI am wondering if there was any reports (on large scale systems) on what hackers were looking for on servers?
- CharlesW 3y agoCloudflare's WAF managed rulesets changelog might be helpful for this: https://developers.cloudflare.com/waf/change-log https://developers.cloudflare.com/waf/change-log For example, they did an emergency update on 1/22 in response to CVE-2023-22527.
- simpaticoder 3y agoThank you! I've been self-hosting for about a year running a 400-line http/s server of my own design, and it's remarkable all the attacker traffic my 3 open ports (22, 80, 443) get, although I've never taken the time to analyze what the attackers are actually trying to do! This post fills in a LOT of blanks. Would be cool to do the same thing for the weird stuff I see in /var/log/auth.log! It's crazy that attackers would bother with me since the code is entirely open source and there is no server-side state. The best outcome for an attacker would be root access on a $5/mo VPS, and perhaps some (temporary) defacement of the domain. A domain no-one visits!
- epcoa 3y agoThese are all automated bots. No one is “bothering”. You open the 3 most well known ports you’re going to get connections. They don’t know what you’re running nor do they care.
- simpaticoder 3y agoBy "bothering with me" I mean "add my IP to the long list of IPs they are scanning". By the way, I find it annoying that my logs get filled with this kind of trash. It has the perverse effect of making me long for something like Google Analytics since they rarely if ever bother running a javascript runtime.
- epcoa 3y agoThat long list isn’t curated, it’s every publicly routable IPv4 address. It really does not take long to run some canned probes against 3.7 billion addresses. Making your service IPv6 only tends to cut down on this traffic. You’re anthropomorphizing a script on some botnets.
- ericpauley 3y agoThis isn’t entirely true. Many scanners do preference specific IP ranges such as cloud providers. Cloud IPs receive substantially more scanning traffic than darknet IPs or even random corporate IPs.
- sph 3y agoSSH is constantly hammered by bots as well: % ssh example.com Last failed login: Sun Jan 28 16:59:35 UTC 2024 from 180.101.88.233 on ssh:notty There were 5385 failed login attempts since the last successful login. Last login: Sat Jan 27 13:33:30 2024 from xxx.xxx.xxx.xxx 5.3k failed attempts in ~30 hours. I know, I should be setting up fail2ban.
- SoftTalker 3y agofail2ban will reduce your log noise but it's another thing to manage, you can end up locking yourself out also, and if you're using good passwords (or better, public key auth) it's not really providing any additional safety. Ideally you don't need ssh open to the whole world anyway, and can restrict it to a certain subnet or set of addresses. Then your attacks will drop to nearly zero.
- NavinF 3y ago> I should be setting up fail2ban. No, that does nothing if you followed best practices and disabled password login. fail2ban is just another Denial of Service risk that has the added bonus of bloating your firewall table and slowing down all your new connections
- idoubtit 3y agofail2ban is not very performant and it will only reduce the amount of attempts. An alternative is to add a nftables rule (or iptables or whatever firewall). Something like : table inet filter { chain input { tcp dport ssh accept comment "Accept SSH" tcp dport 22 ct state new limit rate over 2/minute drop But even with rate limiting, the logs are still polluted by auth attempts. Changing the port does little. The only solution we found was to configure port knocking (with direct access from a few whitelisted IPs).
- miyuru 3y agoMost of my servers are IPv6 only and there are no failed ssh attempts on those. I install fail2ban just in case and firewall the IPv4 address, since I don't SSH via IPv4.
- nerdponx 3y ago
- rsolva 3y agoIf someone has experience with CrowdSec (a kind of crowdsourced fail2ban), I would like to hear your opinion!
- snowwrestler 3y agoBack when I started managing self-hosted sites, I would look through access logs as well. We even had an IDS for while that would aggregate the data and flag incoming attack attempts for us. Eventually I stopped proactively reviewing logs and stopped paying for the IDS. It was a waste of time and a distraction. It's not hard to find really useful content that summarizes common vulnerabilities and attacks, and just use that to guide your server management. There are a ton of best practices guides for any common web server technology. Just executing these best practices to 100% will put you way ahead of almost all attackers. And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities. Where logs are super helpful is in diagnosing problems after they happen. We used log analysis software to store and search logs and this was helpful 2-3 times to help find (and therefore address) the root cause of attacks that succeeded. (In every case it turned out to be a known vulnerability that we had been too slow to patch.)
- dandrew5 3y ago> And then the next best use of your time and resources is to prioritize the fastest possible patching cadence, since the vast majority of attacks target disclosed vulnerabilities. Just curious, do you leverage any tools to decide when to patch or is it time-interval based? We currently attempt[0] to update our packages quarterly but it would be nice to have a tool alert us of known vulnerabilities so we can take action on them immediately. [0] "Attempt" meaning we can't always upgrade immediately if the latest version contains difficult-to-implement breaking changes or if it's a X.0.0 release that we don't yet trust
- eyegor 3y agoBesides pointing pentester tools like metasploit at yourself, there are some nice scanners out there. Examples in no particular order: https://github.com/quay/clair https://github.com/quay/clair https://github.com/anchore/grype/ https://github.com/anchore/grype/ https://github.com/eliasgranderubio/dagda/ https://github.com/eliasgranderubio/dagda/ https://github.com/aquasecurity/trivy https://github.com/aquasecurity/trivy So then you set up something like a cron job to scan everything for you and email the results once a week or whatever if you don't want to monitor things actively.
- dangus 3y agoI would be interested in reading something similar but focused on less common ports and services, like game servers that run on high ports and more typically use UDP. I wonder if that’s more or less of a “safe” situation.
- jbverschoor 3y agoYou could and should alway monitor protocol errors (any protocol).
- evantbyrne 3y agoThe elephant in the room is that–at least in my experience–a lot of these attacks come from hostile nation states. This is going to be controversial, but one may find it useful to block entire IP ranges of problematic states that you cannot do business with. I was able to block 100% of probes to one of my new services by doing this.
- macintux 3y ago15+ years ago I was reviewing server logs for small, local businesses we hosted and came to the conclusion we should just block all APNIC IP addresses.
- nerdponx 3y agoIt's not an elephant, it's just that many people aren't willing to block legitimate users from those regions.
- evantbyrne 3y agoAmerican websites don't typically have customers dialing in from North Korea. Just saying that IP blocking is something more businesses should consider. Traffic can also be routed to different subdomains for businesses that need to provide a subset of services to a region.
- tjbiddle 3y ago
- Erratic6576 3y agoI once got a message in my logs “you have been powned” :/
- ijhuygft776 3y agowhat did you do about it
- Erratic6576 3y agoI don’t remember well. I guess I Worried and stopped reading logs. Aah. I diverted my domain to cloudflare, allowing only traffic from 1 country in. So instead of exposing my IP publicity through my registrar, I set the firewall to only allow traffic in from cloudflare. I would have loved to install PfSense as well but it was out of my budget
- ijhuygft776 3y agoso, you, are, still, hacked.
- 8organicbits 3y agoI can visit http://example.com/i_hacked_you http://example.com/i_hacked_you and it will be in the logs but no hacking has occurred.
- Erratic6576 3y agoImpossible. My 3 servers are hackproof now. 100 % safe
- iboisvert 3y agoAs someone who knows very little about security, this is really interesting, thanks! A question though: how would one know if there has been a breach? These examples look relatively easy to detect, but I guess there would be more complex cases?
- kevindamm 3y agoThis is why some people run a honeypot in their network... and even those won't necessarily catch everything if the honeypot only mimics services that the attacker isn't probing for. You can set up tripwires on access and egress of sensitive data but that's only part of the surface area (and if the system gets attacked those tripwires could be disabled, if the attacker either knows what to look for or has a plan for a side channel for exfiltrating data). Really the only good answer is defense in depth and keep looking for any indicators of odd behavior, and wall out unrelated systems entirely from each other, keep the DMZ and public facing bits as simple as possible.
- macintux 3y agoI also know very little, but something that struck me upon reading your question: if a breach is successful, the logs can't be relied upon for detection/analysis if they're on the same server. It's important to ship them elsewhere.
- tamimio 3y agoIOC or indicators of compromise, but if you know little it is always advisable to hire someone to go through it on demand or periodically as there’s no one trick to rule them all.
- takemine 3y agoYou can use honeypot that bait hackers . I am running a non-intrusive one where you put baits in your servers or laptop, when hackers see it, they'll try to use them.
- emmanueloga_ 3y agoWAF seems to be an essential piece for any website with even a little bit of visibility / traffic on the net. Some questions: * Comparison of AWS WAF vs Cloudflare vs Others? * Many services like EC2 charge for data transfer [1], so how much of your monthly/yearly costs of hosting goes toward fending scans like these? Does AWS count any traffic blocked by the WAF toward the transfer limits? -- 1: https://aws.amazon.com/ec2/pricing/on-demand/ https://aws.amazon.com/ec2/pricing/on-demand/
- rfmoz 3y agoWith Fastly WAF - SignalSciences, Distil Networks - Imperva, Akamai - StackPath. There are a few companies that started alone in this ecosystem and found the evolution path with the mayor CDN networks, those who have failed themselves in provide a good internal alternative service. Human security - PerimeterX, Haproxy WAF, Datadome, are other players to different target audience. If you have a good control over the app exposed, maybe you only need a WAF in the sense of stop stuff outside your infra. The sql injections, weird urls trying the way to /etc/passwd or related things look from the past and only makes noise nowadays. The real issue is when someone hits you in a rate impossible to manage with your resources or when it cost you more than the securing layer.
- nickjj 3y agoIf you're on AWS the AWS WAF is pretty low cost. You can expect to pay less than $10 / month and still get an ok amount of value on a decently popular site. The problem is you have to manually configure a lot, the rate limiting aspect is way worse than Cloudflare and while the AWS WAF can geolocate an IP address and block by country it does not send the country code back to you in a header where as Cloudflare does. The last one stings because it's super handy to have an accurate country code attached to each request, especially if it's something you don't need to think about or waste I/O time calling out to a 3rd party service in the background to backfill in that data later.
- emmanueloga_ 3y agoThis is helpful! I found some CDK libraries that allows for connecting a load balancer or Cloudfront to WAF with a few lines of code. I'll give it a try! [1] [2]. -- 1: https://github.com/awslabs/aws-solutions-constructs/tree/main/source/patterns/%40aws-solutions-constructs/aws-wafwebacl-cloudfront#overview https://github.com/awslabs/aws-solutions-constructs/tree/mai... 2: https://constructs.dev/search?q=waf&cdk=aws-cdk&cdkver=2&langs=typescript&sort=downloadsDesc&offset=0 https://constructs.dev/search?q=waf&cdk=aws-cdk&cdkver=2&lan...
- m0rissette 3y agoMan people don’t use shadow anymore and expose creds through /etc/passwd; craziness
- SebFender 3y agoInteresting, well prepared and presented - a necessary read for many.
- urbandw311er 3y agoAs a thought experiment - if there was public money to back this, would it be making us all safer to run a series of honeypot servers that automatically start DDOSing the various C&C servers that attempt to compromise them?
- PeterisP 3y agoSuch a response would generally be a crime; existing cybercrime legislation generally does not have any clauses permitting retaliation as "self defence", and also very often you'd be DDoSing an innocent third party, another victim whose compromised device is abused to route traffic, and also affecting their neighbors on the same network.
- urbandw311er 3y agoFair point, I hadn’t considered that. Perhaps it might encourage people to keep their systems patched if the outcome of not doing so was being Ddos’d!
- qaq 3y agoTangentially the sad thing about whole cyber sec space is that well resourced APTs like say APT-29 Cozy Bear. Have enough resources and actually run labs where they deploy all top Endpoint solutions and validate their offensive tools against them.
- yawaramin 3y agoDoes anyone else have the experience that almost every single one of the 'attacks' is using HTTP and not following a 301 redirect to HTTPS? I have my internet-facing web server set up to redirect all HTTP -> HTTPS and this thwarts almost every 'attack'. I have to say, they're not very smart about it.
- gary_0 3y agoI do that as well, but nothing gets through anyways because I'm not running Wordpress or a version of Apache from 2012. I also have nginx set up to reject connections without the expected 'Host' header, and that quickly bounces a few attackers. But many still end up getting all the way to a 404 or 400 for their attack URL.
- aborsy 3y agoI’m looking for a secure authentication/proxy application to put in front of the webservers that have to be exposed to the internet. The application will authenticate the user with SSO or hardware key, before forwarding the traffic to the right internal address. Cloudflare Tunnels are great, but CF doesn’t allow the TLS pass through. CF man in the middles and decrypts the traffic. So far I have looked into Teleport, authelia, authentik, and keycloak, perhaps combined with Traefik. Any feedback on the level of security of these tools for being exposed to the public internet?
- lukax 3y agoIf your auth provider supports is OAuth 2 OIDC you should check oauth2-proxy. It's just one binary or a container sidecar. https://github.com/oauth2-proxy/oauth2-proxy https://github.com/oauth2-proxy/oauth2-proxy
- deleted 3y ago[deleted]
- FuriouslyAdrift 3y agoGranted we are a Microsoft-based shop, but Microsoft Entra Application Proxy has worked out great for exposing our internal web based apps to the outside for mobile/home workers. https://learn.microsoft.com/en-us/entra/identity/app-proxy/application-proxy https://learn.microsoft.com/en-us/entra/identity/app-proxy/a...
- aborsy 3y agoThis seems to be very similar to the Cloudflare tunnels. They are reverse proxies in the cloud, with TLS termination. The traffic is terminated and scanned in the cloud. https://learn.microsoft.com/en-us/entra/identity/app-proxy/application-proxy-security https://learn.microsoft.com/en-us/entra/identity/app-proxy/a... A version with end to end encryption will be great!
- zetalemur 3y agoInteresting. Came to similar conclusions when analyzing my (httpd) access logs for https://turmfalke.httpd.app/demo.html https://turmfalke.httpd.app/demo.html ... but so far nothing really out of the ordinary.
- pid1wow 3y ago> directory traversal The correct term is directory enumeration. Traversal usually means something about ../../
- daneel_w 3y agoCurious enumerations on the most common items. There's definitely a topical bias. By far the most common attack attempt I see on all the various webhosts I administer is WordPress-oriented (despite not present on any of the hosts), which doesn't even get an honorable mention by the author. Perhaps he hosts WordPress content and didn't discern attacks from legitimate traffic.
- tamimio 3y agoYeah some thing in websites I have, even the personal one, they get on average tens of these attacks, and again, it seems they are after .env too and other php related directories. Big portion of these attacks come from TOR networks too.
- elendee 3y agoautoGPT thanks you for this lucid writeup
- takemine 3y agoNice analysis! You should protect your infra to avoid this kind of scanning: - Disable password login for SSH, use keys instead. - Limit access to known IPs (with a managed vpn) - Use Cloudflare: Their WAF is really good - Forward logs to an other service that can analysis logs (datadog is nice) shameless plug: started a small honeypot service[1] if anyone would need it as a last resort[1] to catch hackers in your servers . Feedbacks appreciated! [1] https://hackersbait.com https://hackersbait.com
- oars 3y agoGreat post, thanks for sharing.
- Adachi91 3y agoI always find it fun to respond in strange ways to "malicious" requests on all my webservers derived from a defcon talk[0] that I watched a while ago, there's a lot of great fun to be had in honeypots and other things of the nature. [0] https://www.youtube.com/watch?v=4OztMJ4EL1s https://www.youtube.com/watch?v=4OztMJ4EL1s
- tgrzinic 3y agoNice article. Btw how do you make sure you are not hacked or that some frivolous attacker didn't accessed something valuable?
- wkjagt 3y agoThere are quite a few examples in there of “the bad guy” trying to find files (like .env) that are accidentally left somewhere. From my PHP days I remember that indeed Apache/PHP just serves up any file from anywhere as a static file if it isn’t PHP. My memory is pretty vague on this and I don’t remember if this behaviour is configurable but I guess it must be. Having done mostly Ruby on Rails since, it feels so strange now that a web server can be some kind of a file browser into your code base. Am I remembering this correctly? Are there other languages/web servers that work like this?
- altilunium 3y ago[dead]
- morpheos137 3y agoI know many people decry security through obscurity but some times it works. For example several years ago I changed the ssh port on a VPS I was running to a non-standard port. Logged login attempts from strange IP addresses went from dozens per day to zero.