5 ms·

> Being able to run metasploit and wireshark does not make one a hacker. By doing this, pentesters test for yesterday's hacks, not tomorrow's. Be careful here.

by movedx 6y ago

> Being able to run metasploit and wireshark does not make one a hacker. By doing this, pentesters test for yesterday's hacks, not tomorrow's. Be careful here. This is bordering on elitism. Having someone come into a business and check for "yesterday's hacks" is better than no one doing any checks at all, therefore such skills are still valuable and worthwhile. In learning how networking works; how operating systems are designed and implemented; why and how the OWASP Top Ten work; and knowing solutions to these problems is still a valuable skill set. What you're suggesting is everyone has to be willing and able to write fresh exploitation on the spot when they're testing a client's network when in fact there's a lot that can be discovered (and resolved) with basic scans and simple questions based off of Security+ grade knowledge.

GekkePrutser 6y ago
I know, but most of these "hacks" are identified by internal scans already. The pentest doesn't add much value then. The issue is more internal resistance to change in the management team. Like I said I know most companies already fail at the basics. But these are normally well known already, just not fixed due to political pressure. Having the security team's management be better at influencing would pre-empt these issues. Pentests serve to bring these known issues under discussion again, but they often don't bring any unknown issues to light. Also, they tend to be too artificial. In one example: In a recent pentest I know of, they sent some remote access malware to the admins, which they all ignored and reported. However as they couldn't continue they called one admin's manager and asked him to tell the admin to click on the link so that they could proceed. The outcome of the pentest was that if a malicious actor sent a malicious remote access malware and if it was clicked on, they would have remote access. Well duh. Of course this could be mitigated by having separate workstations for email and admin activity, which was an issue that was already understood and a mitigation in progress. This is what I mean by pentesting not raising new issues. Another example: The company in question already test new web apps for known attacks, and is quite successful (in fact I've never seen a new app come through the certification process in the first round). What I'd expect from a pentest is to tell us something we don't know :) It's an interesting second-opinion but it's not the amazing X-ray it's promised to be. > What you're suggesting is everyone has to be willing and able to write fresh exploitation on the spot when they're testing a client's network Which is exactly what a serious adversary would be doing in a targeted hack!
- OminousWeapons 6y ago
  > I know, but most of these "hacks" are identified by internal scans already. The pentest doesn't add much value then. The issue is more internal resistance to change in the management team. This sounds like management's fault for using a service they don't need yet. If there are glaring, obvious vulns that are repeatedly pointed out but aren't getting fixed then how useful is it to point out additional more subtle vulns? I understand the point you are trying to make about pentests telling you data you already know, but how many people are going to take the time to enumerate subtle vulns when there are basic ones that can be hit to great effect? > Which is exactly what a serious adversary would be doing in a targeted hack! I don't think it makes sense to worry about what a serious adversary could do when a moderately skilled one can already wreck you with known exploits.
  - GekkePrutser 6y ago
    Well, the issue is that these vulnerabilities can't be hit to great effect, because they're usually well understood and mitigated by other means (see the admins being very conscious of email malware). In many cases the pentesters cheat by asking for special permissions or for someone to click something they wouldn't normally have done. Just so they can be seen to 'have found something'. I think a targeted adversary is more worrying because they usually are driven by high gains, which means something that's a big risk to the company. Like strategic information that can cripple the company. Whereas a more moderate adversary will usually trigger a ransomware campaign on some low-secured laptops or something which can be mitigated in a day or 2 by restoring backups. Big interruption yes. Company-killer no. Also an advanced adversary will usually operate unseen altogether and penetrate the highest levels of security. A ransomware attack is much more obvious. I would view them as different things altogether. An advanced adversary might use the same techniques for their initial access but due to the many layers of security this won't be enough to reach the most critical information.
- Izmaki 6y ago
  So your point is "bad pentests are bad and provide no value". Yes. You are correct.
tptacek 6y ago
Low-quality consultant penetration tests that rely on scanner results are a bigger problem in our industry than elitism.
noch 6y ago
> Be careful here. This is bordering on elitism. Be careful about the soft bigotry of low expectations. A generation has grown up thinking that skill/knowledge elitism is a real thing and that it's oppressive. Instead we should learn to identify people who do difficult things, recognise how they do difficult things well, systematically emulate their methods as we attempt difficult things, and constantly work at the edge of our ability. The only thing that happens if we don't do difficult things is that we become people who can't do difficult/valuable things.
- movedx 6y ago
  What part of my response led you to believe I inferred we shouldn't learn difficult things? I'm simply try to show that people who haven't mastered the industry are still valuable.
  - noch 6y ago
    > What part of my response led you to believe I inferred we shouldn't learn difficult things? The first sentence, and its emphasis on calling out elitism (and concomitant gate-keeping). > I'm simply try to show that people who haven't mastered the industry are still valuable. Despite your noble intentions, I don't think such comforting statements are useful, and they are probably harmful: On the one hand, the market will decide what skills and what level of skills it wants to pay for (and how brutal when, after often being comforted, consoled, and reassured, I find no one willing to pay me for my meagre skills), and on the other hand, there are more than enough people who are confidently happy to run metasploit and burp, shut their laptops, and demand a pay-cheque.
ozim 6y ago
Decent sysadmin should be able to setup automated scanning, know about OWASP, keep systems patched. That should be absolute baseline for sysadmin work. Checking for yesterdays hacks is valuable but that is called auditing and you should have checklist for that. Pentesting should be focusing on hacking and uncovering stuff that was not in audits like making custom exploits on the spot. Problem is that running scanners and checking basic stuff should be an audit but that does not sound as cool as "pentest" and "white hat hacker"/"cybersecurity specialists" sounds way cooler than auditor.
- movedx 6y ago
  > Pentesting I think I see the problem. You're assuming several things here. Firstly you're assuming that pentesting is the definition of cyber security when in fact it's one aspect of a huge area. Secondly you're implying that everyone wants to be a pentester. And finally I feel like you read the article and came to the conclusion it was only about becoming a pentester, when in fact it was about getting into a much larger field that contains maybe jobs with many different objectives.
  - ozim 6y ago
    System administrator should be main actor in security. Thing is no one wants to be system administrator now, everyone wants to be "cybersecurity expert/ethical hacker". Which in my opinion "ethical hacker" should be something that we call pentesters and we don't need system administrators to be "ethical hackers". I do not equate pentesting with all security, it is that people don't want to do boring job of sysadmins. People don't want to spend all day going through logs, going through alerts and sorting out false positives. Which is real job there and is boring and that is not what "hackers do". If that article would have title "How to become system administrator" no one would read it or comment it. Pentesting is really small part of security that should be an elite, because most of the work is in system administration.
celticninja 6y ago
The majority of hacks that companies suffer will be 'yesterdays' hacks. No one is wasting a zero day to get the droneparts.com database.
badrabbit 6y ago
Also, metasploit isn't used just for yesterdays hacks, it's great for ever your 0days to manage sessions and do some of the repetitive/boring leg work for you

by movedx 6y ago

GekkePrutser 6y ago

I know, but most of these "hacks" are identified by internal scans already. The pentest doesn't add much value then. The issue is more internal resistance to change in the management team. Like I said I know most companies already fail at the basics. But these are normally well known already, just not fixed due to political pressure. Having the security team's management be better at influencing would pre-empt these issues. Pentests serve to bring these known issues under discussion again, but they often don't bring any unknown issues to light. Also, they tend to be too artificial. In one example: In a recent pentest I know of, they sent some remote access malware to the admins, which they all ignored and reported. However as they couldn't continue they called one admin's manager and asked him to tell the admin to click on the link so that they could proceed. The outcome of the pentest was that if a malicious actor sent a malicious remote access malware and if it was clicked on, they would have remote access. Well duh. Of course this could be mitigated by having separate workstations for email and admin activity, which was an issue that was already understood and a mitigation in progress. This is what I mean by pentesting not raising new issues. Another example: The company in question already test new web apps for known attacks, and is quite successful (in fact I've never seen a new app come through the certification process in the first round). What I'd expect from a pentest is to tell us something we don't know :) It's an interesting second-opinion but it's not the amazing X-ray it's promised to be. > What you're suggesting is everyone has to be willing and able to write fresh exploitation on the spot when they're testing a client's network Which is exactly what a serious adversary would be doing in a targeted hack!

OminousWeapons 6y ago
> I know, but most of these "hacks" are identified by internal scans already. The pentest doesn't add much value then. The issue is more internal resistance to change in the management team. This sounds like management's fault for using a service they don't need yet. If there are glaring, obvious vulns that are repeatedly pointed out but aren't getting fixed then how useful is it to point out additional more subtle vulns? I understand the point you are trying to make about pentests telling you data you already know, but how many people are going to take the time to enumerate subtle vulns when there are basic ones that can be hit to great effect? > Which is exactly what a serious adversary would be doing in a targeted hack! I don't think it makes sense to worry about what a serious adversary could do when a moderately skilled one can already wreck you with known exploits.
- GekkePrutser 6y ago
  Well, the issue is that these vulnerabilities can't be hit to great effect, because they're usually well understood and mitigated by other means (see the admins being very conscious of email malware). In many cases the pentesters cheat by asking for special permissions or for someone to click something they wouldn't normally have done. Just so they can be seen to 'have found something'. I think a targeted adversary is more worrying because they usually are driven by high gains, which means something that's a big risk to the company. Like strategic information that can cripple the company. Whereas a more moderate adversary will usually trigger a ransomware campaign on some low-secured laptops or something which can be mitigated in a day or 2 by restoring backups. Big interruption yes. Company-killer no. Also an advanced adversary will usually operate unseen altogether and penetrate the highest levels of security. A ransomware attack is much more obvious. I would view them as different things altogether. An advanced adversary might use the same techniques for their initial access but due to the many layers of security this won't be enough to reach the most critical information.
Izmaki 6y ago
So your point is "bad pentests are bad and provide no value". Yes. You are correct.

tptacek 6y ago

Low-quality consultant penetration tests that rely on scanner results are a bigger problem in our industry than elitism.

noch 6y ago

> Be careful here. This is bordering on elitism. Be careful about the soft bigotry of low expectations. A generation has grown up thinking that skill/knowledge elitism is a real thing and that it's oppressive. Instead we should learn to identify people who do difficult things, recognise how they do difficult things well, systematically emulate their methods as we attempt difficult things, and constantly work at the edge of our ability. The only thing that happens if we don't do difficult things is that we become people who can't do difficult/valuable things.

movedx 6y ago
What part of my response led you to believe I inferred we shouldn't learn difficult things? I'm simply try to show that people who haven't mastered the industry are still valuable.
- noch 6y ago
  > What part of my response led you to believe I inferred we shouldn't learn difficult things? The first sentence, and its emphasis on calling out elitism (and concomitant gate-keeping). > I'm simply try to show that people who haven't mastered the industry are still valuable. Despite your noble intentions, I don't think such comforting statements are useful, and they are probably harmful: On the one hand, the market will decide what skills and what level of skills it wants to pay for (and how brutal when, after often being comforted, consoled, and reassured, I find no one willing to pay me for my meagre skills), and on the other hand, there are more than enough people who are confidently happy to run metasploit and burp, shut their laptops, and demand a pay-cheque.

ozim 6y ago

Decent sysadmin should be able to setup automated scanning, know about OWASP, keep systems patched. That should be absolute baseline for sysadmin work. Checking for yesterdays hacks is valuable but that is called auditing and you should have checklist for that. Pentesting should be focusing on hacking and uncovering stuff that was not in audits like making custom exploits on the spot. Problem is that running scanners and checking basic stuff should be an audit but that does not sound as cool as "pentest" and "white hat hacker"/"cybersecurity specialists" sounds way cooler than auditor.

movedx 6y ago
> Pentesting I think I see the problem. You're assuming several things here. Firstly you're assuming that pentesting is the definition of cyber security when in fact it's one aspect of a huge area. Secondly you're implying that everyone wants to be a pentester. And finally I feel like you read the article and came to the conclusion it was only about becoming a pentester, when in fact it was about getting into a much larger field that contains maybe jobs with many different objectives.
- ozim 6y ago
  System administrator should be main actor in security. Thing is no one wants to be system administrator now, everyone wants to be "cybersecurity expert/ethical hacker". Which in my opinion "ethical hacker" should be something that we call pentesters and we don't need system administrators to be "ethical hackers". I do not equate pentesting with all security, it is that people don't want to do boring job of sysadmins. People don't want to spend all day going through logs, going through alerts and sorting out false positives. Which is real job there and is boring and that is not what "hackers do". If that article would have title "How to become system administrator" no one would read it or comment it. Pentesting is really small part of security that should be an elite, because most of the work is in system administration.

celticninja 6y ago

The majority of hacks that companies suffer will be 'yesterdays' hacks. No one is wasting a zero day to get the droneparts.com database.

badrabbit 6y ago

Also, metasploit isn't used just for yesterdays hacks, it's great for ever your 0days to manage sessions and do some of the repetitive/boring leg work for you