6 ms·

Smart Contract Security for Pentesters

63 points by dyates 5y ago

mratsim 5y ago
I'd like to add that there is a critical shortage of security auditors for smart contracts and blockchain protocols. Projects are willing to spend up to millions to squash away vulnerabilities. For example Balancer opened a bug bounty for their v2 with $2M USD for 1 critical bug: https://docs-v2.balancer.finance/core-concepts-1/security/bug-bounties https://docs-v2.balancer.finance/core-concepts-1/security/bu...
- R0b0t1 5y ago
  Do they have any interest in hiring regular employees? Most of the people I know who go after bounties barely make $30,000/yr. If I see bounties I see people who aren't really willing to pay.
  - mratsim 5y ago
    You should apply to auditing firms (Trail of Bits, Open Zeppelin, Quantstamp, ...). They are all booked 3 months in advance at the very least and would love to onboard new blood.
    - toomuchtodo 5y ago
      What’s the comp look like?
      - seibelj 5y ago
        High - solid blockchain devs can pull a lot of money. Let me underline _solid_, there are a lot of people who took a tutorial and try to sell themselves as experts.
    - dguido 5y ago
      Yes! Please do apply. We have a tight-knit team of experts, industry leading tools, and a work environment that promotes continued learning. You'll get paid well and quickly become a leading industry expert. Here's our job req for a blockchain security analyst: https://jobs.lever.co/trailofbits/4f459855-3299-462f-9e73-299a840d5baf https://jobs.lever.co/trailofbits/4f459855-3299-462f-9e73-29...
  - Jhsto 5y ago
    Note the difference between "I could not find any bugs" and "I proved there are no bugs". I would assume only the latter are hired. Yet, program proving as in formal verification is a very academic specialisation. The reluctance to hire these people seems partly a) unawareness these methods exist and b) capability to evaluate whether someone knows enough. To my anecdotal experience founders of DeFi applications are not that tech savvy, so instead of trying to understand Solidity (untrivial) they instead place bug bounties or hire a special firm to give a stamp of approval for a product launch.
    - R0b0t1 5y ago
      My take on formal verification was that we are still not close to being able to usefully prove the validity of the types of nontrivial programs that make up DeFi contracts. It can help, sure, but companies serious about security need to invest in internal auditing (that may not seem to be generating obvious returns) instead of hoping for a bug bounty Hail Mary. If these firms are out there and are being hired I suppose that counts.
motohagiography 5y ago
Naive question: how is looting vulnerable smart contracts even illegal? Without a legal framework of smart contract enforcement, recognition of literally-hypothetical assets as valuable, the public nature of blockchains that would preclude "unauthorized access," and unlike an exchange holding assets on behalf of customers - smart contracts are effectively leaving money on the ground for anyone clever enough to pick it up. Clearly I haven't given it as much thought as the people involved, but it seems like if I'm not using my abilities full-time to hack and loot smart contracts, I'm missing the most direct and best possible effort/reward application of that kind of skill.
- jude- 5y ago
  IANAL, but I'd imagine that for US folks, violating the intent of the code -- namely, doing something while lacking the authorization to do so -- is illegal, even if the code lets you do it. Otherwise, basically all forms of criminal hacking would be legal. Of course, if the smart contract expressly permitted anyone to take tokens out of it via any means allowed by the platform, that's a different story.
  - TheRealPomax 5y ago
    Except in the US you can't sue on just intent, the letter of the law still needs to agree. So if the smart contract permits something, by not forbidding it, that something is entirely legal.
    - jude- 5y ago
      That can't be right. That would mean that all forms of hacking are legal, since after all, the remote server permitted you to run your shell code.
  - jcranmer 5y ago
    The law you're thinking of is CFAA, and it's not clear that this violates the CFAA. The requisite element is "accesses [...] without authorization or exceeds authorized access." Given the general enthusiasm of blockchain proponents to believe that "the code is law", it's a pretty easy argument to make that taking advantage of poorly-written code is well within the user's authorized capabilities. Will it win in court? shrug
ramimac 5y ago
I would highly recommend anyone interested in pentesting smart contracts look at the work Trail of Bits has been putting out: https://blog.trailofbits.com/?s=smart+contract&submit=Search https://blog.trailofbits.com/?s=smart+contract&submit=Search Start with "246 Findings From our Smart Contract Audits: An Executive Summary" [1] [1] https://blog.trailofbits.com/2019/08/08/246-findings-from-our-smart-contract-audits-an-executive-summary/ https://blog.trailofbits.com/2019/08/08/246-findings-from-ou...
Anointmous 5y ago
I hate the term "pentest", but apparently people who want lingo over the ability to do anything have won out over the decades. Besides being a meaningless inaccurate shortening of the phrase, an actually "pen test" would be part of putting a pen register on a phone. It just indicates that the newbies who created the term didn't know anything before.
ketamine__ 5y ago
Was the issue with Fei actually a bug?
rob-olmos 5y ago
I've been interested in any smart contract languages/VMs that are somehow more capable of being provably correct/secure. The only one I've come across is Kadena, which internally uses the Z3 prover, but I haven't looked into the source code in depth or if it's able to be applied to custom smart contracts (dApp) as well. Are there other blockchains that are similar? Is there a strict subset and prover for Solidity or other languages? Or things like proven smart contract kernels that can be built on top of? Eg, OpenZeppelin Contracts, but with provers rather than only audits.
- jude- 5y ago
  Disclaimer: I work on this. You should check out Clarity (clarity-lang.org). It's an on-chain interpreted language, so what you see is what will run. It's statically typed and decidable, such that you can reason about all the halting states of the program as well as the upper bounds on the amount of computing resources used. It's used by the Stacks blockchain and Algorand.
- dguido 5y ago
  Ugh, I have been advocating "Solidity--" for years and can't get funding to build it (Trail of Bits). We use two tools to offer quick turnaround automated testing and verification for Solidity: Echidna (like QuickCheck for Solidity) and Manticore (a symbolic verifier). They each let you write high level properties in the span of 1-2 weeks that cover a large amount of potential use cases. Here's an example of what that looks like: https://github.com/trailofbits/publications/blob/master/reviews/Liquity.pdf https://github.com/trailofbits/publications/blob/master/revi... Here's Echidna: https://github.com/crytic/echidna https://github.com/crytic/echidna and Manticore: https://github.com/trailofbits/manticore https://github.com/trailofbits/manticore Sometimes we also use custom static analyses built around Slither's IR during projects too: https://github.com/crytic/slither/wiki/SlithIR https://github.com/crytic/slither/wiki/SlithIR

63 points by dyates 5y ago

I'd like to add that there is a critical shortage of security auditors for smart contracts and blockchain protocols. Projects are willing to spend up to millions to squash away vulnerabilities. For example Balancer opened a bug bounty for their v2 with $2M USD for 1 critical bug: https://docs-v2.balancer.finance/core-concepts-1/security/bug-bounties https://docs-v2.balancer.finance/core-concepts-1/security/bu...

R0b0t1 5y ago
Do they have any interest in hiring regular employees? Most of the people I know who go after bounties barely make $30,000/yr. If I see bounties I see people who aren't really willing to pay.
- mratsim 5y ago
  You should apply to auditing firms (Trail of Bits, Open Zeppelin, Quantstamp, ...). They are all booked 3 months in advance at the very least and would love to onboard new blood.
  - toomuchtodo 5y ago
    What’s the comp look like?
    - seibelj 5y ago
      High - solid blockchain devs can pull a lot of money. Let me underline _solid_, there are a lot of people who took a tutorial and try to sell themselves as experts.
  - dguido 5y ago
    Yes! Please do apply. We have a tight-knit team of experts, industry leading tools, and a work environment that promotes continued learning. You'll get paid well and quickly become a leading industry expert. Here's our job req for a blockchain security analyst: https://jobs.lever.co/trailofbits/4f459855-3299-462f-9e73-299a840d5baf https://jobs.lever.co/trailofbits/4f459855-3299-462f-9e73-29...
- Jhsto 5y ago
  Note the difference between "I could not find any bugs" and "I proved there are no bugs". I would assume only the latter are hired. Yet, program proving as in formal verification is a very academic specialisation. The reluctance to hire these people seems partly a) unawareness these methods exist and b) capability to evaluate whether someone knows enough. To my anecdotal experience founders of DeFi applications are not that tech savvy, so instead of trying to understand Solidity (untrivial) they instead place bug bounties or hire a special firm to give a stamp of approval for a product launch.
  - R0b0t1 5y ago
    My take on formal verification was that we are still not close to being able to usefully prove the validity of the types of nontrivial programs that make up DeFi contracts. It can help, sure, but companies serious about security need to invest in internal auditing (that may not seem to be generating obvious returns) instead of hoping for a bug bounty Hail Mary. If these firms are out there and are being hired I suppose that counts.

motohagiography 5y ago

Naive question: how is looting vulnerable smart contracts even illegal? Without a legal framework of smart contract enforcement, recognition of literally-hypothetical assets as valuable, the public nature of blockchains that would preclude "unauthorized access," and unlike an exchange holding assets on behalf of customers - smart contracts are effectively leaving money on the ground for anyone clever enough to pick it up. Clearly I haven't given it as much thought as the people involved, but it seems like if I'm not using my abilities full-time to hack and loot smart contracts, I'm missing the most direct and best possible effort/reward application of that kind of skill.

jude- 5y ago
IANAL, but I'd imagine that for US folks, violating the intent of the code -- namely, doing something while lacking the authorization to do so -- is illegal, even if the code lets you do it. Otherwise, basically all forms of criminal hacking would be legal. Of course, if the smart contract expressly permitted anyone to take tokens out of it via any means allowed by the platform, that's a different story.
- TheRealPomax 5y ago
  Except in the US you can't sue on just intent, the letter of the law still needs to agree. So if the smart contract permits something, by not forbidding it, that something is entirely legal.
  - jude- 5y ago
    That can't be right. That would mean that all forms of hacking are legal, since after all, the remote server permitted you to run your shell code.
- jcranmer 5y ago
  The law you're thinking of is CFAA, and it's not clear that this violates the CFAA. The requisite element is "accesses [...] without authorization or exceeds authorized access." Given the general enthusiasm of blockchain proponents to believe that "the code is law", it's a pretty easy argument to make that taking advantage of poorly-written code is well within the user's authorized capabilities. Will it win in court? shrug

ramimac 5y ago

I would highly recommend anyone interested in pentesting smart contracts look at the work Trail of Bits has been putting out: https://blog.trailofbits.com/?s=smart+contract&submit=Search https://blog.trailofbits.com/?s=smart+contract&submit=Search Start with "246 Findings From our Smart Contract Audits: An Executive Summary" [1] [1] https://blog.trailofbits.com/2019/08/08/246-findings-from-our-smart-contract-audits-an-executive-summary/ https://blog.trailofbits.com/2019/08/08/246-findings-from-ou...

Anointmous 5y ago

I hate the term "pentest", but apparently people who want lingo over the ability to do anything have won out over the decades. Besides being a meaningless inaccurate shortening of the phrase, an actually "pen test" would be part of putting a pen register on a phone. It just indicates that the newbies who created the term didn't know anything before.

ketamine__ 5y ago

Was the issue with Fei actually a bug?

rob-olmos 5y ago

I've been interested in any smart contract languages/VMs that are somehow more capable of being provably correct/secure. The only one I've come across is Kadena, which internally uses the Z3 prover, but I haven't looked into the source code in depth or if it's able to be applied to custom smart contracts (dApp) as well. Are there other blockchains that are similar? Is there a strict subset and prover for Solidity or other languages? Or things like proven smart contract kernels that can be built on top of? Eg, OpenZeppelin Contracts, but with provers rather than only audits.

jude- 5y ago
Disclaimer: I work on this. You should check out Clarity (clarity-lang.org). It's an on-chain interpreted language, so what you see is what will run. It's statically typed and decidable, such that you can reason about all the halting states of the program as well as the upper bounds on the amount of computing resources used. It's used by the Stacks blockchain and Algorand.
dguido 5y ago
Ugh, I have been advocating "Solidity--" for years and can't get funding to build it (Trail of Bits). We use two tools to offer quick turnaround automated testing and verification for Solidity: Echidna (like QuickCheck for Solidity) and Manticore (a symbolic verifier). They each let you write high level properties in the span of 1-2 weeks that cover a large amount of potential use cases. Here's an example of what that looks like: https://github.com/trailofbits/publications/blob/master/reviews/Liquity.pdf https://github.com/trailofbits/publications/blob/master/revi... Here's Echidna: https://github.com/crytic/echidna https://github.com/crytic/echidna and Manticore: https://github.com/trailofbits/manticore https://github.com/trailofbits/manticore Sometimes we also use custom static analyses built around Slither's IR during projects too: https://github.com/crytic/slither/wiki/SlithIR https://github.com/crytic/slither/wiki/SlithIR