5 ms·

> Now, all of this might have been fine had Beyond Trust not written a feature which allowed users to directly, programmatically interact with psql (the postgre

> Now, all of this might have been fine had Beyond Trust not written a feature which allowed users to directly, programmatically interact with psql (the postgres command line interface). That's the buried lede. Yes, there was a vulnerability in psql... but that's so much less a problem than the huge gaping hole of allowing users to directly interact with psql. No DB can be safe if you are turning untrusted user commands into psql executions. It'd be like giving untrusted users ssh access and then complaining when they find a privilege elevation exploit.

zoeysmithe 2y ago
Just a guess but this looks like politically powerful dev culture overwriting cybersecurity culture, demanding, thus getting an exception from management for 'productivity' and 'being agile.' I dont think we appreciate how much of a wild west things are with the incredible mix of hugely complex and powerful tools available trivially to developers and the concept of "move fast, break things." Especially as corporate sees devs like they see salesmen (big moneymakers who deserve perks, exceptions) and top-down security culture as a cost center. The other buried ledes are that postgres allows emojis (not sure if that's intended but it works) and that you can just run system commands and scripts directly from postgres cli. I imagine a lot of eyes are going to be on new hardening guidelines for postgres now. I also imagine the first high performance enterprise friendly drop-in db written in something like rust is going to one day be a big deal.
- Terr_ 2y ago
  Hey now, a large portion of developers are seen as cost-centers too! Not everybody has the skill of flattering managers into approving greenfield projects, and then transferring away before they break horribly. :p
- randmeerkat 2y ago
  > Especially as corporate sees devs like they see salesmen… You’re onto something here. People perceive the world through the lens of their education and environment. Sales, legal, finance, are all easy constructs for a business leader to view the rest of the world through. The secret of the game isn’t to have the best tech or to code the most, it’s to “outsell” your competing business unit.
0xbadcafebee 2y ago
The only reason BeyondTrust implemented that was it wasn't untrusted user commands. They sanitized the data, so it should have been fine. The unfortunate problem was that the sanitizer didn't sanitize. Systems are built on a set of expectations. Undermine the expectations and you undermine the system.
- cogman10 2y ago
  > They sanitized the data, so it should have been fine. This is a 101 rookie level approach to SQL or injection defense. It's dumb for exactly the same reason why this is dumb "SELECT * FROM foo WHERE bar=" + sanitize(userInput) The correct way to do something like this will always be parameterized input which looks something like this "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); Why? Because that the postgres protocol splits out the command and the data for the command in a way that can't be injected. Something that should be viewed as impossible to do when data and command are merged into 1 String. IF this company wanted to build dynamic queries, then the only correct way to do that is to limit input to only valid variables. IE "isValidColumnName(userInput)" before sending the request. And even then, you'd not use psql to do that. You simply can't use a generalized sanitizer and expect good results.
  - Nijikokun 2y ago
    His argument is that you are building on the assumption that this is safe: "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); But what happens when it turns out that isn't safe?
    - jerf 2y ago
      Then you, or someone else, fix the underlying library. This being Postgres, that process was likely completed decades ago. Anticipating the next question, "but what if it is still unsafe?", the answer is that there is no fundamental reason why this can't be safe code. Security exploits like this require a cooperation between the receiving code and the incoming data, and it is in fact perfectly possible for the receiving code to be completely safe. There is no such thing as data just so amazingly hackerish that it can blast through all protections. There has to be a hole, and this is among the best-tested and most scrutinized code paths in the world.
gowld 2y ago
Let's be clear: Beyond Trust is not a company that wrote a database-backed web app and made the all-too-common mistake of writing insecure code that tickled a bug in the database that allowed privilege escalation. Beyond Trust's is a company whose entire contribution is adding a security layer to prevent privilege escalation, and their solution here was to bypass Postgres's standard functionality and use this weird `psql` hack instead. They had one job, and they failed at it. This amateur-level mistake should sink the entire company.
- 1oooqooq 2y ago
  but they got the right sales people to get to the IRS and they hired the "right" (wrong) certification company. so that's three jobs at least. we now have one job to ask for accountability and will not do it.
- pjc50 2y ago
  Didn't happen when Crowdstrike broke all their customers. The problem is that getting information security right is a matter of process control, which everyone hates, and so CEOs are absolute suckers for being sold a product which magically "adds on" security. This is like trying to buy "anti-lead-paint" rather than actually remove all your existing lead paint.
singpolyma3 2y ago
Giving untrusted users ssh access... You mean like every shared hosting company or shell provider?
- charcircuit 2y ago
  Which is why shared hosting companies get hacked relatively more.
somat 2y ago
There is an interesting project https://github.com/Abstrct/Schemaverse https://github.com/Abstrct/Schemaverse where the whole game takes place entirely within a postgres database that the players connect directly to. The author discovered quite a few... well lets just call them policy errors in postgres, but had a hard time filing bug reports, mainly because the response was usually an incredulous "why on earth would you even do that in the first place?. But the author has fun with this, There is a trophy in the game that you can only get by putting your name in the trophy table.

zoeysmithe 2y ago

Just a guess but this looks like politically powerful dev culture overwriting cybersecurity culture, demanding, thus getting an exception from management for 'productivity' and 'being agile.' I dont think we appreciate how much of a wild west things are with the incredible mix of hugely complex and powerful tools available trivially to developers and the concept of "move fast, break things." Especially as corporate sees devs like they see salesmen (big moneymakers who deserve perks, exceptions) and top-down security culture as a cost center. The other buried ledes are that postgres allows emojis (not sure if that's intended but it works) and that you can just run system commands and scripts directly from postgres cli. I imagine a lot of eyes are going to be on new hardening guidelines for postgres now. I also imagine the first high performance enterprise friendly drop-in db written in something like rust is going to one day be a big deal.

Terr_ 2y ago
Hey now, a large portion of developers are seen as cost-centers too! Not everybody has the skill of flattering managers into approving greenfield projects, and then transferring away before they break horribly. :p
randmeerkat 2y ago
> Especially as corporate sees devs like they see salesmen… You’re onto something here. People perceive the world through the lens of their education and environment. Sales, legal, finance, are all easy constructs for a business leader to view the rest of the world through. The secret of the game isn’t to have the best tech or to code the most, it’s to “outsell” your competing business unit.

0xbadcafebee 2y ago

The only reason BeyondTrust implemented that was it wasn't untrusted user commands. They sanitized the data, so it should have been fine. The unfortunate problem was that the sanitizer didn't sanitize. Systems are built on a set of expectations. Undermine the expectations and you undermine the system.

cogman10 2y ago
> They sanitized the data, so it should have been fine. This is a 101 rookie level approach to SQL or injection defense. It's dumb for exactly the same reason why this is dumb "SELECT * FROM foo WHERE bar=" + sanitize(userInput) The correct way to do something like this will always be parameterized input which looks something like this "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); Why? Because that the postgres protocol splits out the command and the data for the command in a way that can't be injected. Something that should be viewed as impossible to do when data and command are merged into 1 String. IF this company wanted to build dynamic queries, then the only correct way to do that is to limit input to only valid variables. IE "isValidColumnName(userInput)" before sending the request. And even then, you'd not use psql to do that. You simply can't use a generalized sanitizer and expect good results.
- Nijikokun 2y ago
  His argument is that you are building on the assumption that this is safe: "SELECT * FROM foo WHERE bar=?" bindParameter(1, userInput); But what happens when it turns out that isn't safe?
  - jerf 2y ago
    Then you, or someone else, fix the underlying library. This being Postgres, that process was likely completed decades ago. Anticipating the next question, "but what if it is still unsafe?", the answer is that there is no fundamental reason why this can't be safe code. Security exploits like this require a cooperation between the receiving code and the incoming data, and it is in fact perfectly possible for the receiving code to be completely safe. There is no such thing as data just so amazingly hackerish that it can blast through all protections. There has to be a hole, and this is among the best-tested and most scrutinized code paths in the world.

gowld 2y ago

Let's be clear: Beyond Trust is not a company that wrote a database-backed web app and made the all-too-common mistake of writing insecure code that tickled a bug in the database that allowed privilege escalation. Beyond Trust's is a company whose entire contribution is adding a security layer to prevent privilege escalation, and their solution here was to bypass Postgres's standard functionality and use this weird `psql` hack instead. They had one job, and they failed at it. This amateur-level mistake should sink the entire company.

1oooqooq 2y ago
but they got the right sales people to get to the IRS and they hired the "right" (wrong) certification company. so that's three jobs at least. we now have one job to ask for accountability and will not do it.
pjc50 2y ago
Didn't happen when Crowdstrike broke all their customers. The problem is that getting information security right is a matter of process control, which everyone hates, and so CEOs are absolute suckers for being sold a product which magically "adds on" security. This is like trying to buy "anti-lead-paint" rather than actually remove all your existing lead paint.

singpolyma3 2y ago

Giving untrusted users ssh access... You mean like every shared hosting company or shell provider?

charcircuit 2y ago
Which is why shared hosting companies get hacked relatively more.

somat 2y ago

There is an interesting project https://github.com/Abstrct/Schemaverse https://github.com/Abstrct/Schemaverse where the whole game takes place entirely within a postgres database that the players connect directly to. The author discovered quite a few... well lets just call them policy errors in postgres, but had a hard time filing bug reports, mainly because the response was usually an incredulous "why on earth would you even do that in the first place?. But the author has fun with this, There is a trophy in the game that you can only get by putting your name in the trophy table.