4 ms·

Hi, Postgres.ai founder here. Production is not affected, the chatbot works with thin clones provided by Database Lab https://gitlab.com/postgres-ai/database-l

Hi, Postgres.ai founder here. Production is not affected, the chatbot works with thin clones provided by Database Lab https://gitlab.com/postgres-ai/database-lab/ https://gitlab.com/postgres-ai/database-lab/. In general, you are right. If, in your case, using the "raw" databases is unacceptable, it is better to anonymize data first (of course, the physical layout will change in this case). However, Joe doesn't reveal the data. Engineers see only EXPLAIN (ANALYZE, BUFFERS) plans. It may be considered acceptable. Of course, there are ways to retrieve particular values with some tricks like "explain analyze ... limit (select value from...)"), but this will be noticeable. Massive data leaks are not possible, Joe filters the output, presenting only metadata -- the whole idea was to provide the data for backend engineers, many of those don't have access to production data. The question you're raising is broader: let's think how many times we send some personal data to our colleagues in Slack, how is it controlled now? are we okay with that? Emails, tokens, and so on. At the same time, Joe does boost the development speed because it becomes easier to troubleshoot SQL performance and discuss it with colleagues, collecting reliable facts. What would you say if it would be: 1. A chatbot for on-prem Mattermost? 2. A specialized GUI, SaaS or standalone?

diminish 7y ago
Joe sounds great. Theoretically you could reveal certain type information by writing specialized attack queries for checking certain information, for example: WHERE username='joe' AND is_admin IS true or WHERE salary > 100000
ysavir 7y ago
> The question you're raising is broader: let's think how many times we send some personal data to our colleagues in Slack, how is it controlled now? are we okay with that? Emails, tokens, and so on. People's existing tendency to expose critical data via Slack shouldn't be reinforced, and coming from a product that handles sensitive data by design, it's disheartening to see this line of thinking here. I would hope your product makes it harder for me to compromise data, not easier. > At the same time, Joe does boost the development speed because it becomes easier to troubleshoot SQL performance and discuss it with colleagues, collecting reliable facts. Great! Now make that troubleshooting accessible via an authenticated URL and I'm in. Especially because I may want to revisit a particular explain sometime down the line, and simple anchor-tag based list of bookmarkable past queries is easier to navigate than having to use Slack searches. If I were to use this tool, I would envision the ideal version of it being self-hosted and able to function independent from any other service. If there was then an option to add a Slack hook that notified people, that would be good. But it shouldn't be central to the product, and the information should strictly travel from the self-hosted service to slack via hooks, exposing only the URLs/titles of result pages. No need to expose the queries or DB structure. I think the current Slack implementation is a cool proof of concept, but this isn't a product I would seriously consider until it is able to perform its tasks in isolation. As for "improv[ing] the level of collaboration of developers and DBAs", great that you're thinking about that, but that's my problem to solve, not Joe's. And depending on my team's existing solutions, this may even rule out Joe simply due to our current tool set.
- samokhvalov 7y ago
  Thank you for very detailed feedback!! > People's existing tendency to expose critical data via Slack shouldn't be reinforced I agree with you. However, depending on company's policy, raw SQL, even with some concrete values, and EXPLAIN ANALYZE plans may be considered as acceptable for using them in Slack. If it is considered harmful, we have a way to inject anonymization step into the snapshot preparation tool -- see https://gitlab.com/postgres-ai/database-lab/-/blob/master/scripts/create_zfs_snapshot.sh#L173 https://gitlab.com/postgres-ai/database-lab/-/blob/master/sc... (we're thinking how to improve usability here, not losing flexibility). For example, you can run PostgreSQL Anonymizer (https://gitlab.com/dalibo/postgresql_anonymizer https://gitlab.com/dalibo/postgresql_anonymizer). Of course, the physical layout will change, so troubleshooting of, say, VACUUM commands would be affected. But rows cardinality, normally, won't change, so for large enough tables, it totally makes sense do go this route, for SQL troubleshooting&optimization tasks. > Great! Now make that troubleshooting accessible via an authenticated URL and I'm in. We are thinking about the GUI version actively, so stay tuned. And on CLI version as well, but CLI will work in psql (which I'm personally a big fan of, spent 15 years there or so). This means that all the data will be present. In the case of GUI, of course, we are able to work on meta-level, similarly to the Slack version. > If I were to use this tool, I would envision the ideal version of it being self-hosted and able to function independent from any other service. I hear you well, and it generally aligns with our team's vision. Thank you again for awesome feedback! > As for "improv[ing] the level of collaboration of developers and DBAs", great that you're thinking about that, but that's my problem to solve, not Joe's. And depending on my team's existing solutions, this may even rule out Joe simply due to our current tool set. Could you provide some examples of such solutions?
  - lmm 7y ago
    > Could you provide some examples of such solutions? Well, if we're already doing our collaborating via Discord or Microsoft Teams or whatever then a Slack-based tool is at a significant disadvantage.

diminish 7y ago

Joe sounds great. Theoretically you could reveal certain type information by writing specialized attack queries for checking certain information, for example: WHERE username='joe' AND is_admin IS true or WHERE salary > 100000

ysavir 7y ago

> The question you're raising is broader: let's think how many times we send some personal data to our colleagues in Slack, how is it controlled now? are we okay with that? Emails, tokens, and so on. People's existing tendency to expose critical data via Slack shouldn't be reinforced, and coming from a product that handles sensitive data by design, it's disheartening to see this line of thinking here. I would hope your product makes it harder for me to compromise data, not easier. > At the same time, Joe does boost the development speed because it becomes easier to troubleshoot SQL performance and discuss it with colleagues, collecting reliable facts. Great! Now make that troubleshooting accessible via an authenticated URL and I'm in. Especially because I may want to revisit a particular explain sometime down the line, and simple anchor-tag based list of bookmarkable past queries is easier to navigate than having to use Slack searches. If I were to use this tool, I would envision the ideal version of it being self-hosted and able to function independent from any other service. If there was then an option to add a Slack hook that notified people, that would be good. But it shouldn't be central to the product, and the information should strictly travel from the self-hosted service to slack via hooks, exposing only the URLs/titles of result pages. No need to expose the queries or DB structure. I think the current Slack implementation is a cool proof of concept, but this isn't a product I would seriously consider until it is able to perform its tasks in isolation. As for "improv[ing] the level of collaboration of developers and DBAs", great that you're thinking about that, but that's my problem to solve, not Joe's. And depending on my team's existing solutions, this may even rule out Joe simply due to our current tool set.

samokhvalov 7y ago
Thank you for very detailed feedback!! > People's existing tendency to expose critical data via Slack shouldn't be reinforced I agree with you. However, depending on company's policy, raw SQL, even with some concrete values, and EXPLAIN ANALYZE plans may be considered as acceptable for using them in Slack. If it is considered harmful, we have a way to inject anonymization step into the snapshot preparation tool -- see https://gitlab.com/postgres-ai/database-lab/-/blob/master/scripts/create_zfs_snapshot.sh#L173 https://gitlab.com/postgres-ai/database-lab/-/blob/master/sc... (we're thinking how to improve usability here, not losing flexibility). For example, you can run PostgreSQL Anonymizer (https://gitlab.com/dalibo/postgresql_anonymizer https://gitlab.com/dalibo/postgresql_anonymizer). Of course, the physical layout will change, so troubleshooting of, say, VACUUM commands would be affected. But rows cardinality, normally, won't change, so for large enough tables, it totally makes sense do go this route, for SQL troubleshooting&optimization tasks. > Great! Now make that troubleshooting accessible via an authenticated URL and I'm in. We are thinking about the GUI version actively, so stay tuned. And on CLI version as well, but CLI will work in psql (which I'm personally a big fan of, spent 15 years there or so). This means that all the data will be present. In the case of GUI, of course, we are able to work on meta-level, similarly to the Slack version. > If I were to use this tool, I would envision the ideal version of it being self-hosted and able to function independent from any other service. I hear you well, and it generally aligns with our team's vision. Thank you again for awesome feedback! > As for "improv[ing] the level of collaboration of developers and DBAs", great that you're thinking about that, but that's my problem to solve, not Joe's. And depending on my team's existing solutions, this may even rule out Joe simply due to our current tool set. Could you provide some examples of such solutions?
- lmm 7y ago
  > Could you provide some examples of such solutions? Well, if we're already doing our collaborating via Discord or Microsoft Teams or whatever then a Slack-based tool is at a significant disadvantage.