5 ms·

Does anyone else find it a bit off-putting how much emphasis they keep putting on "open source library"? I don't think I've read about this without the word ope

by jchw 4y ago

Does anyone else find it a bit off-putting how much emphasis they keep putting on "open source library"? I don't think I've read about this without the word open source appearing more than once in their own messaging about it. Why is it so important to emphasize that the library with the bug is open source? The cynic in me wants to believe that it's a way of deflecting blame somehow, to make it seem like they did their due diligence but were thwarted by something outside of their control. I don't think it holds. If you use an open source library with no warranty, you are responsible (legally and otherwise) to ensure that it is sufficient. For example, if you break HIPAA compliance due to an open source library, it is still you who is responsible for that. But of course, they're not claiming it's anyone else's fault anywhere explicitly, so it's uncharitable to just assume that's what they meant. Still, it rubs me the wrong way. I can't fight the feeling that it's a wink wink nudge nudge to give them more slack than they'd otherwise get. It feels like it's inviting you to just criticize redis-py and give them a break. The open postmortem and whatnot is appreciated and everything, but sometimes it's important to be mindful of what you emphasize in your postmortems. People read things even if you don't write them, sometimes.

bitxbitxbitcoin 4y ago
Not surprising from a company that calls itself openai. The “open source” keyword stuffing is so people associate the open from openai with open source. Psyops I mean marketing 101.
amtamt 4y ago
Was postmortem generated by chatGPT?
dilap 4y ago
I half agree, but I also half-sympathize with them, because it really wasn't their fault -- it was a quite-bad bug in a very fundamental library. Bugs happen, though. Especially in Python.
- airstrike 4y ago
  > Especially in Python. as opposed to...?
  - moffkalast 4y ago
    As opposed to not in Python.
    - deathanatos 4y ago
      … like JavaScript? Bash? C? PHP? Certainly none of those are widely used and have a reputation for making it easy to keep the gun aimed squarely at the foot.
      - moffkalast 4y ago
        Those would be roughly similar. The main difference would be between dynamically typed interpreted languages and statically typed compiled ones I guess. At least I think I make less mistakes when the compiler literally tells me what's wrong before I even run the thing. It's awful and slow to develop that way, but it is more reliable for when that's a requirement. So compared to ones like Kotlin or Rust.
  - dilap 4y ago
    Go, for one. In my experience errors are more common (for both cultural and technological reasons) in Python than in Go. I would guess something similar applies to Rust, though I don't have personal experience. There's wide variation in C, but with careful discrimination, you can find very high-quality libraries or software (redis itself being an excellent example). I don't have rigourous data to baack this stuff up, but I'm pretty convinced it's true, based on my own experience.
- kljhghfgdfjkgh 4y ago
  it really was their fault. they chose to ship the bug. it doesn't matter in the last that someone else previously published the code under a license with no warranty whatsoever.
adrianmonk 4y ago
I noticed it too, but it doesn't necessarily bother me. Possibly they're just trying to say, "This incident may have made us look like we're complete amateurs who don't have any clue about security, but it wasn't like that." Using someone else's library doesn't absolve you of responsibility, but failing to be vigilant at thoroughly vetting and testing external dependencies is a different kind of mistake than creating a terrible security bug yourself because your engineers don't know how to code or everyone is in too much of a rush to care about anything.
- jvm___ 4y ago
  Doesn't bother me either. All the car companies issue recalls regularly, sometimes an issue only shows up when the system hits capacity or you run into an edge case.
- mewpmewp2 4y ago
  Yes, I agree with that sentiment, and I thought precisely the same. I know as an engineer that I would feel compelled to mention that it was an obscure bug in an open source library, if that was the case. Not to excuse myself of responsibility, but because I would feel so ashamed if I myself introduced such an obvious security flaw. I would still of course consider myself responsible for what happened. A lot of the time when people make mistakes, they explain themselves so as they are afraid to be perceived as completely stupid or incompetent for making that mistake, not excusing themselves of taking responsibility even though people frequently think that excuses or explanation means that you are trying to absolve yourself of what you did. There's a huge difference to me between having an obscure bug like this and introducing that type of security issue because you couldn't logically consider it. First one can be resolved in the future by introducing processes and make sure all open source libraries are from trusted sources, but second one implies that you are fundamentally unable to think and therefore also probably improve on that.
- mlsu 4y ago
  Why? The result for the end consumer is identical whether they have their PII leaked from "an external library" vs a vendor's own home-baked solution. It's not really a different kind of mistake, it's exactly the same kind of mistake, because it is exactly the same mistake! This is talking the talk, and not walking the walk, when it comes to security. Publishing a writeup that passes the buck to some (unnamed) overworked and underpaid open source maintainer is worse, not better!
skybrian 4y ago
I think you’re reading too much into it. Being an open source library is relevant because it means it’s third party and doesn’t come with a support agreement, so fixing a bug is a somewhat different process than if it were in your own code or from a proprietary vendor. Yes, it’s technically up to you to vet all your dependencies, but in practice, often it doesn’t happen, people make assumptions that the code works, and that’s relevant too.
- JohnFen 4y ago
  > in practice, often it doesn’t happen, people make assumptions that the code works True, but that's an inexcusable practice and always has been. We as an industry need to stop accepting it.
  - isopede 4y ago
    What do you mean by "stop accepting it?" All of us rely on millions of lines of code that we have not personally audited every single day. Have you audited every framework you use? Your kernel? Drivers? Your compiler? Your CPU microcode? Your bootrom? The firmware in every gizmo you own? If "Reflections on Trusting Trust" has taught us anything, it's turtles all the way down. At some point, you have to either trust something, or abandon all hope and trust nothing.
    - JohnFen 4y ago
      > Have you audited every framework you use? Your compiler? Your CPU microcode? Your bootrom? Of course not. I exclude the CPU microcode, bootrom, and the like from the discussion because that's not part of the product being shipped. But it's also true that I don't do a deep dive analyzing every library I use, etc. I'm not saying that we should have to. What I'm saying is that when a bug pops up, that's on us as developers even when the bug is in a library, the compiler, etc. A lot of developers seem to think that just because the bug was in code they didn't personally write, that means that their hands are clean. That's just not a viable stance to take. The bug should have been caught in testing, after all. If your car breaks down because of a design failure in a component the auto manufacturer bought from another supplier, you'll still (rightfully) hold the auto manufacturer responsible.
JohnFen 4y ago
> The cynic in me wants to believe that it's a way of deflecting blame somehow That's how it reads to me as well. Of course, it doesn't deflect blame at all. Any time you include code in your project, no matter where the code came from, you are responsible for the behavior of that code.
JamesBarney 4y ago
This doesn't come across this way to me at all. They just described what happened. Do you expect them to jump in front of a bus for the library they're using, and beg for forgiveness for not ensuring the widely used libraries they're leveraging are bug free? There are very few companies that couldn't get caught by this type of bug.
nickvincent 4y ago
Basically agree -- feels off-putting, but not technically a wrong detail to add. An additional reason it rubs me the wrong way, however, is that I believe open-source software code is especially critical to ChatGPT family's capabilities. Not just for code-related queries, but for everything! (e.g. see this "lineage-tracing" blog post: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1#3e764cf66cd84fd481fb166ec43188ca https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...) Thus, I honestly think firms operating generative AI should be walking on eggshells to avoid placing blame on "open-source". Rather, they really should going out of their way to channel as much positive energy towards it as possible. Still, agree the charitable interpretation is that this just purely descriptive.
ishaanbahal 4y ago
The emphasis could also have been done to educate folks using this combination to check their setup. Though version reference in the postmartem should also be posted, as a general guidance to their own readers, but at least a quick google search leads you to it. https://github.com/redis/redis-py/releases/tag/v4.5.3 https://github.com/redis/redis-py/releases/tag/v4.5.3 For anyone reading this and using a combination of asyncio and py-redis, please bump your versions. Similar issues I've encountered with asyncio python and postgres too in the past when trying to pool connections. It's really not easy to debug them either.
layer8 4y ago
I think you’re overreacting. What bothered me is that they didn’t link to the actual bug or provide a reference ID.
babl-yc 4y ago
I don't find it over-emphasized. Many in the Twitter-sphere are acting as if they aren't being appreciative of open source software and I don't see it that way. The technical root cause was in the open source library. There's a patch available and more likely than not OpenAI will continue to use the library. Being overly sensitive to blame would be distracting to the technical issue at hand. It's great they are posting this post-mortem to raise awareness that the libraries you use can have bugs and to consider that risk when building systems.
- fabianhjr 4y ago
  Root cause analysis would likely also include the lack of threat modeling / security evaluation of their dependencies Would likely also question the lack of resources allocated to these open source projects by companies with a lot of profits from, in part, using those open source projects.
kobalsky 4y ago
the library is provided by the redis team themselves and the bug is awful [1]. I know it's not redis' fault but this bug could hit anyone. Connections may be left in a dirty state where they return data from the previous request in the following one. [1] https://github.com/redis/redis-py/issues/2624 https://github.com/redis/redis-py/issues/2624
thequadehunter 4y ago
Personally, I think it was partially a virtue signal to show that they use open source software and collaborate with the maintainers.
mlsu 4y ago
The gaping hole in this write-up goes something like: "In order to prevent a bug like this from happening in the future, we have stepped up our review process for external dependencies. In addition, we are conducting audits around code that involves sensitive information." Of course, we all know what actually happened here: - we did no auditing; - because our audit process consists of "blame someone else when our consumers are harmed"; - because we would rather not waste dev time on making sure our consumers are not harmed If you want to know why no software "engineering" is happening here, this is your answer. Can you imagine if a bridge collapsed, and the builder of the bridge said, "iunno, it's the truck's fault for driving over the bridge."
- marshmellman 4y ago
  Are you confident that an audit would have uncovered this bug? I’d be surprised if audits are effective at finding subtle bugs and race conditions, but I could be wrong.
  - hobobaggins 4y ago
    Depends on the type of audit. Subtle bugs are often uncovered by fuzzing, for example, but a race condition might not be found without substantial load.
- JW_00000 4y ago
  When a bridge collapses, everyone does try to shift blame onto someone else :)
chamakits 4y ago
I've also noticed it, and I can't help but interpret it as their way of shifting blame. Which is irresponsible. It's their product, and they need to take accountability for the bug occurring. It's a serious bug, but in the grand scheme of things, not earth shattering, and not something that I think would discourage usage of their product. But their treatment of the bug causes more concerns than the bug itself. They are shifting the blame away from the work they did using a library with a bug, rather than their process by which that library made it into their product. And I don't understand how they can't see how that reflects poorly on them as an AI company. I find it so confusing that at the end of the day, OpenAI's biggest product is having created a good process by which to create value out of a massive amount of data, and build a good API on top of it. And the open source library is effectively something they processed into their product and built an API based off of it. So it creates (to me) some amount of doubt about how they will react when faced with similar challenges to their core product. How will they behave when the data they consume impacts their product negatively? From limited experience, they'll shift the blame to the data, not their process, and keep it pushing. It seems likely that this is only the beginning of OpenAI having a large customer base, with a high impact on many products. This is a disappointing result on their first test on how they'll manage issues and bugs with their products.
cwkoss 4y ago
If the FTC had teeth and good judgement, they'd force OpenAI to rename themselves.
jatins 4y ago
I think you are reading a bit between the lines, and didn't feel them blaming the library as much as stating that the bug happened because of an issue in the library. Maybe they could have sugarcoated it between 10 layers of corporate jargon but I'd rather take this over that
metanonsense 4y ago
I don't know. To me it's simply an explanation of what has happened. I think its exactly what I would have written if I was in their position. And show me the one company that has audited all source code of all used open source projects, at least in a way that is able to rule out complex bugs like this. I have once found a memory corruption bug in Berkeley DB wrecking our huge production database, which I would have never found in any pre-emptive source code audit, however detailed. Edit: On second thought, maybe they could have just written "external library" instead of "open source library".

by jchw 4y ago

bitxbitxbitcoin 4y ago

Not surprising from a company that calls itself openai. The “open source” keyword stuffing is so people associate the open from openai with open source. Psyops I mean marketing 101.

amtamt 4y ago

Was postmortem generated by chatGPT?

dilap 4y ago

I half agree, but I also half-sympathize with them, because it really wasn't their fault -- it was a quite-bad bug in a very fundamental library. Bugs happen, though. Especially in Python.

airstrike 4y ago
> Especially in Python. as opposed to...?
- moffkalast 4y ago
  As opposed to not in Python.
  - deathanatos 4y ago
    … like JavaScript? Bash? C? PHP? Certainly none of those are widely used and have a reputation for making it easy to keep the gun aimed squarely at the foot.
    - moffkalast 4y ago
      Those would be roughly similar. The main difference would be between dynamically typed interpreted languages and statically typed compiled ones I guess. At least I think I make less mistakes when the compiler literally tells me what's wrong before I even run the thing. It's awful and slow to develop that way, but it is more reliable for when that's a requirement. So compared to ones like Kotlin or Rust.
- dilap 4y ago
  Go, for one. In my experience errors are more common (for both cultural and technological reasons) in Python than in Go. I would guess something similar applies to Rust, though I don't have personal experience. There's wide variation in C, but with careful discrimination, you can find very high-quality libraries or software (redis itself being an excellent example). I don't have rigourous data to baack this stuff up, but I'm pretty convinced it's true, based on my own experience.
kljhghfgdfjkgh 4y ago
it really was their fault. they chose to ship the bug. it doesn't matter in the last that someone else previously published the code under a license with no warranty whatsoever.

adrianmonk 4y ago

I noticed it too, but it doesn't necessarily bother me. Possibly they're just trying to say, "This incident may have made us look like we're complete amateurs who don't have any clue about security, but it wasn't like that." Using someone else's library doesn't absolve you of responsibility, but failing to be vigilant at thoroughly vetting and testing external dependencies is a different kind of mistake than creating a terrible security bug yourself because your engineers don't know how to code or everyone is in too much of a rush to care about anything.

jvm___ 4y ago
Doesn't bother me either. All the car companies issue recalls regularly, sometimes an issue only shows up when the system hits capacity or you run into an edge case.
mewpmewp2 4y ago
Yes, I agree with that sentiment, and I thought precisely the same. I know as an engineer that I would feel compelled to mention that it was an obscure bug in an open source library, if that was the case. Not to excuse myself of responsibility, but because I would feel so ashamed if I myself introduced such an obvious security flaw. I would still of course consider myself responsible for what happened. A lot of the time when people make mistakes, they explain themselves so as they are afraid to be perceived as completely stupid or incompetent for making that mistake, not excusing themselves of taking responsibility even though people frequently think that excuses or explanation means that you are trying to absolve yourself of what you did. There's a huge difference to me between having an obscure bug like this and introducing that type of security issue because you couldn't logically consider it. First one can be resolved in the future by introducing processes and make sure all open source libraries are from trusted sources, but second one implies that you are fundamentally unable to think and therefore also probably improve on that.
mlsu 4y ago
Why? The result for the end consumer is identical whether they have their PII leaked from "an external library" vs a vendor's own home-baked solution. It's not really a different kind of mistake, it's exactly the same kind of mistake, because it is exactly the same mistake! This is talking the talk, and not walking the walk, when it comes to security. Publishing a writeup that passes the buck to some (unnamed) overworked and underpaid open source maintainer is worse, not better!

skybrian 4y ago

I think you’re reading too much into it. Being an open source library is relevant because it means it’s third party and doesn’t come with a support agreement, so fixing a bug is a somewhat different process than if it were in your own code or from a proprietary vendor. Yes, it’s technically up to you to vet all your dependencies, but in practice, often it doesn’t happen, people make assumptions that the code works, and that’s relevant too.

JohnFen 4y ago
> in practice, often it doesn’t happen, people make assumptions that the code works True, but that's an inexcusable practice and always has been. We as an industry need to stop accepting it.
- isopede 4y ago
  What do you mean by "stop accepting it?" All of us rely on millions of lines of code that we have not personally audited every single day. Have you audited every framework you use? Your kernel? Drivers? Your compiler? Your CPU microcode? Your bootrom? The firmware in every gizmo you own? If "Reflections on Trusting Trust" has taught us anything, it's turtles all the way down. At some point, you have to either trust something, or abandon all hope and trust nothing.
  - JohnFen 4y ago
    > Have you audited every framework you use? Your compiler? Your CPU microcode? Your bootrom? Of course not. I exclude the CPU microcode, bootrom, and the like from the discussion because that's not part of the product being shipped. But it's also true that I don't do a deep dive analyzing every library I use, etc. I'm not saying that we should have to. What I'm saying is that when a bug pops up, that's on us as developers even when the bug is in a library, the compiler, etc. A lot of developers seem to think that just because the bug was in code they didn't personally write, that means that their hands are clean. That's just not a viable stance to take. The bug should have been caught in testing, after all. If your car breaks down because of a design failure in a component the auto manufacturer bought from another supplier, you'll still (rightfully) hold the auto manufacturer responsible.

JohnFen 4y ago

> The cynic in me wants to believe that it's a way of deflecting blame somehow That's how it reads to me as well. Of course, it doesn't deflect blame at all. Any time you include code in your project, no matter where the code came from, you are responsible for the behavior of that code.

JamesBarney 4y ago

This doesn't come across this way to me at all. They just described what happened. Do you expect them to jump in front of a bus for the library they're using, and beg for forgiveness for not ensuring the widely used libraries they're leveraging are bug free? There are very few companies that couldn't get caught by this type of bug.

nickvincent 4y ago

Basically agree -- feels off-putting, but not technically a wrong detail to add. An additional reason it rubs me the wrong way, however, is that I believe open-source software code is especially critical to ChatGPT family's capabilities. Not just for code-related queries, but for everything! (e.g. see this "lineage-tracing" blog post: https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1#3e764cf66cd84fd481fb166ec43188ca https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...) Thus, I honestly think firms operating generative AI should be walking on eggshells to avoid placing blame on "open-source". Rather, they really should going out of their way to channel as much positive energy towards it as possible. Still, agree the charitable interpretation is that this just purely descriptive.

ishaanbahal 4y ago

The emphasis could also have been done to educate folks using this combination to check their setup. Though version reference in the postmartem should also be posted, as a general guidance to their own readers, but at least a quick google search leads you to it. https://github.com/redis/redis-py/releases/tag/v4.5.3 https://github.com/redis/redis-py/releases/tag/v4.5.3 For anyone reading this and using a combination of asyncio and py-redis, please bump your versions. Similar issues I've encountered with asyncio python and postgres too in the past when trying to pool connections. It's really not easy to debug them either.

layer8 4y ago

I think you’re overreacting. What bothered me is that they didn’t link to the actual bug or provide a reference ID.

babl-yc 4y ago

I don't find it over-emphasized. Many in the Twitter-sphere are acting as if they aren't being appreciative of open source software and I don't see it that way. The technical root cause was in the open source library. There's a patch available and more likely than not OpenAI will continue to use the library. Being overly sensitive to blame would be distracting to the technical issue at hand. It's great they are posting this post-mortem to raise awareness that the libraries you use can have bugs and to consider that risk when building systems.

fabianhjr 4y ago
Root cause analysis would likely also include the lack of threat modeling / security evaluation of their dependencies Would likely also question the lack of resources allocated to these open source projects by companies with a lot of profits from, in part, using those open source projects.

kobalsky 4y ago

the library is provided by the redis team themselves and the bug is awful [1]. I know it's not redis' fault but this bug could hit anyone. Connections may be left in a dirty state where they return data from the previous request in the following one. [1] https://github.com/redis/redis-py/issues/2624 https://github.com/redis/redis-py/issues/2624

thequadehunter 4y ago

Personally, I think it was partially a virtue signal to show that they use open source software and collaborate with the maintainers.

mlsu 4y ago

The gaping hole in this write-up goes something like: "In order to prevent a bug like this from happening in the future, we have stepped up our review process for external dependencies. In addition, we are conducting audits around code that involves sensitive information." Of course, we all know what actually happened here: - we did no auditing; - because our audit process consists of "blame someone else when our consumers are harmed"; - because we would rather not waste dev time on making sure our consumers are not harmed If you want to know why no software "engineering" is happening here, this is your answer. Can you imagine if a bridge collapsed, and the builder of the bridge said, "iunno, it's the truck's fault for driving over the bridge."

marshmellman 4y ago
Are you confident that an audit would have uncovered this bug? I’d be surprised if audits are effective at finding subtle bugs and race conditions, but I could be wrong.
- hobobaggins 4y ago
  Depends on the type of audit. Subtle bugs are often uncovered by fuzzing, for example, but a race condition might not be found without substantial load.
JW_00000 4y ago
When a bridge collapses, everyone does try to shift blame onto someone else :)

chamakits 4y ago

I've also noticed it, and I can't help but interpret it as their way of shifting blame. Which is irresponsible. It's their product, and they need to take accountability for the bug occurring. It's a serious bug, but in the grand scheme of things, not earth shattering, and not something that I think would discourage usage of their product. But their treatment of the bug causes more concerns than the bug itself. They are shifting the blame away from the work they did using a library with a bug, rather than their process by which that library made it into their product. And I don't understand how they can't see how that reflects poorly on them as an AI company. I find it so confusing that at the end of the day, OpenAI's biggest product is having created a good process by which to create value out of a massive amount of data, and build a good API on top of it. And the open source library is effectively something they processed into their product and built an API based off of it. So it creates (to me) some amount of doubt about how they will react when faced with similar challenges to their core product. How will they behave when the data they consume impacts their product negatively? From limited experience, they'll shift the blame to the data, not their process, and keep it pushing. It seems likely that this is only the beginning of OpenAI having a large customer base, with a high impact on many products. This is a disappointing result on their first test on how they'll manage issues and bugs with their products.

cwkoss 4y ago

If the FTC had teeth and good judgement, they'd force OpenAI to rename themselves.

jatins 4y ago

I think you are reading a bit between the lines, and didn't feel them blaming the library as much as stating that the bug happened because of an issue in the library. Maybe they could have sugarcoated it between 10 layers of corporate jargon but I'd rather take this over that

metanonsense 4y ago

I don't know. To me it's simply an explanation of what has happened. I think its exactly what I would have written if I was in their position. And show me the one company that has audited all source code of all used open source projects, at least in a way that is able to rule out complex bugs like this. I have once found a memory corruption bug in Berkeley DB wrecking our huge production database, which I would have never found in any pre-emptive source code audit, however detailed. Edit: On second thought, maybe they could have just written "external library" instead of "open source library".