5 ms·

This was a problem with data lakes / analytics. Small inconsistencies would trash runs; that problem now goes away and removes a lot of janky half-fixes to work

• FridgeSeal 6y ago
  Of course the real issue there is those tool treating S3 as something it's not (a filesystem) and building things on it with the expectation of certain guarantees that it explicitly didn't provide before now. I dig the feature (strong consistency) but in some ways it just enables tools that were abusing it to just do so more easily.
  • staticassertion 6y ago
    Like RedShift? AWS uses S3 as a database, seems reasonable that their customers would.
    • FridgeSeal 6y ago
      That doesn't mean it blindly "mounts" it as a disk and attempts to use it as a filesystem. They're almost certainly using it in a manner that is aware of the specificities of how S3 works.
      • foxhop 6y ago
        What if the blind mount works 99.9% of the time? I knew what I was doing was risky but well with in my tolerance for pain. Now that we have a strong consistency my code is fixed. That's pretty impressive.
  • Decker87 6y ago
    Isn't that the point? To provide some useful functionality for others to build on (and pay for)?