huntaub

14 ms·

1. 181.▲
   by huntaub 2y ago

   That's exactly the kind of thing that I've been hearing lots of teams having to solve individually, and I'm glad that this set up worked out for you. Would love to see you try it for bioinformatics (another industry where thi
2. 182.▲
   by huntaub 2y ago

   Those costs only apply to data transfer into and out of AWS. If you're running EC2 instances in AWS, your Regatta file system is in AWS, and your S3 bucket is in AWS -- then you shouldn't incur additional data transfer fees.
3. 183.▲
   by huntaub 2y ago

   Without getting too much into the details of the system, our durable cache is designed for 5 9s of durability (and we're working on a version that will provide 11 9s of durability soon). You can't achieve those durability numbers
4. 184.▲
   by huntaub 2y ago

   All connected file system clients see strong, read-after-write consistency. Most file operations are synchronized to S3 within a few minutes of completion.
5. 185.▲
   by huntaub 2y ago

   I could totally be misreading DirectPV, but it appears to be a way to use K8s Persistent Volumes to manage things like NVME drives which are attached to each node, and doesn't provide any tie in to S3 (outside of the fact that it'
6. 186.▲
   by huntaub 2y ago

   Great question! We fill the same role as AWS Storage Gateway (and I used to work closely with that team when I was at AWS, lots of respect for what they do). AWS Storage Gateway is built primarily as an appliance to be installed on instance
7. 187.▲
   by huntaub 2y ago

   Yes! This is my expectation. Lots of the big companies have already done this with in-house architecture. With Regatta, we want to democratize building stateless applications that can take advantage of the low-cost storage of S3.
8. 188.▲
   by huntaub 2y ago

   Great call out. Some kinds of data, like ACLs and specific kinds of metadata, don't live in S3. Full disclosure, we don't support ACLs today (but plan to soon). We keep file system metadata in the durable cache. For some files (wh
9. 189.▲
   by huntaub 2y ago

   It depends on what you mean by crash-consistent. I would expect that we handle crash-consistency at the client fine (since it is the same crash-consistency of NFSv3) and craash-consistency at the server also fine (since we are able to detec
10. 190.▲
    by huntaub 2y ago

    It can offer an advantage over the built-in caching, but it depends on your exact access patterns. For example, if you are running ClickHouse on multiple servers and accessing the same reference data, it's more efficient to cache tha
11. 191.▲
    by huntaub 2y ago

    Ha, thank you for the FTP comment, I was hoping someone would make it. > I really wish that NFSv3 and Linux had built-in file hashing ioctls that could delegate some of this expensive work to the backend as it would make it much easier t
12. 192.▲
    by huntaub 2y ago

    Thank you -- this seems like a fantastic service. You see I consistently miss that "h" in synchronize.
13. 193.▲
    by huntaub 2y ago

    Wow, thanks for the nice note! No questions are silly, and I'll also note that we now have a docs site ( https://docs.regattastorage.com ) and feel free to email me (hleath [at] regattastorage.com) if I don't fully addre
14. 194.▲
    by huntaub 2y ago

    Thanks for the question. We stage writes to a durable, shared caching layer. This allows us to respond quickly to your application when it performs these operations (<1ms), but then asynchronously send those operations to S3 later. When
15. 195.▲
    by huntaub 2y ago

    :sunglasses: We think it's important to be where our customers are, and we're looking to prioritize the next regions that we launch in based on customer demand. We expect to be in more regions by the end of the year.
16. 196.▲
    by huntaub 2y ago

    This is exactly the problem that we solve! You don't need any local disk on your EC2 instance in order to use Regatta or work with data in S3. Our high-speed caching layer plays the role as this local disk for you, so that you can work
17. 197.▲
    by huntaub 2y ago

    The tl;dr of this is -- yes. We have a durable caching layer that we use to stage writes before we asynchronously replicate them to S3. This means that we are able to quickly (<1ms) perform operations like single-byte updates and renames
18. 198.▲
    by huntaub 2y ago

    I think this is exactly right, but there are lots of people who don't want to manage their own NFS servers -- that's who we're targeting with Regatta. Notably, I think that v4 delegations gets you close but not close enough t
19. 199.▲
    by huntaub 2y ago

    Great call out -- we'll get that done. Thanks!
20. 200.▲
    by huntaub 2y ago

    This is a managed cloud service. If you're interested in using Regatta on-premises, I'd love to hear from you -- shoot me some mail at hleath [at] regattastorage.com
21. 201.▲
    by huntaub 2y ago

    I think that "owning the storage" is such an important part of this. I'm excited that folks who use this will continue to have access to their data directly through S3, so if they ever decide to move off of Regatta, all their
22. 202.▲
    by huntaub 2y ago

    Yes, I think it's similar product, but we're looking to provide high performance on all dimensions (latency, throughput, and IOPS). I totally agree with you that Elastic Throughput solves this problem, but it can be expensive for
23. 203.▲
    by huntaub 2y ago

    That's correct -- every file is an S3 object. If you change the middle of a large file, Regatta will store the change on our durable caching layer efficiently (and most writes complete in under 1ms). Regatta will then asynchronously up
24. 204.▲
    by huntaub 2y ago

    It's similar to JuiceFS, but JuiceFS writes and reads data from S3 in a proprietary block format. This means that you cannot connect JuiceFS to existing data sets in S3, and you cannot use data written through JuiceFS from the S3 API d
25. 205.▲
    by huntaub 2y ago

    I don't think so, I see them as complementary. MinIO is great when you have downstream applications which speak the S3 API that need acceleration of that data. Regatta is designed for applications which speak file semantics (think, app
26. 206.▲
    by huntaub 2y ago

    I think this is a great call out, and you're correct. One example that comes to mind is that NFSv3 doesn't support flags on the rename() operation (such as RENAME_WHITEOUT), which means that you can't use them as an overlay u
27. 207.▲
    by huntaub 2y ago

    Thanks for the question! Mountpoint for Amazon S3 is a FUSE layer that doesn't support full POSIX semantics. For example, you can't use Mountpoint for Amazon S3 for random writes to existing files, appends, or renames. This means
28. 208.▲
    by huntaub 2y ago

    Thanks for the question! > How do you handle the cache server crashing before syncing to S3? Do the cache servers have local disk as well? Our caching layer is highly durable, which is (in my opinion) the key for doing this kind of stagi
29. 209.▲
    by huntaub 2y ago

    Thanks for the question! Snowflake and Databricks aren't storage products, but are managed compute platforms on top of storage that probably looks a lot like this. Snowflake allows you to easily connect different data sets to your data
30. 210.▲
    Launch HN: Regatta Storage (YC F24) – Turn S3 into a local-like, POSIX cloud FS

    587 points by huntaub 2y ago|320 comments