Yes, sradman. You describe it correctly here as a HTAP architecture. What we do differently is that the same scaleout consistent metadata layer is used for (1)

Yes, sradman. You describe it correctly here as a HTAP architecture. What we do differently is that the same scaleout consistent metadata layer is used for (1) online FS (NDB - mysql clustrer), (2) offline FS (hive), and (3) HopsFS-S3. That is, the data files for Hive are stored in HopsFS, that in turn stores its data in S3. Because of our metadata architecture, we ensure the consistency of hive data files with hive metadata using foreign keys, and we have extended metadata to store all the feature metadata that is kept consistent with good auld foreign keys and transactions. We also automatically synchronize the metadata to elasticsearch, so you can do free-text search for features, files, extended metadata, etc. So, you can search up to PBs of files/dirs/tables/features/extended-metadata in milliseconds. You can find more info here on the research behind it: https://www.logicalclocks.com/research https://www.logicalclocks.com/research