6 ms·

One thing I haven't seen anyone note about clickhouse though which would be really important to many for data durability, is that it does not use fsync anywhere

One thing I haven't seen anyone note about clickhouse though which would be really important to many for data durability, is that it does not use fsync anywhere at all.

caust1c 7y ago
It's pretty clearly laid out in the docs. Hopefully anyone seriously considering using Clickhouse reads the docs thoroughly and understands what they're implementing.
- codexon 7y ago
  What do you mean clearly laid out? This is the only mention of fsync I could find through google or their own search function. https://clickhouse.yandex/docs/en/operations/settings/settings/#fsync-metadata https://clickhouse.yandex/docs/en/operations/settings/settin...
  - caust1c 7y ago
    The title of the page might be a little snarky, but it's in the introduction that transactional queries are not supported: https://clickhouse.yandex/docs/en/introduction/features_considered_disadvantages/ https://clickhouse.yandex/docs/en/introduction/features_cons... Sure it's not specifically about `fsync` but presumably this is what the consumer of the database actually wants to know.
    - codexon 7y ago
      Fsync is not synonymous with transactions. Not using fsync anywhere means there's a wide window that can be over 10 minutes long when data can be lost when a server gets an unplanned shutdown.
      - valyala 7y ago
        Btw, it should be easy adding fsync to ClickHouse. For instance VictoriaMetrics uses similar file format as ClickHouse, and it issues proper fsyncs at least every second, so it may lose only the last second of data on unclean shutdown such as OOM, hardware reset or `kill -9`. [1], [2]. [1] https://medium.com/@valyala/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704 https://medium.com/@valyala/wal-usage-looks-broken-in-modern... [2] https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282 https://medium.com/@valyala/how-victoriametrics-makes-instan...
FridgeSeal 7y ago
I’m not understanding the implication of this, could you explain?
- caust1c 7y ago
  The implication is that clickhouse can't easily support transactional queries. That's why it's an OLAP not OLTP database. (On-Line Analytics Processing vs On-Line Transaction Processing).
  - FridgeSeal 7y ago
    Oh that’s not too bad, they’re very explicit about not having transaction support, thanks for explaining.
    - deleted 7y ago
      [deleted]
  - codexon 7y ago
    This is not the implication at all. Clickhouse can easily add fsync, they just choose not to do it. Mongodb also did not use fsync and was ridiculed for it, yet no one mentions this about clickhouse.
    - pritambaral 7y ago
      > Mongodb also did not use fsync and was ridiculed for it, yet no one mentions this about clickhouse. MongoDB claimed to be a replacement for RDBMS-es (which includes OLTP). ClickHouse is explicit about being OLAP-only. MongoDB also hid the fact that they weren't doing fsync, especially when showing off "benchmarks" against OLTP RDBMS-es, while ClickHouse has not tried to show themselves as a replacement for OLTP RDBMS-es. > Clickhouse can easily add fsync, they just choose not to do it. For good reason. It's not a simple matter of choosing one of two options. The choice has consequences: performance.
      - deleted 7y ago
DasIch 7y ago
I can't find anything about this in the docs except[1]. I also can't find any issues in their bug tracker related to clickhouse not using fsync[2]. I can however find code that actually calls fsync[3][4]. To be fair I haven't read enough to determine how this (doesn't) affect durability. Nevertheless I'm wondering do you have a source for this claim? [1]: https://clickhouse.yandex/docs/en/operations/settings/settings/#fsync-metadata [2]: https://github.com/ClickHouse/ClickHouse/search?q=fsync&type=Issues [3]: https://github.com/ClickHouse/ClickHouse/blob/355b1e5594119e036a2d62988bfa42bc8b1a1687/dbms/src/IO/WriteBufferFromFileDescriptor.cpp#L113 [4]: https://github.com/ClickHouse/ClickHouse/blob/e765733a26cfc4cecc13c981686560338256a6b1/dbms/src/IO/WriteBufferAIO.cpp#L98
- codexon 7y ago
  As I mentioned, there's only 1 place where it says anything about fsync, and in that page, it says that is only for creating .sql files. https://groups.google.com/d/msg/clickhouse/cjJ6v8uzu0Q/jGV59S0VBQAJ https://groups.google.com/d/msg/clickhouse/cjJ6v8uzu0Q/jGV59... > The reason is because CH does not use fsync (for performance) https://www.linkedin.com/in/dzhuravlev/ https://www.linkedin.com/in/dzhuravlev/
gautamcgoel 7y ago
Yes, I have the same model. I think I bought it for between $650 and $750.

One thing I haven't seen anyone note about clickhouse though which would be really important to many for data durability, is that it does not use fsync anywhere at all.

caust1c 7y ago

It's pretty clearly laid out in the docs. Hopefully anyone seriously considering using Clickhouse reads the docs thoroughly and understands what they're implementing.

codexon 7y ago
What do you mean clearly laid out? This is the only mention of fsync I could find through google or their own search function. https://clickhouse.yandex/docs/en/operations/settings/settings/#fsync-metadata https://clickhouse.yandex/docs/en/operations/settings/settin...
- caust1c 7y ago
  The title of the page might be a little snarky, but it's in the introduction that transactional queries are not supported: https://clickhouse.yandex/docs/en/introduction/features_considered_disadvantages/ https://clickhouse.yandex/docs/en/introduction/features_cons... Sure it's not specifically about `fsync` but presumably this is what the consumer of the database actually wants to know.
  - codexon 7y ago
    Fsync is not synonymous with transactions. Not using fsync anywhere means there's a wide window that can be over 10 minutes long when data can be lost when a server gets an unplanned shutdown.
    - valyala 7y ago
      Btw, it should be easy adding fsync to ClickHouse. For instance VictoriaMetrics uses similar file format as ClickHouse, and it issues proper fsyncs at least every second, so it may lose only the last second of data on unclean shutdown such as OOM, hardware reset or `kill -9`. [1], [2]. [1] https://medium.com/@valyala/wal-usage-looks-broken-in-modern-time-series-databases-b62a627ab704 https://medium.com/@valyala/wal-usage-looks-broken-in-modern... [2] https://medium.com/@valyala/how-victoriametrics-makes-instant-snapshots-for-multi-terabyte-time-series-data-e1f3fb0e0282 https://medium.com/@valyala/how-victoriametrics-makes-instan...

FridgeSeal 7y ago

I’m not understanding the implication of this, could you explain?

caust1c 7y ago
The implication is that clickhouse can't easily support transactional queries. That's why it's an OLAP not OLTP database. (On-Line Analytics Processing vs On-Line Transaction Processing).
- FridgeSeal 7y ago
  Oh that’s not too bad, they’re very explicit about not having transaction support, thanks for explaining.
  - deleted 7y ago
    [deleted]
- codexon 7y ago
  This is not the implication at all. Clickhouse can easily add fsync, they just choose not to do it. Mongodb also did not use fsync and was ridiculed for it, yet no one mentions this about clickhouse.
  - pritambaral 7y ago
    > Mongodb also did not use fsync and was ridiculed for it, yet no one mentions this about clickhouse. MongoDB claimed to be a replacement for RDBMS-es (which includes OLTP). ClickHouse is explicit about being OLAP-only. MongoDB also hid the fact that they weren't doing fsync, especially when showing off "benchmarks" against OLTP RDBMS-es, while ClickHouse has not tried to show themselves as a replacement for OLTP RDBMS-es. > Clickhouse can easily add fsync, they just choose not to do it. For good reason. It's not a simple matter of choosing one of two options. The choice has consequences: performance.
    - deleted 7y ago

DasIch 7y ago

I can't find anything about this in the docs except[1]. I also can't find any issues in their bug tracker related to clickhouse not using fsync[2]. I can however find code that actually calls fsync[3][4]. To be fair I haven't read enough to determine how this (doesn't) affect durability. Nevertheless I'm wondering do you have a source for this claim? [1]: https://clickhouse.yandex/docs/en/operations/settings/settings/#fsync-metadata [2]: https://github.com/ClickHouse/ClickHouse/search?q=fsync&type=Issues [3]: https://github.com/ClickHouse/ClickHouse/blob/355b1e5594119e036a2d62988bfa42bc8b1a1687/dbms/src/IO/WriteBufferFromFileDescriptor.cpp#L113 [4]: https://github.com/ClickHouse/ClickHouse/blob/e765733a26cfc4cecc13c981686560338256a6b1/dbms/src/IO/WriteBufferAIO.cpp#L98

codexon 7y ago
As I mentioned, there's only 1 place where it says anything about fsync, and in that page, it says that is only for creating .sql files. https://groups.google.com/d/msg/clickhouse/cjJ6v8uzu0Q/jGV59S0VBQAJ https://groups.google.com/d/msg/clickhouse/cjJ6v8uzu0Q/jGV59... > The reason is because CH does not use fsync (for performance) https://www.linkedin.com/in/dzhuravlev/ https://www.linkedin.com/in/dzhuravlev/

gautamcgoel 7y ago

Yes, I have the same model. I think I bought it for between $650 and $750.