4 ms·

OCaml for Kafka stream processing, analytics and telemetry data

• tingfirst 1y ago
  For OCaml users interested in data streaming processing (similar to Flink or Spark), but looking for a faster and more efficient option, check out this OCaml plugin Timeplus Proton. Concise, safe, highly performant and fun -> Streaming Queries - Process large datasets with constant memory usage -> Async Inserts - High-throughput data ingestion with automatic batching -> Compression - LZ4 and ZSTD support for reduced network overhead -> TLS Security - Secure connections with certificate validation -> Connection Pooling - Efficient resource management for high-concurrency applications -> Rich Data Types - Full support for #ClickHouse types including Arrays, Maps, Enums, DateTime64 -> Idiomatic OCaml - Functional API leveraging OCaml's strengths
• rainmaker11 1y ago
  Interesting. How does the performance look like?
  • mfreeman451 1y ago
    [dead]
  • gangtao 1y ago
    Micheal, the author of the project did some performance test and he might annouce it later, but one of his first test show Async Insert with Rate: 3422488 rows/second, but I beleive after optimization and tuning, it could be much faster.