powturbo

7 ms·

1.
Show HN: TurboBench, the Compression Lie Detector, 100 Codecs, Daily Update(github.com)
5 points by powturbo 16d ago|0 comments
2.
Show HN:TurboBench: Data Compression Benchmark. The Compression Lie Detector(github.com)
2 points by powturbo 20d ago|1 comments
3.
by powturbo 20d ago
TurboBench: The Ultimate Data Compression Benchmark 100+ Codecs. The Powerhouse in a Single Executable — The Compression Lie Detector Tired of misleading benchmarks with I/O overhead, cache effects, and CPU throttling? TurboBench deliv
4.
by powturbo 3y ago
Try TurboRC using 16-bits format: https://github.com/powturbo/Turbo-Range-Coder
5.
by powturbo 3y ago
Thank for your analyse. It seems it's a corner case, benchmarking a base64 encoded random file is not a typical case. igzip has no more advantage above libdeflate. It has only fast compression at the levels 0,1,2 but with mediocre com
6.
by powturbo 3y ago
EDIT: I've seen the file you're referencing is 4Gi-Base64. This file is not very compressible (75% with gzip). It's possible that igzip is simply storing the file or some parts of it without compression. This explain why it c
7.
by powturbo 3y ago
Sorry, a digit was missing, but now https://github.com/powturbo/TurboBench/issues/43 Well, a correct benchmarking is not done with special data, but with datasets that represent a large set of distributions.
8.
by powturbo 3y ago
libdeflate compress better and has faster decompression than igzip. See the silesia single core in-memory benchmark here [1] comparing zlib,libdeflate,igzip,... https://github.com/powturbo/TurboBench/issues/4
9.
by powturbo 3y ago
file html8 : 100MB random html pages from a 1m Alexa Top sites corpus. Number of pages = 1178 Average length = 84886 bytes CPU: AMD 7840HS 3.8-5.1GHZ
10.
Web Content Compression Benchmark.zlib brotli zstd zlib_ng libdeflate igzip,...(github.com)
2 points by powturbo 3y ago|1 comments
11.
by powturbo 3y ago
You can use TurboBench [1] to benchmark libdeflate against zlib, igzip, zlib-ng and others. Download TurboBench from Releases [2] Here Some Benchmarks: - https://github.com/zlib-ng/zlib-ng/issues/1486 - http
12.
by powturbo 3y ago
** Cython bindings for Turbo Base64 [1] ** - 20-30x faster than the standard library - Benchmarks faster than any other C base64 library - Fastest implementation of AVX, AVX2, and AVX512 base64 encoding - No other dependencies [1] - https:
13.
Show HN: The fastest Turbo-Base64 for Python: 20-30x faster than the std library(github.com)
2 points by powturbo 3y ago|1 comments
14.
by powturbo 3y ago
You can use TurboBench [1] to benchmark igzip, zlib-ng and others. Download TurbBench from Releases [2] Here Some Benchmarks: - https://github.com/zlib-ng/zlib-ng/issues/1486 - https://github.com&#
15.
by powturbo 3y ago
Yes, especially SIMD Neon where gcc producing horrible Neon code for all versions < gcc-12 even by using simd intrinsics. From version 12 gcc is at same level as clang. You can try it : https://github.com/powturbo/Tu
16.
by powturbo 3y ago
> Obviously it's not impossible, since it's happening on my laptop. Well, zstd cli is more optimized for multithreading and asynchrounous i/o. zstd is overlapping cpu decompression with reading/writing. Additionally z
17.
by powturbo 3y ago
"In memory" means without i/o (disk,ssd,network), only in RAM. Streaming means compressing/decompressing in chunks and can also be "IN memory".
18.
by powturbo 3y ago
No, No and No. It's algorithmically impossible that zstd is decompressing faster than lz4 by using the same environment. You are comparing the zstd with asynchrounous I/O + multithreading against a single thread simple lz4 cli. Ad
19.
by powturbo 3y ago
lz4 is ALWAYS decompressing faster (> 2x) thand zstd
20.
by powturbo 3y ago
LZAV now integrated into TurboBench [1] Download TurboBench for windows from releases [2] Lenovo IdeaPad 5 Pro - Ryzen 6600HS - DDR5 6400 - gcc-13.1 File: silesia.tar 220MB (Mixed text/binary) C Size ratio% C MB/s D
21.
by powturbo 3y ago
see also Discussion with a brotli developer [2] and "Chrome: Implement zstd content-encoding" [3] : [1] - Benchmark App: https://github.com/powturbo/TurboBench [2] - https://twitter.com/powtur
22.
Show HN: TurboBench: Dynamic/Static web content compression benchmark(github.com)
2 points by powturbo 3y ago|1 comments
23.
Bug Forces Intel to Halt Some Xeon Sapphire Rapids Shipments(tomshardware.com)
1 points by powturbo 3y ago|0 comments
24.
by powturbo 3y ago
Thanks for the confirmaation. If your query engine can't exploit fine grained access, then general purpose compressors like lz4 or zstd with large compressed blocks are the better choice for numerical data, especially in combination w
25.
by powturbo 3y ago
Gorilla [1] and Gorilla based algos [2] are simply overrated. Store these values as 32 bits floats instead of 64 bits and you get instant 50% reduction without any compression. This is valid for allmost all time series data. Most of time se
26.
Show HN: Time Series Benchmark TurboPFor,TurboFloat,TurboFloat LzX,TurboGorilla(github.com)
3 points by powturbo 3y ago|3 comments
27.
by powturbo 3y ago
There is nothing to earn from non GPL license, maybe some small reputation and more stars on github, but that's all. Contribution is minimal if any, and when then you have more work. Dual license is better so others can't compla
28.
by powturbo 3y ago
- Turbo-Base64 is used in clikhouse - Turbo-Base64 is available in archlinux: https://aur.archlinux.org/packages/turbo-base64
29.
by powturbo 3y ago
- 100% C (C++ headers), as simple as memcpy - No other base64 library encode or decode faster - Scalar can be faster than other SSE or ARM Neon based base64 libraries - SSE faster than other SSE/AVX/AVX2! base64 library - Fastest
30.
Show HN: Turbo Base64 library. AVX512 Faster than memcpy and any other base64(github.com)
5 points by powturbo 3y ago|4 comments

More ›