4 ms·

Did you feed it the simple8b paper along with your prompt?

jcgrillo 2y ago
No, that's an interesting idea though. Hard to imagine how that would help with the code correctness issues, though. I haven't even dug into algorithmic correctness yet so I have no real idea whether there's room for improvement there--although I sure do suspect there is! EDIT: Oh my. After digging into the code I found this gem: fn encode_rle(&self, value: u64, count: usize) -> u64 { let selector = Simple8bSelector::RLE as u64; (selector << 60) | ((count as u64) << 30) | (value & 0x3FFFFFFF) } And this one: fn try_rle(&self, input: &[u64]) -> Option<(u64, usize)> { if input.is_empty() { return None; } let value = input[0]; let mut count = 1; for &x in input.iter().skip(1) { if x != value || count >= 0x3FFFFFFF { // Max 30-bit run length break; } count += 1; } Some((value, count)) } What even is going on here? Compare to an actually sane implementation like[1] or[2]. [1]https://github.com/lemire/FastPFor/blob/master/headers/simple8b_rle.h https://github.com/lemire/FastPFor/blob/master/headers/simpl... [2]https://github.com/timescale/timescaledb/blob/403782a5899c754db67c41ea859cf34611fa5bde/tsl/src/compression/simple8b_rle.h https://github.com/timescale/timescaledb/blob/403782a5899c75...
- jcgrillo 2y ago
  Actually I was wrong about where the error was here, encode_rle actually works like it should the shift isn't the problem there. It actually blew up later in a different place. The second one is just a bizarre way to write that but sure, it counts the first N repeats in the input slice.. There's plenty of bizarre stuff in here[1], but mostly the general shape of the idea is directionally correct. A couple notably questionable things, though, like the assumption that RLE is always the way to go if the run length is greater than 8, perplexing style choices, etc. [1] https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c8c83196119b9da9bade70b76d8c27db https://play.rust-lang.org/?version=stable&mode=debug&editio...

No, that's an interesting idea though. Hard to imagine how that would help with the code correctness issues, though. I haven't even dug into algorithmic correctness yet so I have no real idea whether there's room for improvement there--although I sure do suspect there is! EDIT: Oh my. After digging into the code I found this gem: fn encode_rle(&self, value: u64, count: usize) -> u64 { let selector = Simple8bSelector::RLE as u64; (selector << 60) | ((count as u64) << 30) | (value & 0x3FFFFFFF) } And this one: fn try_rle(&self, input: &[u64]) -> Option<(u64, usize)> { if input.is_empty() { return None; } let value = input[0]; let mut count = 1; for &x in input.iter().skip(1) { if x != value || count >= 0x3FFFFFFF { // Max 30-bit run length break; } count += 1; } Some((value, count)) } What even is going on here? Compare to an actually sane implementation like[1] or[2]. [1]https://github.com/lemire/FastPFor/blob/master/headers/simple8b_rle.h https://github.com/lemire/FastPFor/blob/master/headers/simpl... [2]https://github.com/timescale/timescaledb/blob/403782a5899c754db67c41ea859cf34611fa5bde/tsl/src/compression/simple8b_rle.h https://github.com/timescale/timescaledb/blob/403782a5899c75...

jcgrillo 2y ago
Actually I was wrong about where the error was here, encode_rle actually works like it should the shift isn't the problem there. It actually blew up later in a different place. The second one is just a bizarre way to write that but sure, it counts the first N repeats in the input slice.. There's plenty of bizarre stuff in here[1], but mostly the general shape of the idea is directionally correct. A couple notably questionable things, though, like the assumption that RLE is always the way to go if the run length is greater than 8, perplexing style choices, etc. [1] https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c8c83196119b9da9bade70b76d8c27db https://play.rust-lang.org/?version=stable&mode=debug&editio...