13 ms·

I had to read Hyperscan source code for work. Honnestly, once you understood (some of...) the math/automata details, this is by far one of the best big codebas

I had to read Hyperscan source code for work. Honnestly, once you understood (some of...) the math/automata details, this is by far one of the best big codebase ever written. So clean, beautiful, powerful. I've learned a lot from this codebase. Does anyone know more codebases as well written as Hyperscan ?

phanix5 7y ago
Wow I was looking for good codebases to dig into in my free time. This one looks like a good option.
glangdale 7y ago
ex-Hyperscan guy here: thanks for the kind words. I'm sure my team (who are primarily responsible for the cleanliness - as a developer, I make a great "ideas person") would appreciate it. Can you disclose how/why you were reading Hyperscan source for work? Just curious, no agenda.
- dnpp123 7y ago
  Wow thanks for your work then ! It was for a (now failed) startup selling IPS appliances. What really helped us in the end was that we could run the runtime in C (C++ was not possible); however the compiler/tooling we used was fairly old (...) so I had to dig into the codebase details to make it work. As a side note, if I had to say something bad about Hyperscan, it would be the lack of high level documentation. I don't know now, but back then only a couple of blog articles available... I always had been curious : was it something intended to prevent copycats ? Lack of time ? Why not try to explain more the high level math/automata details ? If the high level documentation were to be improved, I'm pretty sure the number of companies integrating Hyperscan would increase, hence Intel sales would increase (since it has been bought by Intel ;)
  - glangdale 7y ago
    The lack of high level docs up to the point of open sourcing was partly due to prevention of copycats but mainly due to lack of resources (chiefly time) to spend time writing. We were a small team and time spent on documenting stuff we all understood pretty well was time spent not chasing customers or improving our product. Later: well, there is a Hyperscan paper and there may be more material coming out later. Also, not to be a jerk, but a lot of people claim that they will read/understand/use this kind of documentation and my experience was that only a fraction actually do, and of those fraction, most of them don't behave in a way that's actually useful enough to justify having made all those docs. One is more likely to wind up with people kibitzing and making inane tweaking suggestions ("use more NFAs! no, use more DFAs"); less likely is meaningful OSS contributions or using the software when they might not have before.

phanix5 7y ago

Wow I was looking for good codebases to dig into in my free time. This one looks like a good option.

glangdale 7y ago

ex-Hyperscan guy here: thanks for the kind words. I'm sure my team (who are primarily responsible for the cleanliness - as a developer, I make a great "ideas person") would appreciate it. Can you disclose how/why you were reading Hyperscan source for work? Just curious, no agenda.

dnpp123 7y ago
Wow thanks for your work then ! It was for a (now failed) startup selling IPS appliances. What really helped us in the end was that we could run the runtime in C (C++ was not possible); however the compiler/tooling we used was fairly old (...) so I had to dig into the codebase details to make it work. As a side note, if I had to say something bad about Hyperscan, it would be the lack of high level documentation. I don't know now, but back then only a couple of blog articles available... I always had been curious : was it something intended to prevent copycats ? Lack of time ? Why not try to explain more the high level math/automata details ? If the high level documentation were to be improved, I'm pretty sure the number of companies integrating Hyperscan would increase, hence Intel sales would increase (since it has been bought by Intel ;)
- glangdale 7y ago
  The lack of high level docs up to the point of open sourcing was partly due to prevention of copycats but mainly due to lack of resources (chiefly time) to spend time writing. We were a small team and time spent on documenting stuff we all understood pretty well was time spent not chasing customers or improving our product. Later: well, there is a Hyperscan paper and there may be more material coming out later. Also, not to be a jerk, but a lot of people claim that they will read/understand/use this kind of documentation and my experience was that only a fraction actually do, and of those fraction, most of them don't behave in a way that's actually useful enough to justify having made all those docs. One is more likely to wind up with people kibitzing and making inane tweaking suggestions ("use more NFAs! no, use more DFAs"); less likely is meaningful OSS contributions or using the software when they might not have before.