4 ms·

Show HN: Checking 320M passwords in milliseconds with serverless bloom filters

33 points by thomasj 9y ago

slackingoff2017 9y ago
Interesting, but you probably didn't need to write your own filter :). Google's Guava library has the best designed and performing Bloom filter I've ever seen. Lockless concurrency, extremely fast hashing function.
- thomasj 9y ago
  I didn't :) I used this package: https://github.com/willf/bloom https://github.com/willf/bloom as I wanted to use Go.
  - slackingoff2017 9y ago
    Ah, good :) . No wheels invented today
    - natch 9y ago
      There are many benefits to reinventing wheels, though. Often the benefits outweigh the costs.
      - slackingoff2017 9y ago
        Said everyone trying to reinvent the wheel ever :) . I would replace that with "rarely". Sometimes it's obviously optimal to roll your own thing but in almost every case it wasn't. My bar is pretty high. Unless you can write an academic paper on the thing you just invented you should try what's out there first.
        natch 9y ago
        That first line is a little flippant. And the rest is opinion, not fact. It really depends on the particulars and the benefits of each path. When you build your own ("invent" is disingenuous and "reinvent" seems pejorative, so going with a more neutral phrasing) you own the result, you own the intermediate steps, you learn more, and you have more flexibility in thinking of new tweaks. It may take more time, but the investment of time pays dividends. Wheels themselves are one of the best examples of something that has benefited from repeated "reinvention," to use the flawed term, over the last few thousands of years. I don't think you'll find a Maserati wheel on a Tesla, and it's not because Maserati wheels are bad, or because somebody cared about your bar or academic papers.
natch 9y ago
"found password in bloom filter:..." is sort of a misleading message. Bloom filters don't tell you that something is present. They can tell you either that something is not present, or that it is likely to be present.
- thomasj 9y ago
  Good point! I'm planning to write a blog post about this and I'll be clearer on this.

33 points by thomasj 9y ago

slackingoff2017 9y ago

Interesting, but you probably didn't need to write your own filter :). Google's Guava library has the best designed and performing Bloom filter I've ever seen. Lockless concurrency, extremely fast hashing function.

thomasj 9y ago
I didn't :) I used this package: https://github.com/willf/bloom https://github.com/willf/bloom as I wanted to use Go.
- slackingoff2017 9y ago
  Ah, good :) . No wheels invented today
  - natch 9y ago
    There are many benefits to reinventing wheels, though. Often the benefits outweigh the costs.
    - slackingoff2017 9y ago
      Said everyone trying to reinvent the wheel ever :) . I would replace that with "rarely". Sometimes it's obviously optimal to roll your own thing but in almost every case it wasn't. My bar is pretty high. Unless you can write an academic paper on the thing you just invented you should try what's out there first.
      - natch 9y ago
        That first line is a little flippant. And the rest is opinion, not fact. It really depends on the particulars and the benefits of each path. When you build your own ("invent" is disingenuous and "reinvent" seems pejorative, so going with a more neutral phrasing) you own the result, you own the intermediate steps, you learn more, and you have more flexibility in thinking of new tweaks. It may take more time, but the investment of time pays dividends. Wheels themselves are one of the best examples of something that has benefited from repeated "reinvention," to use the flawed term, over the last few thousands of years. I don't think you'll find a Maserati wheel on a Tesla, and it's not because Maserati wheels are bad, or because somebody cared about your bar or academic papers.

natch 9y ago

"found password in bloom filter:..." is sort of a misleading message. Bloom filters don't tell you that something is present. They can tell you either that something is not present, or that it is likely to be present.

thomasj 9y ago
Good point! I'm planning to write a blog post about this and I'll be clearer on this.