4 ms·
Looking at the code (src/microsearch/engine.py), we have: class SearchEngine: def __init__(self, k1: float = 1.5, b: float = 0.75): sel
by cabalamat 3y ago
Looking at the code (src/microsearch/engine.py), we have:
class SearchEngine:
def __init__(self, k1: float = 1.5, b: float = 0.75):
self._index: dict[str, dict[str, int]] = defaultdict(lambda: defaultdict(int))
self._documents: dict[str, str] = {}
self.k1 = k1
self.b = b
I've no idea what `k1` or `b` are. Nor is there a single comment in the entire file. Are comments considered unfashionable these days?
Looking at `_documents`, I'm guessing the keys are URLs and the values contents of those URLs, but i might be wrong.
The whole thing looks like it would've been a useful resource for people to learn how to build search engines with, that people could build on, had the writer been bothered to document it. But as it is I'm disappointed with the poor code.
- boyter 3y agoOn mobile device but it’s the standard weighting values for either TF/IDF or BM25. In this case BM25. A comment would be useful but they are also instantly recognisable to anyone familiar with the problem.
- 6510 3y ago> instantly recognisable to anyone familiar with the problem. I always love reading those when not familiar. It's almost as funny as reading something one already knows, waiting for the punch line...
- jffry 3y agoThat's explained in the article, which serves as the documentation for the code within the article. The link for BM25 goes to some of the math, and a little more searching the internet about BM25 parameters can lead you to some relevant articles on how to choose them.
- enoch2090 3y agoBM25 is just too famous in traditional searching and not worth explaining it all over again.
- Barrin92 3y agothis trend of `a: float` always reminds me of the Rich Hickey "you don't want types, you want proper names" talk. I really hate this (feels to me Go inspired) tendency of undescriptive single letter variable, with the type system abused as a naming assistant. Names can convey proper semantic information about what your program does, use them godammit
- alexmolas 3y agoI agree with you that better names are always preferable. But type hints also work as documentation. We can have both. However, in this particular case the undescriptive names are like this for historical reasons. I agree these are not the best names, but are the names used in the literature. If I was working in a physics I would probably use "c" as the speed of light or "kb" as the Boltzmann constant, which are non very descriptive names.
- planb 3y agoThis is right, but if you are implementing a formula known in the domain or documented elsewhere, you can (and should) use the letters used in the formula (in this case, b and k1) instead of making up names.
- sampo 3y ago> tendency of undescriptive single letter variable There are 2 schools of thought on which one is clearer, F = G * m1 * m2 / r**2 or force = gravitational_constant * mass_of_body_1 * mass_of_body_2 / distance_between_bodies ** 2
- trashtester 3y agoPhycisist : > F = G * m1 * m2 / r**2 Computer scientist: nonrelativistic_gravitational_force = ( Physics.NonRelativistic.Gravity.gravitational_constant * body1.NonRelativistic.mass() * body2.NonRelativistic.mass() / body1.NonRelativistic.distanceTo(body2) ** 2 )*
- 3y ago
- alexmolas 3y agoHi, author here. If I wanted a catchy title for the post I needed to cut the number of LOC as much as possible;) Joking apart, thanks for your feedback. I agree that usually it's better to have documentation and code together, but in this case since it's an educational project I decided to split code and documentation, and document the code in a blog post.
- marginalia_nu 3y agok1 and b are tuning parameters for the BM-25 ranking function[1]. These are not names OP's invented. Virtually every implementation and definitely every textbook uses these variable names. You give them the name k1 and b because otherwise nobody who's into information retrieval will understand what they do. [1] https://en.wikipedia.org/wiki/Okapi_BM25 https://en.wikipedia.org/wiki/Okapi_BM25