4 ms·
It implements full text search without repeated strcmp() over half a gigabyte of strings, so search completes in reasonable time. Compare that to PyPi which do
by _wmd 13y ago
It implements full text search without repeated strcmp() over half a gigabyte of strings, so search completes in reasonable time.
Compare that to PyPi which does exactly what I described, and even during off peak periods, searches can sometimes take up to 20 seconds
- srinidhiv 13y agoWhy not just fork the pypi code and fix it, then?
- donaldstufft 13y agoHave you looked at the PyPI codebase? ;) The real answer is I have. I'm a PyPI administrator now and have been enacting a lot of these benefits on PyPI itself. Just this weekend I've deployed a CDN in front of all of PyPI so that package downloads have sped up by 1-3 orders of magnitude depending on location. Previously I had pushed hard for getting a real valid and trusted SSL certificate on PyPI, moving the user provided html/js from packages.python.org to pythonhosted.org in order to prevent CSRF fixation attacks. Like I said Crate is an experiment. One that has enabled me to push for good changes in PyPI.
- meowface 13y agoI think you're doing good work here, and I agree that PyPI is quite a bit of a mess. One little suggestion/question, though. CPAN: Comprehensive Perl Archive Network Rubygems: (self-explanatory) PyPI: Python Package Index Crate.io: Crate? Crate.io? Nothing about the name or any part of the domain really mentions Python, or Python packages or libraries. It's not a bad name, but it doesn't really seem like a name for a large Python package index. What's the exact rationale behind the naming?
- donaldstufft 13y agoCrate is a synonym for Package and I liked the name :)
- jl6 13y agoI believe a krait is a kind of snake if you want to go for that angle.
- lightcatcher 13y agoIf you aren't aware, packages in the Rust programming language are called crates.
- flexd 13y agoIf it actually ends up replacing PyPI it could just live on the same domain as PyPi though (provided they are okay with that). But it needs a lot of work on the front end in order to be better :-), I like the backend fixes and speed improvements though.
- zamalek 13y agoDitto. Plane.io.
- orf 13y agoReally? Why do they strcmp() over half a gb of strings, that seems pretty stupid and inefficient.