7 ms·

Show HN: Pythondocs.xyz – Live search for Python documentation

74 points by danosull 4y ago

Hi everyone! I've been working on a web search interface for Python's documentation as a personal project, and I think it's ready for other people to use... Please give it a go (and join me in praying to the server gods): https://pythondocs.xyz/ https://pythondocs.xyz/ Here's the tech stack for those interested: - Parser: Beautiful Soup + Mozilla Bleach - Database: in-memory SQLite (aiosqlite) + SQLAlchemy - Web server: FastAPI + Uvicorn + Jinja2 - Front end: Tailwind CSS + htmx + Alpine.js I have ideas for future improvements but hopefully the current version is useful to someone. Let me know what you think!

ystad 4y ago
Looks really nice!. Feature-request: Would love auto-suggest as I type. one of my favorite example(s) is Algolia's doc search https://www.algolia.com/doc/ https://www.algolia.com/doc/
- danosull 4y ago
  Thank you! And great feature suggestion. I'll put it on the list.
deleted 4y ago
[deleted]
clircle 4y ago
Does python not have built in documentation search? In R I use `??searchterm`.
- azeirah 4y ago
  It has help(), but I don't think there's a nice search feature included. I mean you could use string methods to search for something of course, help just returns a string. Then again, it's Python. There's probably some kind of import search from helpTools Or something among those lines, because... it always has this kind of stuff
  - westurner 4y ago
    The `pydoc` / `python -m pydoc` module doesn't do search; it would be O(n) for every search like grep without e.g. a sphinx searchindex: https://docs.python.org/3/library/pydoc.html https://docs.python.org/3/library/pydoc.html Sphinx searchindex.js does Porter stemming for English and other languages: https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/search/en.py https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/search/... sphinxcontrib.websupport.search supports Xapian, Whoosh (Python), and null: https://github.com/sphinx-doc/sphinxcontrib-websupport/blob/master/sphinxcontrib/websupport/search/__init__.py https://github.com/sphinx-doc/sphinxcontrib-websupport/blob/... sphinx-elasticsearch https://github.com/zeitonline/sphinx_elasticsearch https://github.com/zeitonline/sphinx_elasticsearch : > This is a stand-alone extraction of the functionality used by readthedocs.org, compatible with elasticsearch-6. MeiliSearch (rust) compared with ElasticSearch (java), Algolia, TypeSense (C++): https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html#about-meilisearch https://docs.meilisearch.com/learn/what_is_meilisearch/compa... Is there a good way to index each sphinx doc set's searchindex.js?
drcongo 4y ago
Small feedback: I tried `Iterable` as the search term and there's no mention of `typing.Iterable` on the first page - case sensitivity may help?
- danosull 4y ago
  Very useful feedback, thank you. The ordering of search results is currently very crude - it's basically just alphabetical. I plan to add "order by relevance" in the future, and I'll definitely bear capitalisation in mind.
thealistra 4y ago
I tried "typeof" and "is" and results were not what I wanted. Is there indexing for language keywords?
- danosull 4y ago
  Not right now, although it's something I'm interested in adding! You can get usually get quite good results by selecting Standard Library and Definitions, though. "is" is a special case as I currently enforce a 3 character minimum to keep the number of live search results sane.
amanzi 4y ago
Looks like it was a fun project but I prefer the flexibility of https://devdocs.io https://devdocs.io as I can keep all my framework docs searchable from one place. Also the instant lookup in DevDocs is super useful, especially for Python: https://devdocs.io/python~3.10/ https://devdocs.io/python~3.10/
- lepture 4y ago
  I agree. And the search results are not good in pythonsocs.xyz. Although I’ve upvoted this thread.
  - danosull 4y ago
    Useful comments, thanks! Better search results is my next big feature. I'm thinking about "sort by relevance", improved keyword and Python language parsing, and so on. And DevDocs is great :)
showmecode 4y ago
This may be a weird ask but it would be great if you could provide more emphasis to source code links for standard library results. E.g. searching for heapq would also show a link to https://github.com/python/cpython/blob/main/Lib/heapq.py https://github.com/python/cpython/blob/main/Lib/heapq.py I know these links are already at the very top of most pages in the python documentation but they are easily missed and it would be good for search tools to provide a nudge to the actual source.
- danosull 4y ago
  That's a great, suggestion thank you. A related thought is that I'd like to pull out results like built-in functions and Standard Library modules to make them more prominent, and perhaps a link to the source code could be presented alongside...

74 points by danosull 4y ago

ystad 4y ago

Looks really nice!. Feature-request: Would love auto-suggest as I type. one of my favorite example(s) is Algolia's doc search https://www.algolia.com/doc/ https://www.algolia.com/doc/

danosull 4y ago
Thank you! And great feature suggestion. I'll put it on the list.

deleted 4y ago

[deleted]

clircle 4y ago

Does python not have built in documentation search? In R I use `??searchterm`.

azeirah 4y ago
It has help(), but I don't think there's a nice search feature included. I mean you could use string methods to search for something of course, help just returns a string. Then again, it's Python. There's probably some kind of import search from helpTools Or something among those lines, because... it always has this kind of stuff
- westurner 4y ago
  The `pydoc` / `python -m pydoc` module doesn't do search; it would be O(n) for every search like grep without e.g. a sphinx searchindex: https://docs.python.org/3/library/pydoc.html https://docs.python.org/3/library/pydoc.html Sphinx searchindex.js does Porter stemming for English and other languages: https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/search/en.py https://github.com/sphinx-doc/sphinx/blob/5.x/sphinx/search/... sphinxcontrib.websupport.search supports Xapian, Whoosh (Python), and null: https://github.com/sphinx-doc/sphinxcontrib-websupport/blob/master/sphinxcontrib/websupport/search/__init__.py https://github.com/sphinx-doc/sphinxcontrib-websupport/blob/... sphinx-elasticsearch https://github.com/zeitonline/sphinx_elasticsearch https://github.com/zeitonline/sphinx_elasticsearch : > This is a stand-alone extraction of the functionality used by readthedocs.org, compatible with elasticsearch-6. MeiliSearch (rust) compared with ElasticSearch (java), Algolia, TypeSense (C++): https://docs.meilisearch.com/learn/what_is_meilisearch/comparison_to_alternatives.html#about-meilisearch https://docs.meilisearch.com/learn/what_is_meilisearch/compa... Is there a good way to index each sphinx doc set's searchindex.js?

drcongo 4y ago

Small feedback: I tried `Iterable` as the search term and there's no mention of `typing.Iterable` on the first page - case sensitivity may help?

danosull 4y ago
Very useful feedback, thank you. The ordering of search results is currently very crude - it's basically just alphabetical. I plan to add "order by relevance" in the future, and I'll definitely bear capitalisation in mind.

thealistra 4y ago

I tried "typeof" and "is" and results were not what I wanted. Is there indexing for language keywords?

danosull 4y ago
Not right now, although it's something I'm interested in adding! You can get usually get quite good results by selecting Standard Library and Definitions, though. "is" is a special case as I currently enforce a 3 character minimum to keep the number of live search results sane.

amanzi 4y ago

Looks like it was a fun project but I prefer the flexibility of https://devdocs.io https://devdocs.io as I can keep all my framework docs searchable from one place. Also the instant lookup in DevDocs is super useful, especially for Python: https://devdocs.io/python~3.10/ https://devdocs.io/python~3.10/

lepture 4y ago
I agree. And the search results are not good in pythonsocs.xyz. Although I’ve upvoted this thread.
- danosull 4y ago
  Useful comments, thanks! Better search results is my next big feature. I'm thinking about "sort by relevance", improved keyword and Python language parsing, and so on. And DevDocs is great :)

showmecode 4y ago

This may be a weird ask but it would be great if you could provide more emphasis to source code links for standard library results. E.g. searching for heapq would also show a link to https://github.com/python/cpython/blob/main/Lib/heapq.py https://github.com/python/cpython/blob/main/Lib/heapq.py I know these links are already at the very top of most pages in the python documentation but they are easily missed and it would be good for search tools to provide a nudge to the actual source.

danosull 4y ago
That's a great, suggestion thank you. A related thought is that I'd like to pull out results like built-in functions and Standard Library modules to make them more prominent, and perhaps a link to the source code could be presented alongside...