5 ms·

Show HN: PyTorch search engine

65 points by chris_f 4y ago

cariaso 4y ago
that seems like a super valuable domain being very under used.
- version_five 4y ago
  I think (at the risk of sounding like a shill; I don't use them) you.com is an interesting search engine that is not very well known. I'd be much happier seeing it be used for that than some we.com type branding play (although I take your point that in the hands of some branding expert it could have maybe a more clear connotation than anything related to search.
chris_f 4y ago
Here are some more example queries: https://you.com/niche/pytorch?q=install+osx&fromSearchBar=true https://you.com/niche/pytorch?q=install+osx&fromSearchBar=tr... https://you.com/niche/pytorch?q=save+trained+model https://you.com/niche/pytorch?q=save+trained+model https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GPU https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GP...
- learndeeply 4y ago
  For the first example the snippet isn't correct, it just says `conda list`. The website (TutorialPoint) it links to is also wrong and useless.
  - chris_f 4y ago
    Appreciate the feedback, it is helpful! The relevance tuning is a continuous work in progress.
- p1esk 4y ago
  Sounds like a good idea, but I'd like to see how is it better than googling these phrases together with pytorch keyword. Not saying it's not better, just not clear if it is.
  - chris_f 4y ago
    In the broadest sense, a highly targeted search engine (like this) can provide better results because Google has to determine user search intent AND return the right results from trillions of webpages. The advantage of this search engine is that all of the users are looking for the same type of information, and the result sources can be curated to ensure high quality and relevant results. The more targeted the topic, the harder the time Google has to provide quality signal through the noise of SEO and sheer volume of content on the web. In addition to the above, the UI provides some cool features like allowing horizontal scrolling of sources to provider higher information density (important for discovery), and some source content can be viewed in the side pane without leaving the page. But ultimately it would be good to hear if this approach does make it easier to find relevant and higher quality Pytorch info.
    - p1esk 4y ago
      I just tried searching for: pytorch Distributed model parallel, and google fails completely to return relevant results in the first few pages, even if I put Distributed model parallel in quotes. You.com also mostly fails, but it returns a section specifically from Discuss Pytorch forums which does have relevant results, so yes, this is a good sign.
      - chris_f 4y ago
        That's great! And I think the results even improve a little more if you just search 'distributed model parallel' without 'pytorch' [0] So still some work to do, but I've personally found this type of search engine really valuable for some of my interests where I am able to curate the sites that might not normally rank, but I know are higher quality. [0] https://you.com/niche/pytorch?q=distributed+model+parallel https://you.com/niche/pytorch?q=distributed+model+parallel
mendeza 4y ago
If you could search through github projects (torchvision, mmdetection, etc) and index high quality pytorch snippets, that would be awesome. I learn a lot about how to manipulate tensors and achieving specific deep learning tasks from GitHub projects, that’s how I answer various questions on how to implement key ideas with pytorch.
- chris_f 4y ago
  Good stuff! Github is one of the sources, but not specific repos. I actually think we can break out Github into individual repo sources pretty easily.
wanderingmind 4y ago
This is great. However how to find other such niche searches. A request for you.com/niche returns a 404 page which means you cannot find other niche searches like this one
- chris_f 4y ago
  Yeah, that's a good point. As new ones get built, we will need a way to make them discoverable. Also, just as a heads up, there is a link at the bottom of the search page [0] if anyone would like to build one of these search pages on any topic that might be interesting. It's just a form now bc we are working through the process. [0] https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=1193569441872791 https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=119356944...
  - wanderingmind 4y ago
    Thank you Chris. If I may add, maybe have an approach to get some community feedback on such niche search as well. This can help to narrow down the list. Great work. I'm going to use this search for my work
kjkisielewicz 4y ago
Niche search is a huge problem with Google and other big players these days, happy someone’s facing this issue head on. Would be great if you could do this for Web3 / crypto, and other such niches where tons of new info is coming in and poorly indexed
- chris_f 4y ago
  Any specific sites? Happy to spin one of these up for you focused on Web3. These work the best when the search engine creator has domain expertise to showcase the best sources. I could try to find some sources, but my guess at the best Web3 sites would probably miss the mark.
- herval 4y ago
  What kind of info do you need to find on the crypto space that can’t be easily found with Thegraph or similar tools?

65 points by chris_f 4y ago

cariaso 4y ago

that seems like a super valuable domain being very under used.

version_five 4y ago
I think (at the risk of sounding like a shill; I don't use them) you.com is an interesting search engine that is not very well known. I'd be much happier seeing it be used for that than some we.com type branding play (although I take your point that in the hands of some branding expert it could have maybe a more clear connotation than anything related to search.

chris_f 4y ago

Here are some more example queries: https://you.com/niche/pytorch?q=install+osx&fromSearchBar=true https://you.com/niche/pytorch?q=install+osx&fromSearchBar=tr... https://you.com/niche/pytorch?q=save+trained+model https://you.com/niche/pytorch?q=save+trained+model https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GPU https://you.com/niche/pytorch?q=check+if+pytorch+is+using+GP...

learndeeply 4y ago
For the first example the snippet isn't correct, it just says `conda list`. The website (TutorialPoint) it links to is also wrong and useless.
- chris_f 4y ago
  Appreciate the feedback, it is helpful! The relevance tuning is a continuous work in progress.
p1esk 4y ago
Sounds like a good idea, but I'd like to see how is it better than googling these phrases together with pytorch keyword. Not saying it's not better, just not clear if it is.
- chris_f 4y ago
  In the broadest sense, a highly targeted search engine (like this) can provide better results because Google has to determine user search intent AND return the right results from trillions of webpages. The advantage of this search engine is that all of the users are looking for the same type of information, and the result sources can be curated to ensure high quality and relevant results. The more targeted the topic, the harder the time Google has to provide quality signal through the noise of SEO and sheer volume of content on the web. In addition to the above, the UI provides some cool features like allowing horizontal scrolling of sources to provider higher information density (important for discovery), and some source content can be viewed in the side pane without leaving the page. But ultimately it would be good to hear if this approach does make it easier to find relevant and higher quality Pytorch info.
  - p1esk 4y ago
    I just tried searching for: pytorch Distributed model parallel, and google fails completely to return relevant results in the first few pages, even if I put Distributed model parallel in quotes. You.com also mostly fails, but it returns a section specifically from Discuss Pytorch forums which does have relevant results, so yes, this is a good sign.
    - chris_f 4y ago
      That's great! And I think the results even improve a little more if you just search 'distributed model parallel' without 'pytorch' [0] So still some work to do, but I've personally found this type of search engine really valuable for some of my interests where I am able to curate the sites that might not normally rank, but I know are higher quality. [0] https://you.com/niche/pytorch?q=distributed+model+parallel https://you.com/niche/pytorch?q=distributed+model+parallel

mendeza 4y ago

If you could search through github projects (torchvision, mmdetection, etc) and index high quality pytorch snippets, that would be awesome. I learn a lot about how to manipulate tensors and achieving specific deep learning tasks from GitHub projects, that’s how I answer various questions on how to implement key ideas with pytorch.

chris_f 4y ago
Good stuff! Github is one of the sources, but not specific repos. I actually think we can break out Github into individual repo sources pretty easily.

wanderingmind 4y ago

This is great. However how to find other such niche searches. A request for you.com/niche returns a 404 page which means you cannot find other niche searches like this one

chris_f 4y ago
Yeah, that's a good point. As new ones get built, we will need a way to make them discoverable. Also, just as a heads up, there is a link at the bottom of the search page [0] if anyone would like to build one of these search pages on any topic that might be interesting. It's just a form now bc we are working through the process. [0] https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=1193569441872791 https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=119356944...
- wanderingmind 4y ago
  Thank you Chris. If I may add, maybe have an approach to get some community feedback on such niche search as well. This can help to narrow down the list. Great work. I'm going to use this search for my work

kjkisielewicz 4y ago

Niche search is a huge problem with Google and other big players these days, happy someone’s facing this issue head on. Would be great if you could do this for Web3 / crypto, and other such niches where tons of new info is coming in and poorly indexed

chris_f 4y ago
Any specific sites? Happy to spin one of these up for you focused on Web3. These work the best when the search engine creator has domain expertise to showcase the best sources. I could try to find some sources, but my guess at the best Web3 sites would probably miss the mark.
herval 4y ago
What kind of info do you need to find on the crypto space that can’t be easily found with Thegraph or similar tools?