daoudc

10 ms·

61.
A community powered search engine(youtube.com)
1 points by daoudc 4y ago|0 comments
62.
One hour to fix a broken crawler(youtube.com)
1 points by daoudc 4y ago|0 comments
63.
by daoudc 4y ago
We have a distributed crawler that volunteers can install as a Firefox extension to crawl the web ;)
64.
by daoudc 4y ago
Thanks, the organisation is a large part of what we're working on. We do have other plans for financing, e.g. premium versions of the webmaster tools.
65.
by daoudc 4y ago
Tech stack is mainly python for back end and vanilla JS for front end right now. However we'd like to move a lot of the back end to Rust eventually, and for the front end we will probably adopt some framework.
66.
The Book of Mwmbl: a free, non-profit search engine(book.mwmbl.org)
3 points by daoudc 4y ago|8 comments
67.
Community powered search engine – part 1(youtube.com)
1 points by daoudc 4y ago|0 comments
68.
Fixing a broken web crawler in 1h(youtube.com)
2 points by daoudc 4y ago|0 comments
69.
Mwmbl Fall 2022 Update(blog.mwmbl.org)
2 points by daoudc 4y ago|0 comments
70.
by daoudc 4y ago
This is great! If you're interested in search engine UX, I'd be interested in incorporating some of these ideas into https://mwmbl.org - would love to get your input - join us on Matrix https://matrix.to&#x2
71.
by daoudc 4y ago
Not at all, feedback is very welcome!
72.
by daoudc 4y ago
Fascinating, thanks for this! Looks like it became controversial because of a lack of transparency.
73.
by daoudc 4y ago
Our index is still very small. Help make it bigger!
74.
by daoudc 4y ago
Feel free to read the source code if you're concerned about what it does!
75.
by daoudc 4y ago
1% of 1%
76.
by daoudc 4y ago
Roughly this https://github.com/mwmbl/mwmbl/wiki/Crawler-Phase-2-Design It changed a bit in the implementation.
77.
by daoudc 4y ago
The data is available! Not currently documented but check out the code for the API.
78.
by daoudc 4y ago
Wikipedia has to deal with similar issues. It's not easy, but it should be possible.
79.
by daoudc 4y ago
Good point, this is not going to be easy to get right.
80.
by daoudc 4y ago
Yeah the search quality has a way to go!
81.
by daoudc 4y ago
That word (Mwmbl) is actually a link, but I think the link colour is not different enough from the text. Let me see what I can do.
82.
by daoudc 4y ago
I'm hoping that having community moderated search results will limit this problem. The problem will be SEO people trying to infiltrate the community, which may be tricky to solve. But Wikipedia has had to grapple with similar problems,
83.
by daoudc 4y ago
Do people use YaCy? I found it very slow the last time I tried it
84.
The world needs a non-profit search engine(daoudclarke.net)
533 points by daoudc 4y ago|383 comments
85.
by daoudc 4y ago
That's pretty cool, thanks! The loading page between each section seems unnecessary though.
86.
by daoudc 4y ago
This looks cool, but I'm not familiar with the concept of a "tour" for a project. Can you point me to an example? I'm currently trying to promote my open source project Mwmbl (in particular the crawler extension https:&
87.
Show HN: An open source web crawler for the Mwmbl non-profit search engine(github.com)
6 points by daoudc 4y ago|0 comments
88.
by daoudc 4y ago
It is an actual web crawler. I should have put a link to the server side code.
89.
Why a monolithic back-end makes sense for Mwmbl(github.com)
1 points by daoudc 4y ago|0 comments
90.
Show HN: An open source web crawler for the Mwmbl non-profit search engine(github.com)
4 points by daoudc 4y ago|4 comments

More ›