jwngr

8 ms·

61.
by jwngr 9y ago
I'm storing all the search results and will do some analysis on the data. Maybe I'll even get around to adding a new page with some of the interesting stuff I find.
62.
by jwngr 9y ago
I actually had a friend suggest it to me and the Neo4j docs happen to be one of the many tabs I currently have open. I was already so far into using SQLite for this project and I wanted to ship it, so I decided to stick with what I had. I w
63.
by jwngr 9y ago
I never actually touch any of the source HTML. I think that would simply take way too long and would probably result in some very high bandwidth charges. I use three tables from a public dump of Wikipedia's database, which unfortunatel
64.
by jwngr 9y ago
Good suggestion! I intended to have that exact button, but couldn't find a way to put it in the UI without making things more confusing. I expect I'll add it in the future. Thanks for the suggestion!
65.
by jwngr 9y ago
No, not an idiot. I didn't use the best terms. I updated them to say "Web (frontend) hosting" which are my static files (the HTML, JS, CSS) which is deployed to Firebase Hosting and "Server (backend) hosting" which
66.
by jwngr 9y ago
Thanks! It's weird, but as soon as I tried upgrading to an identical GCP machine running Debian 9 (stretch) and run my database creation process[1], it took many, many times longer to even download the several GB dumps from Wikipedia v
67.
by jwngr 9y ago
The database creation script[1] has a lot of Unix junk in it, but reading through the comments and echo statements should give you an idea of what it does. The end result is a SQLite database with a size of about 9 GB which has four tables,
68.
by jwngr 9y ago
Thanks! I'm glad you asked. I actually do what I call a bi-directional breadth first search[1]. The gist of it is that instead of just doing a BFS from the source node until I reach the target node, I do a reverse BFS from the target n
69.
by jwngr 9y ago
Yes, definitely a feature I'd like to add. I can change the opening the page on Wikipedia to happen on double click instead of single click. I am still a d3 noob and need to figure out how to implement the highlight path thing.
70.
by jwngr 9y ago
This is an interesting question that I'd like to answer now that I have all the data. I am curious to see how long it will take to find a solution as I believe even the most efficient algorithms for this have a high runtime complexity.
71.
by jwngr 9y ago
I clearly should've done some more user testing ;) I think that's a great suggestion and would be much better than showing an error message. I'll update that once I have some time when the server isn't crashing!
72.
by jwngr 9y ago
I'm definitely not the first to think of it or build a tool for it (lots of similar projects gave me inspiration), but I think I'm the first to make it really fast and with a nice usable UI. And to actually open source the code so
73.
by jwngr 9y ago
[copy of answer from below] The Wikipedia database doesn't differentiate links which appear in the main article versus in the sources or categories sections. It's possible one of the intermediate links is in there. You sometimes n
74.
by jwngr 9y ago
Yeah unfortunately I don't know of any way to differentiate the different types of links. Wikipedia's pagelinks database doesn't different them. I agree it's undesireable but I just cannot figure out how to cull them.
75.
by jwngr 9y ago
Unfortunately I'm not aware of a way to distinguish between the two. Wikipedia stores both types of links in the same database. I would love to cull out all the links in category boxes and sources. If anyone has any ideas, let me know!
76.
by jwngr 9y ago
I'll look into it. Interestingly, I'm using[1] one of the default d3 color scales, which I assumed would be color blind friendly out of the box. [1] https://github.com/jwngr/sdow/blob/a2699dc95d884ec
77.
by jwngr 9y ago
Yes, Sthephen Dolan's project (along with others) was definitely an inspiration for me! Although, I like to think I made a lot of performance and design improvements over his work. I used to have an acknowledgements section in my READM
78.
by jwngr 9y ago
I considered this and may eventually add an option to ignore those kinds of pages, but I ultimately felt like the current mode remains more true to my goal for the project which is to traverse the links as any human would be able to. By the
79.
by jwngr 9y ago
Glad you found one of the Easter eggs :D The graph visualization / performance is definitely not ideal. I spent a ton of time trying to make d3 more performant and layout the graph more nicely, but ultimately I just had to cut my losse
80.
by jwngr 9y ago
The Wikipedia database doesn't differentiate links which appear in the main article versus in the sources or categories sections. It's possible one of the intermediate links is in there. You sometimes need to do a CTRL+f in "
81.
by jwngr 9y ago
I think this is going to be extremely computationally intensive. One of the big performance wins I got when designing the search algorithm[1] was visiting as few nodes in the graph as possible (which I did via a bi-directional breadth-first
82.
by jwngr 9y ago
Creator here. Six Degrees of Wikipedia is a side project I've been sporadically hacking on over the past few years. It was an interesting technical challenge and it's fun to play with the end result. Here's the tech stack:
83.
Show HN: Six Degrees of Wikipedia(sixdegreesofwikipedia.com)
1176 points by jwngr 9y ago|324 comments
84.
Tic-tac-tic-tac-toe(tic-tac-tic-tac-toe.firebaseapp.com)
1 points by jwngr 9y ago|0 comments
85.
Firebase JavaScript SDK on GitHub(github.com)
2 points by jwngr 9y ago|0 comments
86.
by jwngr 10y ago
Not a built-in way per-se, but you do have a handful of options. You can use a third party scheduling service like https://cron-job.org/ . You could also use something like https://cloud.google.com/solutions
87.
Every NFL Score Ever(youtube.com)
1 points by jwngr 10y ago|0 comments
88.
by jwngr 10y ago
(Firebase engineer here) The old SDKs will continue to work! We worked extremely hard to make sure everything was backwards compatible with them. We most certainly do not want to break any existing customer apps. Check out the migration gui
89.
by jwngr 10y ago
That's what the Database Security Rules [1] are for. If you want to learn more, check out my I/O talk tomorrow called "The key to Firebase security" [2]. [1] http://firebase.google.com/docs/database&
90.
VueFire – Firebase meets Vue.js(firebase.com)
2 points by jwngr 10y ago|0 comments

More ›