jzawodn

20 ms·

181.
by jzawodn 15y ago
Ugh. Reply fail. Sorry. I was intending to reply to the parent comment, not yours.
182.
by jzawodn 15y ago
Uhm, what? I was trying to counter balance some of the crap I read in the original posting--was that really not clear?
183.
by jzawodn 15y ago
I think that most people who argue that such a migration "isn't that bad" haven't actually done it. Or at least they haven't done it for anything sizable.
184.
by jzawodn 15y ago
Agreed. This is opinion. Mine is based on years of experience, but I got the impression that the original article was written based on some cherry-picked reports of what a few companies said (as opposed to actually being there and doing it)
185.
by jzawodn 15y ago
You hit the nail right on the head.
186.
by jzawodn 15y ago
Yeah, I guess he wasn't concerned about putting any effort into making it easy for others to read his thoughts. sigh Written text is supposed to be written to be parsed by humans. Why do people forget that?
187.
by jzawodn 15y ago
As the original author of that quote, no. I was running MongoDB in "safe" mode where it doesn't return until the query has been ack'd. Furthermore, I was running with w=2 meaning that at least one other note also had to ack the data.
188.
by jzawodn 15y ago
The photos are removed once the posting is no longer live on the site (roughly). As for how many, I'd have to dig a bit to find that out...
189.
by jzawodn 15y ago
Exactly. The "set amount of time" typically hovers around 60 days, though our archiving process has been off for several months while the migration took place. So we have some catching up to do--somewhere in the neighborhood of 150 million
190.
by jzawodn 15y ago
~2.2 billion is a ton of listings. What you have to realize is that the craigslist wasn't in hundreds of cities on day #1. In recent years, we've had tens of millions of "live" ads on the site, but it took a bit of time to grow to that si
191.
by jzawodn 15y ago
Yup. You hit the nail on the head.
192.
by jzawodn 15y ago
Yes, it is a bit mind-blowing. The only DB boxes I can consistently get data out of at a high rate are those with Fusion-io cards in them. And we're not heavily normalized--just a handful of tables.
193.
by jzawodn 16y ago
That's the dark version.
194.
by jzawodn 16y ago
Any chance of publishing the Xresources version (or at least hex codes in that file) for the "white" version?
195.
MongoDB Pre-Splitting for Faster Data Loading and Importing(blog.zawodny.com)
31 points by jzawodn 16y ago|0 comments
196.
by jzawodn 16y ago
Uh. What happened to evoke this response? You can email jzawodn@craigslist if you don't want to say so in public, but we're not anti-developer. We are anti-abuse (which we see A LOT of) and anti-TOS violations. So I'm curious to understand
197.
by jzawodn 16y ago
#1: currently no, but I'm about to change that and have them do a SAVE every few hours or so. #2: my implementation is pretty far along in my testing but I do need to polish it up, update the docs, and test some edge cases #3: the idea with
198.
Redis Sharding at Craigslist(blog.zawodny.com)
97 points by jzawodn 16y ago|16 comments
199.
by jzawodn 16y ago
It's definitely a worthwhile upgrade for those of us running it in production in demanding environments. Many of the memory optimizations alone will make upgrading worth it.
200.
by jzawodn 16y ago
I hate to "met too" but I'm a big fan of it as well! I've built a few of the python examples into their perl equivalents over the last few months. Someday I should throw 'em up on github.
201.
by jzawodn 16y ago
The touchpad drives me nuts on that machine. :-( Thankfully I have a Thinkpad 410s replacing it tomorrow.
202.
by jzawodn 16y ago
We're self hosted (craigslist.org) and the cost is pretty much what you see if your price out the Fusion-io gear (I'd have to go dig that up).
203.
by jzawodn 16y ago
We're using Fusion-io SSDs in a lot of our database (MySQL) servers. It makes a massive difference.
204.
by jzawodn 16y ago
Of course.
205.
by jzawodn 16y ago
I figured it'd be useful to do... just need to block off a bit of time. Perhaps in the next 24 hours if I'm lucky.
206.
by jzawodn 16y ago
Heh. Sorry to shock you. "Trust but verify" is often useful with infrastructure software. But clearly you have a different approach. No big deal.
207.
by jzawodn 16y ago
Uhm, no. If you read that post you'd see that I was mainly interested in the per-key overhead. I'd seen it spoken of as being in the 100 bytes/key range but never saw confirmation of that. So I did some back of the envelope calculations
208.
by jzawodn 16y ago
Yeah, a few folks have mentioned it already. And I remember seeing some good features in the video I watched about it several weeks ago...
209.
by jzawodn 16y ago
That's a great point about async libs. Look no farther than MySQL to see how much pain it can be to get a good one in your language of choice.
210.
by jzawodn 16y ago
Hmm... Very interesting idea, thanks!

More ›