5 ms·

MongoDB Finds A Major Adopter In Craigslist

95 points by DanielRibeiro 16y ago

citizenkeys 16y ago
This video is a presentation by Jeremy Zawodny, who co-wrote O'Reilly's "High Performance MySQL" ( http://oreilly.com/catalog/9780596101718 http://oreilly.com/catalog/9780596101718 ). If there's anybody worth watching a video about switching from MySQL to MongoDB, it's that guy.
- DanielRibeiro 16y ago
  For a direct link to the 34 min video: http://www.10gen.com/video/mongosv2010/craigslist http://www.10gen.com/video/mongosv2010/craigslist
nwmt 16y ago
Not to take away from the substance of the article itself, but is anyone else surprised that they have 2 billion "documents", which presumably means active ads/listings? That seems like an awful lot.
- runevault 16y ago
  I would have expected more, personally. Craigslist is massive, popular, and been around a long time. That ads up to a TON of listings.
  - jzawodn 16y ago
    ~2.2 billion is a ton of listings. What you have to realize is that the craigslist wasn't in hundreds of cities on day #1. In recent years, we've had tens of millions of "live" ads on the site, but it took a bit of time to grow to that size.
- HarrisonFisk 16y ago
  MongoDB is being used for historical archiving, not for the live site itself. The big reason being that changing table schemas for very large sets of old data is painful with MySQL. So the 2 billion number would be any ad/listing older than a set amount of time. The live data is < 1 TB and is still stored in MySQL.
  - jzawodn 16y ago
    Exactly. The "set amount of time" typically hovers around 60 days, though our archiving process has been off for several months while the migration took place. So we have some catching up to do--somewhere in the neighborhood of 150 million postings, last I counted.
    - biot 16y ago
      I've been hearing some good things about Riak lately and their masterless implementation seems quite interesting. Did Riak ever make your radar and, if so, what were the disadvantages that made you choose MongoDB? Were I to guess based on the video, I would say lack of a Perl client and you'd probably end up having to roll too many of your own solutions on top of it?
- jcampbell1 16y ago
  This looks like a data warehousing of the archive. The two billion listings probably represents all expired ads ever. There is no way they have 2 billion active ads at any one time.
  - Hawramani 16y ago
    The above comment is correct. The archive does have to be accessed by users though, since users can access listings from many years back. The entire archive seems to be under 4 TB from what he described in the video (2 billion documents at 2 kilobytes each). They do not retain photos.
    - jzawodn 16y ago
      Yup. You hit the nail on the head.
      - ratsbane 16y ago
        How much photo data do you handle? How long do you keep it?
        jzawodn 15y ago
        The photos are removed once the posting is no longer live on the site (roughly). As for how many, I'd have to dig a bit to find that out...
Jak3t 16y ago
It's mind-blowing to read that they could get data in mongoDB faster than to get it out of MySQL. I wonder if their structure was so complex or if the queries were complex as well, or is mongoDB that fast? Anyway, great to see sites like this embracing new technologies and revealing the details. [edit] I guess watching the video explains a lot more. Do watch!
- jzawodn 16y ago
  Yes, it is a bit mind-blowing. The only DB boxes I can consistently get data out of at a high rate are those with Fusion-io cards in them. And we're not heavily normalized--just a handful of tables.
zefhous 16y ago
The link should probably be to this page: http://blog.mongodb.org/post/5545198613/mongodb-live-at-craigslist http://blog.mongodb.org/post/5545198613/mongodb-live-at-crai...
zmitri 16y ago
I recently went to a Google Tech Talk on MongoDB, and although the presentation wasn't very hacker oriented, I must say the setup of replication sets and sharding is ridiculously easy (http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+Architecture http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+...). I started up a project using CouchDB and Redis, but was impressed enough with Mongo's scalability/ease of use that I think I might switch over to MongoDB for a bit.
danielrhodes 16y ago
The big tests for Mongo are how well it behaves under heavy load, how easy it is to shard/replicate data at large scale (e.g. what Zawodny was talking about in his presentation about Craigslist having 100 MySQL boxes), and to what degree data is recoverable when inevitable failures happen (there are some big questions here with Mongo since it appears it trades off ACID compliance for speed). It's great to see bigger users using Mongo because that's where these tests really take place. For example, it seems Cassandra got tested this way at Facebook/Reddit/Digg etc. and didn't cut it.
- EwanToo 16y ago
  Pretty much all distributed databases trade off ACID for speed, it's a fundamental issue with database design. If you've got 20 database servers, potentially on the other side of the planet from each other, if you were to be ACID compliant throughout the cluster, each write commit would be delayed up to 500ms or more. If you're running a pair of servers in the same room, as is normally the case with a traditional "active-active" database cluster, a network delay of 1ms often isn't a significant impact.
- Luyt 16y ago
  See http://mongodb-is-web-scale.com/ http://mongodb-is-web-scale.com/ for a humorous angle on the speed vs. ACID compliance tradeoff.
ericd 16y ago
A bit off topic, but since Jeremy seems to be in here, I've been wanting to thank you for High Performance MySQL - it's one of the best technical books I've read in recent years, with a lot of insights that had been hard/impossible to piece together definitively by reading online articles/docs alone. It has taken a lot of uncertainty out of things for me as a dabbling db admin trying to scale.

95 points by DanielRibeiro 16y ago

This video is a presentation by Jeremy Zawodny, who co-wrote O'Reilly's "High Performance MySQL" ( http://oreilly.com/catalog/9780596101718 http://oreilly.com/catalog/9780596101718 ). If there's anybody worth watching a video about switching from MySQL to MongoDB, it's that guy.

DanielRibeiro 16y ago
For a direct link to the 34 min video: http://www.10gen.com/video/mongosv2010/craigslist http://www.10gen.com/video/mongosv2010/craigslist

nwmt 16y ago

Not to take away from the substance of the article itself, but is anyone else surprised that they have 2 billion "documents", which presumably means active ads/listings? That seems like an awful lot.

runevault 16y ago
I would have expected more, personally. Craigslist is massive, popular, and been around a long time. That ads up to a TON of listings.
- jzawodn 16y ago
  ~2.2 billion is a ton of listings. What you have to realize is that the craigslist wasn't in hundreds of cities on day #1. In recent years, we've had tens of millions of "live" ads on the site, but it took a bit of time to grow to that size.
HarrisonFisk 16y ago
MongoDB is being used for historical archiving, not for the live site itself. The big reason being that changing table schemas for very large sets of old data is painful with MySQL. So the 2 billion number would be any ad/listing older than a set amount of time. The live data is < 1 TB and is still stored in MySQL.
- jzawodn 16y ago
  Exactly. The "set amount of time" typically hovers around 60 days, though our archiving process has been off for several months while the migration took place. So we have some catching up to do--somewhere in the neighborhood of 150 million postings, last I counted.
  - biot 16y ago
    I've been hearing some good things about Riak lately and their masterless implementation seems quite interesting. Did Riak ever make your radar and, if so, what were the disadvantages that made you choose MongoDB? Were I to guess based on the video, I would say lack of a Perl client and you'd probably end up having to roll too many of your own solutions on top of it?
jcampbell1 16y ago
This looks like a data warehousing of the archive. The two billion listings probably represents all expired ads ever. There is no way they have 2 billion active ads at any one time.
- Hawramani 16y ago
  The above comment is correct. The archive does have to be accessed by users though, since users can access listings from many years back. The entire archive seems to be under 4 TB from what he described in the video (2 billion documents at 2 kilobytes each). They do not retain photos.
  - jzawodn 16y ago
    Yup. You hit the nail on the head.
    - ratsbane 16y ago
      How much photo data do you handle? How long do you keep it?
      - jzawodn 15y ago
        The photos are removed once the posting is no longer live on the site (roughly). As for how many, I'd have to dig a bit to find that out...

Jak3t 16y ago

It's mind-blowing to read that they could get data in mongoDB faster than to get it out of MySQL. I wonder if their structure was so complex or if the queries were complex as well, or is mongoDB that fast? Anyway, great to see sites like this embracing new technologies and revealing the details. [edit] I guess watching the video explains a lot more. Do watch!

jzawodn 16y ago
Yes, it is a bit mind-blowing. The only DB boxes I can consistently get data out of at a high rate are those with Fusion-io cards in them. And we're not heavily normalized--just a handful of tables.

zefhous 16y ago

The link should probably be to this page: http://blog.mongodb.org/post/5545198613/mongodb-live-at-craigslist http://blog.mongodb.org/post/5545198613/mongodb-live-at-crai...

zmitri 16y ago

I recently went to a Google Tech Talk on MongoDB, and although the presentation wasn't very hacker oriented, I must say the setup of replication sets and sharding is ridiculously easy (http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+Architecture http://www.mongodb.org/display/DOCS/Simple+Initial+Sharding+...). I started up a project using CouchDB and Redis, but was impressed enough with Mongo's scalability/ease of use that I think I might switch over to MongoDB for a bit.

danielrhodes 16y ago

The big tests for Mongo are how well it behaves under heavy load, how easy it is to shard/replicate data at large scale (e.g. what Zawodny was talking about in his presentation about Craigslist having 100 MySQL boxes), and to what degree data is recoverable when inevitable failures happen (there are some big questions here with Mongo since it appears it trades off ACID compliance for speed). It's great to see bigger users using Mongo because that's where these tests really take place. For example, it seems Cassandra got tested this way at Facebook/Reddit/Digg etc. and didn't cut it.

EwanToo 16y ago
Pretty much all distributed databases trade off ACID for speed, it's a fundamental issue with database design. If you've got 20 database servers, potentially on the other side of the planet from each other, if you were to be ACID compliant throughout the cluster, each write commit would be delayed up to 500ms or more. If you're running a pair of servers in the same room, as is normally the case with a traditional "active-active" database cluster, a network delay of 1ms often isn't a significant impact.
Luyt 16y ago
See http://mongodb-is-web-scale.com/ http://mongodb-is-web-scale.com/ for a humorous angle on the speed vs. ACID compliance tradeoff.

ericd 16y ago

A bit off topic, but since Jeremy seems to be in here, I've been wanting to thank you for High Performance MySQL - it's one of the best technical books I've read in recent years, with a lot of insights that had been hard/impossible to piece together definitively by reading online articles/docs alone. It has taken a lot of uncertainty out of things for me as a dabbling db admin trying to scale.