4 ms·

B is for Billion: Wordnik passes 9 billion record mark with MongoDB

32 points by johnny99 16y ago

pierrefar 16y ago
Very sparse on details. Are they partitioning the data? If so how? Is it MongoDB shards or at the application level or...? What's their average record size? Do they see a long-tail kind of access such that MongoDB really does keep the most commonly used records in memory all the time? How many machines are they serving from? What are their specs (particularly RAM)?
- kneath 16y ago
  This is pretty well covered in this presentation: http://www.slideshare.net/fehguy/migrating-from-mysql-to-mongodb-at-wordnik http://www.slideshare.net/fehguy/migrating-from-mysql-to-mon... A few unanswered questions (sharding? Looks like none to me), but a good bit of numbers and specifics on their current setup.
  - ericflo 16y ago
    The 47.7 queries per second figure in those slides surprised me a bit. I'd like to know more details there...is that for a single node, or the entire cluster?
    - fehguy 16y ago
      That's per api server, we have 4 api servers.
  - meghan 16y ago
    Video of this talk is available here: http://blip.tv/file/3621377 http://blip.tv/file/3621377 This article also has some details: http://www.royans.net/arch/mongodb-migration-from-mysql-at-wordnik/ http://www.royans.net/arch/mongodb-migration-from-mysql-at-w...
- deleted 16y ago
  [deleted]
- johnny99 16y ago
  Some of these questions are answered in a separate set of slides, which the author links to in the comments of the post. Hardware looks like a single server, 2x4 core CPUs, 32 GB of RAM, and a FC SAN.
  - ericflo 16y ago
    Single node, eh? I thought MongoDB was not supposed to be run in that configuration.
    - thegoleffect 16y ago
      They're probably running 1+ Mongod process/node per core - not exactly ideal though.
      - fehguy 16y ago
        We run one mongod master, one slave (soon to be two slaves). These are used across all the api servers.
christopherdone 16y ago
Well I'd never heard of Wordnik but this is very cool and I'll be using it. This is like a respectable Dictionary.com+ Urban Dictionary. I don't see the category theory definition of a monad on it, though. Which is somewhat odd.
- fehguy 16y ago
  Hi Christopher, glad you like Wordnik. Do you mean you couldn't find this? http://www.wordnik.com/words/monad http://www.wordnik.com/words/monad
earle 16y ago
This is still a relatively small production database as far as large DB instances go.
- herewego 16y ago
  As far as production websites go, you are completely wrong.

32 points by johnny99 16y ago

Very sparse on details. Are they partitioning the data? If so how? Is it MongoDB shards or at the application level or...? What's their average record size? Do they see a long-tail kind of access such that MongoDB really does keep the most commonly used records in memory all the time? How many machines are they serving from? What are their specs (particularly RAM)?

kneath 16y ago
This is pretty well covered in this presentation: http://www.slideshare.net/fehguy/migrating-from-mysql-to-mongodb-at-wordnik http://www.slideshare.net/fehguy/migrating-from-mysql-to-mon... A few unanswered questions (sharding? Looks like none to me), but a good bit of numbers and specifics on their current setup.
- ericflo 16y ago
  The 47.7 queries per second figure in those slides surprised me a bit. I'd like to know more details there...is that for a single node, or the entire cluster?
  - fehguy 16y ago
    That's per api server, we have 4 api servers.
- meghan 16y ago
  Video of this talk is available here: http://blip.tv/file/3621377 http://blip.tv/file/3621377 This article also has some details: http://www.royans.net/arch/mongodb-migration-from-mysql-at-wordnik/ http://www.royans.net/arch/mongodb-migration-from-mysql-at-w...
deleted 16y ago
[deleted]
johnny99 16y ago
Some of these questions are answered in a separate set of slides, which the author links to in the comments of the post. Hardware looks like a single server, 2x4 core CPUs, 32 GB of RAM, and a FC SAN.
- ericflo 16y ago
  Single node, eh? I thought MongoDB was not supposed to be run in that configuration.
  - thegoleffect 16y ago
    They're probably running 1+ Mongod process/node per core - not exactly ideal though.
    - fehguy 16y ago
      We run one mongod master, one slave (soon to be two slaves). These are used across all the api servers.

christopherdone 16y ago

Well I'd never heard of Wordnik but this is very cool and I'll be using it. This is like a respectable Dictionary.com+ Urban Dictionary. I don't see the category theory definition of a monad on it, though. Which is somewhat odd.

fehguy 16y ago
Hi Christopher, glad you like Wordnik. Do you mean you couldn't find this? http://www.wordnik.com/words/monad http://www.wordnik.com/words/monad

earle 16y ago

This is still a relatively small production database as far as large DB instances go.

herewego 16y ago
As far as production websites go, you are completely wrong.