4 ms·

Isn't "3.1 million disks and 43 million tracks" only a small amount of data and "0.1 seconds" slow (assuming this is only metadata, not the music itself) at cur

by jmount 15y ago

Isn't "3.1 million disks and 43 million tracks" only a small amount of data and "0.1 seconds" slow (assuming this is only metadata, not the music itself) at current machine scales? The article mentions use of a 300 GB 7200 RPM disk drive but I would assume (couldn't find it in the article) they would easily have more than 8GB ram- more than enough to hold everything plus indexing structures.

endersshadow 15y ago
As somebody that works in data warehousing, this strikes me as a very small data set. To have a full text index and return a simple search on "Adele" in 0.8 seconds on only 3.1 million rows (or even 43 million) is not particularly impressive. Was this supposed to be something that surprised me? I regularly deal with billions of rows of very wide data, so I admit that my sense of scale is a little...skewed.
- jbigelow76 15y ago
  Maybe it's because RavenDB is a document database as opposed to an OLAP database (I'm assuming that's what you refer to when you say billions of rows with wide data and not another NoSQL like Mongo). Indexes and data arrangement between the two types are not an apples to apples comparison.
  - endersshadow 15y ago
    I was mostly referring to a relational database, but I do a lot of work with multidimensional databases (mostly SQL Server Analysis Services). You don't typically do full text searching on a cube. In warehousing, you often get wide tables because they're optimized for the reads, not the writes, so are typically second normal form (2NF) rather than the third normal form (3NF) that most people are familiar with.
aaronblohowiak 15y ago
The article says: >Querying for “Query:Adele” again? 32 milliseconds. That would suggest that the 0.1 is for a cache miss to disk, and that would be hinted at by the phrase "cold boot" in an earlier paragraph.
- jmount 15y ago
  Yes, Query:James – 0.1 seconds is a repeated query, so I assumed it was a warm query also.

by jmount 15y ago

endersshadow 15y ago

As somebody that works in data warehousing, this strikes me as a very small data set. To have a full text index and return a simple search on "Adele" in 0.8 seconds on only 3.1 million rows (or even 43 million) is not particularly impressive. Was this supposed to be something that surprised me? I regularly deal with billions of rows of very wide data, so I admit that my sense of scale is a little...skewed.

jbigelow76 15y ago
Maybe it's because RavenDB is a document database as opposed to an OLAP database (I'm assuming that's what you refer to when you say billions of rows with wide data and not another NoSQL like Mongo). Indexes and data arrangement between the two types are not an apples to apples comparison.
- endersshadow 15y ago
  I was mostly referring to a relational database, but I do a lot of work with multidimensional databases (mostly SQL Server Analysis Services). You don't typically do full text searching on a cube. In warehousing, you often get wide tables because they're optimized for the reads, not the writes, so are typically second normal form (2NF) rather than the third normal form (3NF) that most people are familiar with.

aaronblohowiak 15y ago

The article says: >Querying for “Query:Adele” again? 32 milliseconds. That would suggest that the 0.1 is for a cache miss to disk, and that would be hinted at by the phrase "cold boot" in an earlier paragraph.

jmount 15y ago
Yes, Query:James – 0.1 seconds is a repeated query, so I assumed it was a warm query also.