5 ms·

The folks at MarkLogic must be unhappy with being singled out in the following paragraph: Some of the companies building the system opposed an early decision b

by rdhyee 13y ago

The folks at MarkLogic must be unhappy with being singled out in the following paragraph: Some of the companies building the system opposed an early decision by the Medicare agency to use database software from a company called MarkLogic, which handles data differently from systems by companies like IBM and Oracle. Some suggest that its unfamiliar nature slowed their work. By mid-November, more than six weeks after the rollout, the MarkLogic database -- essentially the website's virtual filing cabinet and index -- continued to perform below expectations, according to one person who works in the command center. The company's response is recorded as: In interviews, MarkLogic’s executives faulted inadequate computing power and instability at the site’s data center, as well as the failure to properly integrate their product, problems repeatedly cited by other website vendors. Any insight on what is meant by how MarkLogic "handles data differently"?

arosenbaum 13y ago
Not really a slam at all. Customers choose MarkLogic to replace Oracle and DB2 specifically because it "handles data differently". "Schemaless" aka "Schema on read" is the biggest difference between Only "SQL" and "Not Only SQL" systems. Scale-out on commodity hw vs. scale-up on shared storage is another. Customers often choose MarkLogic instead of other NoSQL systems because of it's ACID capabilities, ability to run multi-statement transactions between multiple databases (including Oracle and MarkLogic in the same transaction) and a multitude of enterprise capabilities from government grade security to point-in-time restore. Read more: http://developer.marklogic.com/pubs/architecture/inside-marklogic-server-r7.pdf http://developer.marklogic.com/pubs/architecture/inside-mark...
- realmandan 13y ago
  Totally, not a slam at all. As long as you don't think a company name and "perform below expectations" sitting next to each other in a NYT long-form piece is a BAD thing...
  - arosenbaum 13y ago
    I'll repeat what our CEO said in the NYTimes article "He said MarkLogic is performing up to standard, but “the network and the storage systems are not properly sized and not properly run.” Put any database on top of the resources that were provided and I think performance "below expectations" would have been likely.
acdha 13y ago
MarkLogic is an XML document store, with the usual shift in access patterns which that entails. I could easily see much of the problem simply being an impedance mismatch with devs & other systems expecting SQL-like behavior. I haven't worked with it directly but at several places it has a reputation for high memory requirements and requiring you to scale by getting single massive servers rather than clustering – I'm not sure if that's still true or if it's an issue for healthcare.gov but it could definitely make it harder to scale in a hurry.
- arosenbaum 13y ago
  MarkLogic is an XML document store like Oracle is a CSV file store. We store compressed trees. We require memory - like all databases - because we actually have rich indexes. Why? Scaling by requiring massive servers is Oracle's story, not MarkLogic. We scale out on commodity servers in a cluster just fine. However, we do not magically create more capacity when adding additional VM's on top of over-subscribed network and storage systems. We don't require proprietary shared storage systems (local, NFS, HDFS - just give us bandwidth) or proprietary networks (TCPIP please, 10G preferred, don't need FC). Cheapest way to scale is a cluster of decent sized machines with a decent amount of IO capacity each. {Q: What do you call a database without indexes, transactions or security? A: A file system!}
  - hga 13y ago
    Ouch. As in, per this NYT article: http://www.nytimes.com/2013/11/23/us/politics/tension-and-woes-before-health-website-crash.html?hpw&rref= http://www.nytimes.com/2013/11/23/us/politics/tension-and-wo... the contractors weren't familiar with its paradigm. They got started so late (not until February in earnest from what we've heard), it sounds like they just didn't have the time to learn it ... in a project (micro-)managed by political and bureaucratic types in the government. Not learning it well enough would likely go along with not provisioning the necessary resources. Usual lesson of "don't try to do too many new things if you're on a tight time and/or money budget". Unfortunately, given that this is almost entirely a political exercise, I don't see how you're going to avoid getting some scapegoating.

by rdhyee 13y ago

arosenbaum 13y ago

Not really a slam at all. Customers choose MarkLogic to replace Oracle and DB2 specifically because it "handles data differently". "Schemaless" aka "Schema on read" is the biggest difference between Only "SQL" and "Not Only SQL" systems. Scale-out on commodity hw vs. scale-up on shared storage is another. Customers often choose MarkLogic instead of other NoSQL systems because of it's ACID capabilities, ability to run multi-statement transactions between multiple databases (including Oracle and MarkLogic in the same transaction) and a multitude of enterprise capabilities from government grade security to point-in-time restore. Read more: http://developer.marklogic.com/pubs/architecture/inside-marklogic-server-r7.pdf http://developer.marklogic.com/pubs/architecture/inside-mark...

realmandan 13y ago
Totally, not a slam at all. As long as you don't think a company name and "perform below expectations" sitting next to each other in a NYT long-form piece is a BAD thing...
- arosenbaum 13y ago
  I'll repeat what our CEO said in the NYTimes article "He said MarkLogic is performing up to standard, but “the network and the storage systems are not properly sized and not properly run.” Put any database on top of the resources that were provided and I think performance "below expectations" would have been likely.

acdha 13y ago

MarkLogic is an XML document store, with the usual shift in access patterns which that entails. I could easily see much of the problem simply being an impedance mismatch with devs & other systems expecting SQL-like behavior. I haven't worked with it directly but at several places it has a reputation for high memory requirements and requiring you to scale by getting single massive servers rather than clustering – I'm not sure if that's still true or if it's an issue for healthcare.gov but it could definitely make it harder to scale in a hurry.

arosenbaum 13y ago
MarkLogic is an XML document store like Oracle is a CSV file store. We store compressed trees. We require memory - like all databases - because we actually have rich indexes. Why? Scaling by requiring massive servers is Oracle's story, not MarkLogic. We scale out on commodity servers in a cluster just fine. However, we do not magically create more capacity when adding additional VM's on top of over-subscribed network and storage systems. We don't require proprietary shared storage systems (local, NFS, HDFS - just give us bandwidth) or proprietary networks (TCPIP please, 10G preferred, don't need FC). Cheapest way to scale is a cluster of decent sized machines with a decent amount of IO capacity each. {Q: What do you call a database without indexes, transactions or security? A: A file system!}
- hga 13y ago
  Ouch. As in, per this NYT article: http://www.nytimes.com/2013/11/23/us/politics/tension-and-woes-before-health-website-crash.html?hpw&rref= http://www.nytimes.com/2013/11/23/us/politics/tension-and-wo... the contractors weren't familiar with its paradigm. They got started so late (not until February in earnest from what we've heard), it sounds like they just didn't have the time to learn it ... in a project (micro-)managed by political and bureaucratic types in the government. Not learning it well enough would likely go along with not provisioning the necessary resources. Usual lesson of "don't try to do too many new things if you're on a tight time and/or money budget". Unfortunately, given that this is almost entirely a political exercise, I don't see how you're going to avoid getting some scapegoating.