59 ms·
InfiniDB goes out of business
- samcrawford 12y agoSad to see. We were one of their early paying customers in 2009/2010 (we got a great deal). Performance was fantastic over very large datasets, but the bugs, storage requirements (very expensive SANs) and query limitations became big problems for us. We moved away after a year. Just recently one of our team has been looking at them again (following a strong benchmark being posted at mysqlperformanceblog). So I went and checked out the forums and saw it was pretty much in a similar state to when I last used it four years ago. Sad to see they couldn't make it work. The team was always really friendly and quick to help with issues - good luck in the future.
- imaginenore 12y agoThere are so many scalable (and often free) databases out there. What do you that can't be done in Cassandra or Mongo or Vertica?
- trhway 12y agois where a good way of doing MPP SQL in Cassandra or Mongo? and Vertica is really far from being free, isn't it?
- imaginenore 12y agoPostgres-XL can do that. There's also Shard-Query and (not free) Amazon Redshift. Or you could just shard your regular community Postgre or MySQL.
- bsg75 12y agoSee also https://github.com/citusdata/cstore_fdw https://github.com/citusdata/cstore_fdw
- bsg75 12y agoVertica has a community tier, free up to 1TB of source data, and a 3 node limit. SQL is not part of the Cassandra or Mongo feature sets. They have certain analytic possibilities, but not if you want to use SQL (window functions for example), or most of the BI client tools associated with data warehousing.
- electrum 12y agoPresto is a fully distributed (MPP) SQL query engine that supports Cassandra: http://prestodb.io/docs/current/connector/cassandra.html http://prestodb.io/docs/current/connector/cassandra.html You can even join together data from Hive, Cassandra, MySQL, PostgreSQL, Kafka, etc., all in one query. We don't have a connector for Mongo yet, but contributions are welcome! The Cassandra connector was actually an external contribution. We don't use it at Facebook.
- samcrawford 12y agoOf course, you're right, but we didn't know that at the time. It turns out there's nothing we were doing that really warranted using a data warehouse (for that is what InfiniDB was really suited for). We were collecting a lot of numerical, time-series data, bulk loading it and then performing summarised queries periodically. Mongo, Cassandra, etc are not good fits for this. Vertica was very expensive. In the end we went with a sharded and partitioned MySQL setup (partitioning really is great if you use it well). It's worked very well.
- imaginenore 12y agoYeah, that's exactly what I suggested in the other comment. If you can get away with sharding, it's the easiest solution, and the developers/DBAs are relatively easy to find. Glad it worked out for you guys.
- jwegan 12y agoFYI, for people using less than 1 TB of data, Vertica does have a free community edition. I used Vertica at my last job and it was blazingly fast compared to Hive (like 5x to 10x faster).
- michaelcampbell 12y agoIs that 1TB compressed or "raw"?
- zenogais 12y agoThis actually sounds like a great fit for Cassandra and Hadoop. You could use Cassandra's built in TimeUUID as the primary key and have your data pre-sorted on disk in time-series. This would make big queries, using something like Hadoop, very efficient.
- jhh 12y agoThat's not necessary though, if it works fast enough on a relational database in my opinion. I think this article has some valuable insight on the topic: http://www.chrisstucchio.com/blog/2013/hadoop_hatred.html http://www.chrisstucchio.com/blog/2013/hadoop_hatred.html
- swartkrans 12y agoHonest question, is MongoDB actually scalable? I keep reading about how it is not for serious business[1][2][3][4]. Who is using Mongo at scale and what sort of loads are they seeing and are the level of problems on par with other solutions? As a developer I like Mongo's API, but I would worry about using it given what has been written about it. [1] http://aphyr.com/posts/284-call-me-maybe-mongodb http://aphyr.com/posts/284-call-me-maybe-mongodb [2] http://svs.io/post/31724990463/why-i-migrated-away-from-mongodb http://svs.io/post/31724990463/why-i-migrated-away-from-mong... [3] http://use-the-index-luke.com/blog/2013-10/mysql-is-to-sql-like-mongodb-to-nosql http://use-the-index-luke.com/blog/2013-10/mysql-is-to-sql-l... [4] https://news.ycombinator.com/item?id=8353228 https://news.ycombinator.com/item?id=8353228
- x0x0 12y agoNo, and even modest write loads will cause immense pain -- there was a single lock per db when I last used them. Plus lots of the standard tricks for getting performance out of dbs don't work for them. I'd recommend staying away unless your performance needs are very moderate and you must have unstructured tables.
- mcot2 12y agoCheck out tokumx, it fixes a lot of the problems with mongo.
- cheald 12y agoSeconded; we've had a very positive experience with it so far. We're running around 1.5TB in it (and growing) and it's working pretty decently. Our biggest problems with it have been CPU saturation due to compression (solvable with sharding) and oplog size (due to supporting ACID; supposedly much better in the upcoming release), but both of those are surmountable. In exchange we get massively better disk usage characteristics, no global locks, ACID compliance, transactions, and generally better performance. It's not perfect, but it solved a lot of our problems.
- rogerbinns 12y ago
- rubinelli 12y agoMy bet is what really took the wind out of InfiniDB's sails was Amazon's Redshift. It's new, shiny, ridiculously easy to use, has a strong pedigree and a fairly low initial capital outlay.
- mhoglan 12y agoNot really. Redshift is nice don't get me wrong, but we did not run into many people that were choosing Redshift over InfiniDB. The major players out there still run everything on premise and behind their firewall. btw, InfiniDB was originally going to be the backend to Redshift. Lets just say the previous executive team 'screwed that up'
- karavelov 12y agoIt looks most of the successful MPP analytic databases are based on PostgreSQL (e.g. Greenplum, Redshift). It's sad to see that InfiniDB could not make MySQL work for them reliably.
- samcrawford 12y agoI don't think it's fair to characterise this as a MySQL vs Postgres situation. Having used InfiniDB quite a bit I am familiar with how it related to MySQL. It basically only used MySQL as a frontend (meaning as an application you could use the mysql libs and query protocol); everything beyond that was InfiniDB's own stuff.
- karavelov 12y agoYes, you are right. This is true also for the "PostgreSQL" databases I mentioned - they use only the frontend of Postgres.
- _pmf_ 12y ago> It's sad to see that InfiniDB could not make MySQL work for them reliably. Integrating with MySQL at this level would probably only make sense if the team was already familiar with the internals; otherwise, PostgreSQL's code base would be a much cleaner choice.
- mhoglan 12y agoThe team knew MySQL very well internally, it is not so much that MySQL did not work out, it was very nice for people who needed to grow out MySQL to jump over to InfiniDB. MySQL going from independent to Sun to Oracle changed a lot of how friendly it is to do these kind of interactions. I imagine the PostgreSQL players in MPP space such as CitusDB have done very similar things that we did with MySQL. And it is not that InfiniDB could not move away from MySQL, but that is a lateral sideways move, and has to be funded for no advancement in benefit.
- T3RMINATED 12y agofinally infinidb was terrible snyways... waste of space on the internet... die.
- scottkrager 12y agoDang....they just raised $7.5m in February? http://venturebeat.com/2014/02/10/calpont-renames-itself-infinidb-takes-7-5m-to-suit-up-for-database-wars/ http://venturebeat.com/2014/02/10/calpont-renames-itself-inf... Curious if they are returning money to VC's or just really burned through that much in 7 months...
- psaintla 12y agoI once worked at a startup that burned through 3x that in about six months. 7.5m in six months is entirely possible.
- rdtsc 12y agoOn what? Did they hire all their friends and payed them huge salaries and signing bonuses. Fancy hardware?
- Arcanum-XIII 12y agoFor actually being in a company like that it's mostly that and bad overall management - at a time we did have a ratio of two (useless...) PM for on dev. Dev underpaid, PM overpaid (as expert freelance, of course, or with bonus), some huge expense made for stuff that make no sense (branding), negotiating badly some contract so that they cost the company more than they provide... Of course after a while this stop since all these mismanagement corrupt the products too, and pretty fast you're left with a huge black hole for money that doesn't produce anything meaningful.
- notastartup 12y agoMy god building what? 7.5 million dollars in 6 month. I hope it's was worth it.
- meritt 12y agoBuilding retirement packages probably
- 12y ago
- mp99e99 12y agoSad, wish they made it.
- bradhe 12y agoInfiniDB is/was a great idea. The unfortunate bit was just what a Rube Goldberg machine of a data store it was. I spent a good few days just getting everything provisioned from our automation.
- NamTaf 12y agoI've been playing with Infobright community edition but also evaluated InfiniDB. I found InfiniDB was not compressing my data nearly at all whereas Infobright was utterly jamming it down - factors of over 300x for even small datasets of ~2m rows. I don't know what the differences are to produce that, but when it comes to storing as much crap as I was looking at, I was willing to design around the limitations that Infobright CE has (ie: no insert/update queries) rathre than deal with the massive extra disk cost. I have currently got 223m rows sitting in Infobright and it's taking about 38MB. I really hope that the OSS project takes off and that InfiniDB sees some better compression implemented, similar to Infobright. The extra features that InifiniDB has over Infobright CE (not only insert/update but also a multi-threaded infile loader, for example) would convert me if only the compression were better. I'm all new to this though so if there's some good reason why they differ so greatly I'm all ears and would love to know. Maybe I screwed something up in the configuration? I'm not sure. Either way, it's sad to see them go. Columnstore databases fill a really useful application that I can only see growing over time as more and more operational data is collected by industry.
- eis 12y ago223 million rows in 38MB would mean less than a fifth of a byte per row. Is your data really that extremely repetetive?
- NamTaf 12y agoPhenomenal. Some is binary (sparse, too - 99% false sort of thing), one other is an integer from -1 to 8, some are floats (decimal degrees for GPS) and one is a datetime. 38MB is what is reported when I use the following query, which I found somewhere: SELECT table_schema, sum (data_length + index_length) / 1024 / 1024 'Data Base Size in MB', TABLE_COMMENT FROM information_schema.TABLES GROUP BY table_schema You're right though - the cost per row sounds really low, I might try to find it via other means tomorrow and report back.
- NamTaf 12y agoYup, just confirmed that the raw folder size of the database is 39.8MB and reports 39MB (not 38) in my query. It is also 222.78M rows. Note that you need to remove the space between the sum and the bracket if you want to run that query. I can only assume that IB does some sort of differential compression on the data to get such a small filesize on that data and that it's an artifact of the machine data I'm using. But that's what I think the next big wave of data will be - stuff generated by data loggers on machines and equipment and analysed to scrape out incremental improvements on efficiency, reliability, etc. from previously relatively low-tech industrial corporations.
- mhoglan 12y agoClear up a few things here for all the speculation. I was an architect at InfiniDB that came on in Nov 2013 to build out the Enterprise Manager which was coming to alleviate many of the provisioning, management and monitoring woes that customers were experiencing and help modernize those aspects. The first beta offering of which was early July, unfortunately the ship had sailed so to speak. So I know first hand how and why things did not work here. As with all things, take your lessons learned, move on. Success is not the path of learning, failure is (see survivorship bias) Some notes: Labeling InfiniDB as MySQL+ is a gross underestimation of what it does. MySQL is used as the front end query parser, and that is about it. Everything else behind it was custom written, and that is where the power is. As with all DB technologies, your use case is the primary thing that determines your mileage. Comparing InfiniDB to MongoDB is one of the first signs to me that you don't fully comprehend the differences between database architectures. For the use cases that InfiniDB was made for, we routinely were faster performing on a smaller footprint. Using InfiniDB as a document store can be done, but that is not what it was made for. What people call 'big datasets' is relative. Some think 500GB is alot, some think 5TB alot. Coming from telecommunications monitoring background, I will appreciate your dataset when we are talking TBs a day of churn per monitoring point with hundreds and thousands of monitoring points. The size of dataset you are working with, along with your use case for analysis is the two most important things in determining the technology stack. InfiniDB operated at these higher end scales very efficiently. There is a reason why Impala was a primary comparison, and we would usually operate on fraction of the hardware they needed. Best technology does not always win. See InfiniDB. Decisions made by previous executive teams years in the past can set a course that cannot be corrected sometimes (not efficiently or without alot of money) Patents are worth their weight in gold. Being open source is great for the community, but is a challenge to a business to build consistent revenue. There were many big projects running InfiniDB with the open source version, but not contributing to revenue. Even if they did sign up with support, you need custom feature development and other big ticket items to make impact. Or you have to build a large customer base paying for support, and that takes time. With the multiple iterations of adapting the technology to different architectures over the years, that was hard to retain those customers consistently. Also many customers will pay for support for their rollout or initial deployment, but when the project is done, they feel they are adequate enough to live with open source only. Just because a company raised $X in a month, does not mean all that money is slated for going forward from that month. On top of that, payroll is not cheap, and you would be surprised how quickly you can burn through money keeping the lights on. For those of you who think people at startups are working for pennies on the dollars, I would advise you it is not the case. And if you are one of those people, I wish you the best, and odds are there are other reasons why you are doing so. Why would good engineers work at discount? Equity? There is not enough of the pie to go around to make that sustainable. Most startups pay competitive market salaries. InfiniDB was at a junction where it was time to go for it, or go home, and that is exactly what happened in 2014. The marketplace for data solutions with Hadoop rising, other MPP vendors consolidating, and bigger players entering the field, made it very competitive, and the time to swing for the fence was now, versus treading water and hoping. Even with stars aligned and everything else, all you have done in a company is weight the opportunity of it succeeding, not guaranteed. I really enjoyed my time with InfiniDB and the team there. I really do feel its a missed opportunity with some decisions that could have been changed several years ago. Not securing patents and probably choosing MySQL as a frontend are some of those. Side note, core group of us at InfiniDB have landed at Tune, a company that has appreciated the technology of InfiniDB and what it can offer for their solutions. Look forward to this new opportunity and what we can provide to the ad and mobile analytics space.