8 ms·

NoSQL, Erlang, and Technology's Effect On Economics and Culture

56 points by jenlankford 13y ago

ryanobjc 13y ago
Great blog article! Good discussion, but there was one element I would like to see discussed more about these 'store your data as a service' is the hidden vendor lock-in! I can understand that when you are pushing for a solution, you make the decision with what's at hand. But this can lock you in to long term decisions, for example people who decided to use Google's app engine. The pricing switcharoo is doubly painful because there is literally no option except rewriting everything! For this reason, I strongly believe that iaas/paas needs to be fully open source. Don't like your provider? Outgrew a provider? Company perished? Fine, run your own. We have somehow switched from open source and the freedom that entails, to open API, and a complete lack of freedom.
- jenlankford 13y ago
  Hey Ryan, thanks for the comment. So, Orchestrate actually has an on-demand export tool so there is no lock-in. We backup your data into S3 and send you all your data in JSON format. We take zero lock-in very seriously :) http://orchestrate.io/blog/2014/03/12/new-feature-data-export-for-zero-lock-in/ http://orchestrate.io/blog/2014/03/12/new-feature-data-expor...
  - alexnewman 13y ago
    But I can't actually run orchestrate.io in my own datacenter? So I don't have data lockin (in the same way that i can select * out of oracle), but I am locked-in to your service?
  - ryanobjc 13y ago
    Thanks for the reply! I feel bad even pointing this stuff out, because I know that startups have to do what they have to do to make the bread as it were. I am very glad there is a data export tool, I'm sure that will alleviate some people's concern. But the issue is that now someone has a bunch of code written to an API they have no implementation for. That is the nature of the lock in! Vendors discovered in the 90s that open APIs were very amenable to lock in - for example CORBA, SQL and more. It was really hard to port from CORBA implementation to another one, since non-functional requirements dominated at that point, even if at the API/code level it was compatible. Good luck with your business!
    - h1karu 13y ago
      This is why I'm happy with Couchdb.. I can always replicate the dataset into my own BigCouch cluster if Cloudant one day becomes prohibitively expensive. The thing I'd miss most if I moved from couchdb to orchestrate is the ability to query the key/value store with map/reduce functions. I have to admit orchestrate's graph stuff is cool. For me the only way orchestrate would be a win is if it turns out to be an order of magnitude cheaper than a couchdb based solution for the use cases I care about. The pricing model is intriguing I must admit.. more so for write-heavy apps. I love the concept, good luck orchestrate
    - dizzyd-oio 13y ago
      Disclosure - I’m one of the engineering team at OiO. Now, that aside… :) With the definition of “lock-in” that you’re using, it’s very difficult to find any service that doesn’t cause the user to be “locked in”. There is always a cost to migration from one service to another. That cost might be code changes to support a new API or it might be capital expenditures to spin up a cluster of a bunch of different databases to provide the same functionality, or it might be the cost of hiring a specialized team of ops engineers to run said cluster..or all of the above. :) So, yes, there is a significant tradeoff if you choose to use OiO - you are trading off the cost of building and maintaining a large system of databases for the convenience and speed of development. Even if we open sourced all the code to run the IaaS portion of our system, you’d still have to find the hardware and manpower to run it...and keep it running over the long term. Once you’ve built up your own expertise, you would pay AGAIN if you chose to migrate to something else. There is no free lunch. Every action entails choice and consequence, but Orchestrate is committed to ensuring data portability and freedom to our users as much as possible, with the hope that they will love it enough to stay and use for a long time to come.
    - jenlankford 13y ago
      This topic runs deep, and we could explore for a while. But I thought this blog post might also be helpful in understanding Orchestrate's position on lock-in: http://blog.ilovacha.com/2014/03/17/using-orchestrate-io/ http://blog.ilovacha.com/2014/03/17/using-orchestrate-io/. A user asked us the same question, and was happy enough with the response from our CEO that he wrote a post about it. Hopefully it helps clear some lingering questions. Thanks all for the feedback and keep the questions coming.
jboynyc 13y ago
Perhaps surprisingly in an interview that's mostly about technical issues, Tristan Sloughter makes some very smart points about labor: global labor issues: With high unemployment, and high under-employment, it is said you should feel lucky simply to have a job and to not ask for more. effect of the two-tier labor market for engineers: I've spoken to many developers who think golden jobs exist all over easily to grab by anyone who will work hard enough. This is not the case at all and the programmers in the companies with slides and free lunches must learn this and unite with the real world or risk finding themselves on the wrong side of labor battle in the future. creation of a 'reserve army': I think the push for everyone to learn to code shows what the future holds. More and more companies in all fields are having uses for programmers, and the easier it is to replace them with cheaper labor the better for their profits. anti-labor bias in U.S. culture: More likely the mass of developers are on the "Right" but not because of libertarian ideology but simply being in the United States. social context of technology use: Simply developing a technology doesn't change the world, but the social battles around it, either directly or indirectly, do. social progress requires socialism: There is enough food and housing, it simply isn't being properly distributed, and this shows that technology itself can not provide a solution.
- mercurial 13y ago
  Interesting interview. However, the notion of a "planned economy", even for a left-wing person such as yours truly, echoes of dreadful communist five-year plans. I remember, in Accelerando, the idea of algorithmically assigning resources, but it sounds very hard to do in practice, even though our current economic model is both unfair and headed for disaster in the not-so-long term.
  - kungfooguru 13y ago
    Our current economic model is also very planned, just not by and for the right people and reasons. The five-year plans were to bring a backward feudal country into the 20th century. That isn't to legitimize the abuses any more than it is legitimize the abuses the US and other countries (slavery and other labor abuses) committed to advance themselves to the level they are at.

56 points by jenlankford 13y ago

Great blog article! Good discussion, but there was one element I would like to see discussed more about these 'store your data as a service' is the hidden vendor lock-in! I can understand that when you are pushing for a solution, you make the decision with what's at hand. But this can lock you in to long term decisions, for example people who decided to use Google's app engine. The pricing switcharoo is doubly painful because there is literally no option except rewriting everything! For this reason, I strongly believe that iaas/paas needs to be fully open source. Don't like your provider? Outgrew a provider? Company perished? Fine, run your own. We have somehow switched from open source and the freedom that entails, to open API, and a complete lack of freedom.

jenlankford 13y ago
Hey Ryan, thanks for the comment. So, Orchestrate actually has an on-demand export tool so there is no lock-in. We backup your data into S3 and send you all your data in JSON format. We take zero lock-in very seriously :) http://orchestrate.io/blog/2014/03/12/new-feature-data-export-for-zero-lock-in/ http://orchestrate.io/blog/2014/03/12/new-feature-data-expor...
- alexnewman 13y ago
  But I can't actually run orchestrate.io in my own datacenter? So I don't have data lockin (in the same way that i can select * out of oracle), but I am locked-in to your service?
- ryanobjc 13y ago
  Thanks for the reply! I feel bad even pointing this stuff out, because I know that startups have to do what they have to do to make the bread as it were. I am very glad there is a data export tool, I'm sure that will alleviate some people's concern. But the issue is that now someone has a bunch of code written to an API they have no implementation for. That is the nature of the lock in! Vendors discovered in the 90s that open APIs were very amenable to lock in - for example CORBA, SQL and more. It was really hard to port from CORBA implementation to another one, since non-functional requirements dominated at that point, even if at the API/code level it was compatible. Good luck with your business!
  - h1karu 13y ago
    This is why I'm happy with Couchdb.. I can always replicate the dataset into my own BigCouch cluster if Cloudant one day becomes prohibitively expensive. The thing I'd miss most if I moved from couchdb to orchestrate is the ability to query the key/value store with map/reduce functions. I have to admit orchestrate's graph stuff is cool. For me the only way orchestrate would be a win is if it turns out to be an order of magnitude cheaper than a couchdb based solution for the use cases I care about. The pricing model is intriguing I must admit.. more so for write-heavy apps. I love the concept, good luck orchestrate
  - dizzyd-oio 13y ago
    Disclosure - I’m one of the engineering team at OiO. Now, that aside… :) With the definition of “lock-in” that you’re using, it’s very difficult to find any service that doesn’t cause the user to be “locked in”. There is always a cost to migration from one service to another. That cost might be code changes to support a new API or it might be capital expenditures to spin up a cluster of a bunch of different databases to provide the same functionality, or it might be the cost of hiring a specialized team of ops engineers to run said cluster..or all of the above. :) So, yes, there is a significant tradeoff if you choose to use OiO - you are trading off the cost of building and maintaining a large system of databases for the convenience and speed of development. Even if we open sourced all the code to run the IaaS portion of our system, you’d still have to find the hardware and manpower to run it...and keep it running over the long term. Once you’ve built up your own expertise, you would pay AGAIN if you chose to migrate to something else. There is no free lunch. Every action entails choice and consequence, but Orchestrate is committed to ensuring data portability and freedom to our users as much as possible, with the hope that they will love it enough to stay and use for a long time to come.
  - jenlankford 13y ago
    This topic runs deep, and we could explore for a while. But I thought this blog post might also be helpful in understanding Orchestrate's position on lock-in: http://blog.ilovacha.com/2014/03/17/using-orchestrate-io/ http://blog.ilovacha.com/2014/03/17/using-orchestrate-io/. A user asked us the same question, and was happy enough with the response from our CEO that he wrote a post about it. Hopefully it helps clear some lingering questions. Thanks all for the feedback and keep the questions coming.

jboynyc 13y ago

Perhaps surprisingly in an interview that's mostly about technical issues, Tristan Sloughter makes some very smart points about labor: global labor issues: With high unemployment, and high under-employment, it is said you should feel lucky simply to have a job and to not ask for more. effect of the two-tier labor market for engineers: I've spoken to many developers who think golden jobs exist all over easily to grab by anyone who will work hard enough. This is not the case at all and the programmers in the companies with slides and free lunches must learn this and unite with the real world or risk finding themselves on the wrong side of labor battle in the future. creation of a 'reserve army': I think the push for everyone to learn to code shows what the future holds. More and more companies in all fields are having uses for programmers, and the easier it is to replace them with cheaper labor the better for their profits. anti-labor bias in U.S. culture: More likely the mass of developers are on the "Right" but not because of libertarian ideology but simply being in the United States. social context of technology use: Simply developing a technology doesn't change the world, but the social battles around it, either directly or indirectly, do. social progress requires socialism: There is enough food and housing, it simply isn't being properly distributed, and this shows that technology itself can not provide a solution.

mercurial 13y ago
Interesting interview. However, the notion of a "planned economy", even for a left-wing person such as yours truly, echoes of dreadful communist five-year plans. I remember, in Accelerando, the idea of algorithmically assigning resources, but it sounds very hard to do in practice, even though our current economic model is both unfair and headed for disaster in the not-so-long term.
- kungfooguru 13y ago
  Our current economic model is also very planned, just not by and for the right people and reasons. The five-year plans were to bring a backward feudal country into the 20th century. That isn't to legitimize the abuses any more than it is legitimize the abuses the US and other countries (slavery and other labor abuses) committed to advance themselves to the level they are at.