3 ms·
Open Sourcing Venice – LinkedIn’s Derived Data Platform
- ZacAttackFTW 4y agoAwesome! Excited to see where this goes in open source!
- adamxche 4y agoGreat to see it's open-sourced. Super excited!
- throwawaygog6 4y agoOpensourcing a component means it no longer provides any competitive. It just helps companies increase their open source credentials or decreases their cost of development over the next few years!
- felixgv 4y agoI think companies looking to decrease their development costs are going to be disappointed with open sourcing... That does not happen, by and large. Increasing creds is an aspect. Increasing the quality of the software is another one, IMHO.
- nisargthakkar 4y agoThe thing with Venice is that it is a datastore. It does not have any business-specific logic. By itself, the only competitive advantage Venice provides is that it is a datastore that has first-class support for ingesting batch data. We feel this is not something that needs to be kept as a closely-guarded secret. We have already discussed the architecture of Venice in conferences and articles. As per decreasing the cost of development over the next few years, it is actually quite the opposite. In the near term, the cost of development increases as the system not-only needs to work in your organization, but also work externally - with external technologies. You are right that reducing the cost of development is a goal, but there are many things that need to be done initially to facilitate that and make the project self-sustaining.
- nisargthakkar 4y agoVenice is a distributed Key-Value store for serving derived data. Since derived data is batch-computed data, the requirements of the system are to support high-throughput writes. We don’t know of any other system (at least open-sourced) that solves this problem at our scale. In the conference where we announced Venice, we learnt about some hacks that other organizations have been employing to address this problem in traditional data stores.
- ensignavenger 4y agoThis is great! Thank you for your many contributions to Open Source. Any plans to open Galene? I am actually working on a side project several years on now that may benefit from Venice at some point :) Thanks again!