4 ms·
Ask HN: If we Open Source our platform, would you use it?
- menegattig 14y agoHello All, We are strongly considering opening the source of our DMP (Data Management Platform), if it proves to be interesting to other developers / companies. Please, feel free to send questions, comments and critics... We will try to reply them all. Regards, Gabriel.
- moneypenny 14y agoLooks great. I was looking into creating something similar in terms of processing, some years ago, but never took it very far. Your work sounds very interesting and I can think of some interesting uses - I have access to some fascinating data sets.
- menegattig 14y agoReally great to know, thanks! Do you think you would also be able to help us maintaining and enhancing it? Could you share more ideas regarding the usage of the platform?
- mvallebr 14y agoPlease, could you share the uses you thoughts for it? This is what we are most interested in. We did the platform for ourselves, and it seems powerful now, we would like to know what could interest others...
- svndznt 14y agoInteresting, We have developed kind of the same concept but agnostic to the underlying runtime. by agnostic, I mean that we can take advantage of Hadoop (via cascading) for our full indexation processes or Storm for the incremental indexation processes. Another difference is the way we handle the data, we chose to use tuples (as in cascading and storm) to transfer the data between the different components.
- mvallebr 14y agoThis is interesting, indeed. As your platform is agnostic to the runtime, wouldn't it be easy to use ours instead of hadoop or storm? Or you don't see any benefit? About the tuples, you still could use them, if I understood it well...
- JackParsons 14y agoHow do you do job scheduling?
- menegattig 14y agoYes, job scheduling and queue... Could you please share more details regarding what are you looking for?
- mvallebr 14y agoScheduling is done by any external tool like cron, a java app running quartz or control-m. However, the processor job knows the last time it was run and know which data it should process to update the time. You might also choose to run the job overriding the last time it ran, to reprocess data it you want.