3 ms·

[Obligatory: Someone is downvoting you, but it isn't me. I upvoted you] Yeah, I get that. The analytics space is very complex and companies, even ones with goo

[Obligatory: Someone is downvoting you, but it isn't me. I upvoted you] Yeah, I get that. The analytics space is very complex and companies, even ones with good engineering teams, don't have the internal knowledge or resources to typically put all this together. In addition to working in this space, my copmany helps companies set up their analytics stack. We typically set them up with one cloud-based data integration tool (the one with the most # of integrations they need at the best price), dbt, and one BI tool (usually Looker or Periscope, in that order). All in, that takes us a few weeks to get them set up and going. I applaud your effort. I just struggle to understand why you accept punting on transformations (and using dbt (amazing library, by the way - great choice)), but then try to tackle something like integrations or BI tools. The complexity of both of those is massive and there are great open source efforts already out there. I'm eager to see where this goes.

sytse 8y ago
"but then try to tackle something like integrations or BI tools. The complexity of both of those is massive and there are great open source efforts already out there." I would love to hear your suggestion for a great open source BI tool. We tried Superset and Metabase but both didn't came close to what we could do with Looker. That is why we're giving Meltano Analyze a shot. BTW Do you want to do a livestreamed video call to discuss further in the 30 next minutes? You have a lot of knowledge. If so please email me and comment here. Update: He did email and livestream will happen on https://www.youtube.com/watch?v=F8tEDq3K_pE https://www.youtube.com/watch?v=F8tEDq3K_pE
- slap_shot 8y ago
  Sure - just shot you an email at website@yourhandle.com
  - jakecodes 8y ago
    What a great interview. @slap_shot, you had great questions and you are so well spoken. Really appreciate the feedback. We're all taking notes here. Hope you will keep an eye on our issue tracker for Meltano and give us your feedback as things come up.
    - mendelk 8y ago
      I have no horse in this race, but this is so cool that one minute you're exchanging comments on HN and the next you're livestreaming a conversation on the topic! What a world :)
      - sytse 8y ago
        Glad you like it!
    - slap_shot 8y ago
      My pleasure! Will watch the repo closely and contribute if I can. Very curious to see where this goes. Also, I had a coffee at 5PM with someone, which is way too late to be drinking coffee, and it is evident in how quickly I'm talking >.<
      - sytse 8y ago
        I liked the fast pace! But normally I do watch YouTube on 150% :) thanks for the chat
  - drewbanin 8y ago
    One of the maintainers of dbt here -- this was an amazing conversation and I'm so happy I caught it. Thanks both for sharing :)
    - sytse 8y ago
      The Meltano team loves dbt. Thanks for making and maintaining it. Maybe the best open source project in the data space.
  - mycelium 8y ago
    Thanks to both of you for your time doing that discussion! @slap_shot and anyone else — I'm curious if you have thoughts on, or even have heard of the Ballerina language? It's a programming language for doing data integration work, built by the ESB/integration consultancy WSO2. It seems to have a lot of eng resources sunk into it but surprisingly little fanfare. The CEO's interview with the Software Engineering Daily podcast was great: https://softwareengineeringdaily.com/2018/07/12/ballerina-language-with-tyler-jewell/ https://softwareengineeringdaily.com/2018/07/12/ballerina-la... The language site tends toward buzzword-salad, but clearly has had a lot of love and thought put into it: https://ballerina.io/philosophy/ https://ballerina.io/philosophy/
    - veritas3241 8y ago
      This look very interesting! I haven't heard of it before, but you've definitely piqued my interest. Thank you for the links.
- MarHoff 8y ago
  Non english speaker here you mentioned an OSS solution called "Inbulk" or something like that during the conversation. Could you spell it I'm pretty interested in finding out more about that project but google return a lot of unrelated result because of the name I guess...
  - slap_shot 8y ago
    https://github.com/embulk/embulk https://github.com/embulk/embulk
    - kakoni 8y ago
      For batch-type of workloads embulk has been really excellent tool for my company (for all extract and load steps. We do most of the transformations in db/warehouse)

sytse 8y ago

"but then try to tackle something like integrations or BI tools. The complexity of both of those is massive and there are great open source efforts already out there." I would love to hear your suggestion for a great open source BI tool. We tried Superset and Metabase but both didn't came close to what we could do with Looker. That is why we're giving Meltano Analyze a shot. BTW Do you want to do a livestreamed video call to discuss further in the 30 next minutes? You have a lot of knowledge. If so please email me and comment here. Update: He did email and livestream will happen on https://www.youtube.com/watch?v=F8tEDq3K_pE https://www.youtube.com/watch?v=F8tEDq3K_pE

slap_shot 8y ago
Sure - just shot you an email at website@yourhandle.com
- jakecodes 8y ago
  What a great interview. @slap_shot, you had great questions and you are so well spoken. Really appreciate the feedback. We're all taking notes here. Hope you will keep an eye on our issue tracker for Meltano and give us your feedback as things come up.
  - mendelk 8y ago
    I have no horse in this race, but this is so cool that one minute you're exchanging comments on HN and the next you're livestreaming a conversation on the topic! What a world :)
    - sytse 8y ago
      Glad you like it!
  - slap_shot 8y ago
    My pleasure! Will watch the repo closely and contribute if I can. Very curious to see where this goes. Also, I had a coffee at 5PM with someone, which is way too late to be drinking coffee, and it is evident in how quickly I'm talking >.<
    - sytse 8y ago
      I liked the fast pace! But normally I do watch YouTube on 150% :) thanks for the chat
- drewbanin 8y ago
  One of the maintainers of dbt here -- this was an amazing conversation and I'm so happy I caught it. Thanks both for sharing :)
  - sytse 8y ago
    The Meltano team loves dbt. Thanks for making and maintaining it. Maybe the best open source project in the data space.
- mycelium 8y ago
  Thanks to both of you for your time doing that discussion! @slap_shot and anyone else — I'm curious if you have thoughts on, or even have heard of the Ballerina language? It's a programming language for doing data integration work, built by the ESB/integration consultancy WSO2. It seems to have a lot of eng resources sunk into it but surprisingly little fanfare. The CEO's interview with the Software Engineering Daily podcast was great: https://softwareengineeringdaily.com/2018/07/12/ballerina-language-with-tyler-jewell/ https://softwareengineeringdaily.com/2018/07/12/ballerina-la... The language site tends toward buzzword-salad, but clearly has had a lot of love and thought put into it: https://ballerina.io/philosophy/ https://ballerina.io/philosophy/
  - veritas3241 8y ago
    This look very interesting! I haven't heard of it before, but you've definitely piqued my interest. Thank you for the links.
MarHoff 8y ago
Non english speaker here you mentioned an OSS solution called "Inbulk" or something like that during the conversation. Could you spell it I'm pretty interested in finding out more about that project but google return a lot of unrelated result because of the name I guess...
- slap_shot 8y ago
  https://github.com/embulk/embulk https://github.com/embulk/embulk
  - kakoni 8y ago
    For batch-type of workloads embulk has been really excellent tool for my company (for all extract and load steps. We do most of the transformations in db/warehouse)