7 ms·

Grouparoo: Open-source app to sync customer data with 3rd party tools

103 points by kine 6y ago

coob 6y ago
Wow, this is exactly what I started looking for at the start of the day. Unfortunately I can't use it yet as there's no Postgres SSL support - filed a GitHub issue here: https://github.com/grouparoo/grouparoo/issues/734 https://github.com/grouparoo/grouparoo/issues/734
- bleonard 6y ago
  Thanks! That should be an easy one. What was your goal when you started the day? :-)
  - coob 6y ago
    Syncing our customer user data to Zendesk without having to write our own syncing service.
    - bleonard 6y ago
      Sounds perfect for us. We have some questions on that issue. Let's make it happen!
- bleonard 6y ago
  Postgres connector enhanced! https://github.com/grouparoo/grouparoo/pull/735 https://github.com/grouparoo/grouparoo/pull/735 Let us know on the issue if it's working for you.
kine 6y ago
I worked with the Grouparoo guys years ago at TaskRabbit. They're a super impressive team. Excited to watch their progress with Grouparoo!
bleonard 6y ago
Thanks for checking us out! Co-founder here, happy to answer any questions. There is so much to do in this space, but we’re excited to be getting started. No engineer wakes up in the morning excited to sync data to Marketo, so we started there - `npm install` and so you can get back to building the core product. We make data self-serve for your non-technical colleagues and we handle all the exhausting integration stuff you don’t want to think about (API nuances, rate limiting, retrying, batching, etc).
- psadri 6y ago
  Does grouparoo need to deal with GDPR? If so, how is it handled?
  - bleonard 6y ago
    Good question! GDPR is about giving users control and visibility over their personal data and controlling personally identifiable information. Grouparoo is an open source app that runs in your own infrastructure (AWS, etc) and it does segmentation of users. The effect of that is that less info leaves your world and goes to third parties. For example, you used to send an address and lifetime value to Braze so that you could make a “high value Bay Area customers” group over there. Now you keep that in house. On top of that, whatever information is leaving now has a chokepoint, so you can stop sending a user to Braze (and everywhere else) if that is the requirement or return all the information about them easily if that is the ask.
- mythshaker 6y ago
  So, what's informed your making this open source?
  - bleonard 6y ago
    For us, open source is about who has control of the data and the integrations. I don't think the world needs another SaaS marketing tool. In the past, you needed a large, focused SaaS vendor to be able to store a million users and their properties/events. AWS and friends have caught up and now it's easy. Because of that, we can take that data into your own environment and use it to increase control, customization, privacy, and compliance. The cost is significantly better, too. Open source is a good way to do that because you know what you are running and can fit it to your needs. Engineers tend to like open source and we've seen interest in extending it. There are thousands of things to connect with, both inside your infra and outside with vendors, and open source makes that possible.
desk_minion 6y ago
Oh wow, this reminds of Hightouch (https://hightouch.io https://hightouch.io) and Rudder (https://rudderstack.com https://rudderstack.com). It's interesting that all of these are positioned around data warehouses, which is generally very messy to deal with.
- deleted 6y ago
  [deleted]
- bleonard 6y ago
  Thanks for the links. Overall, there's an interesting organizational dynamic that we've seen around data enablement. Marketing and other operational teams need it and it's often locked in the product space. It's usually not a priority for the eng team because they are focused on the core product, but the data is there. The important stuff (ETL copy of the product db) is usually not a huge mess. We are inspired by warehouse tools like Looker that made that accessible to more people, giving them autonomy to be successful. Grouparoo takes that one step further to add on top of the data and make it actionable in all the other places that people want it.
  - desk_minion 6y ago
    I see, yeah that makes sense. The integrations on your website kind of clarify your point too. You are moving up the stack, and providing value at that layer. As a general rule that formula works in most industries. I'm a big fan of looker, and I hope to see you guys grow!
  - mmaia 6y ago
    In my experience this scenario is very accurate. Add to it the growing usage of tools like Pipedrive, Pipefy, Airtable, that sometimes prod-eng team is not even aware. As a Rudder and Fivetran user, I can see a very complementary use case for Grouparoo. Where the first two are responsible for unifying events and external data in the DW and Grouparoo to sync user data to other tools. Two other tools that I saw in this space (not Open Source): https://www.calixa.io/ https://www.calixa.io/ https://windsor.io/ https://windsor.io/
shaanr 6y ago
I had a demo of Grouparoo a few weeks ago -- super cool platform that solves so many ETL / integration challenges.
shakedown1 6y ago
At a glance, this looks cool - im in the middle of designing an internal Customer Data Platform/Single Customer View to serve as a single data source for various internal & external marketing tools. Looks like this might be worth looking at. Does it have an API to access the user profile and group data ad-hoc? Can you stream data in? Can it trigger a destination sync when the underlying data changes? Can it do profile merging (visitor -> known customer stitching)? How can you do reporting/analytics?
- bleonard 6y ago
  Great questions, thanks! > Does it have an API to access the user profile and group data ad-hoc? We’ve seen a few approaches here. 1) Yes, there are APIs 2) The Postgres database this runs on is in your data center, so you can read it directly 3) You can write back to your own product database as a “destination” > Can you stream data in? We support events via an API. We’ll store the vents and allow your to create profile properties from them. I’m very interested in creating a Kafka or other message bus sort of integration too that brings in data and/or triggers recalculations. No one has needed that yet, though, so it’s just on our eventual list. > Can it trigger a destination sync when the underlying data changes? We have schedules, table queries, and events to know profile data has changed. When it changes, it then recalculates groups. Then properties or groups change that are being sent to a destination, it automatically exports there. “Hey Mailchimp, the user changed their first name and should now be tagged as VIP.” > Can it do profile merging (visitor -> known customer stitching)? We have the concept of anonymous id before login. When we realize two profiles are the same (usually after logging in from another device or something), the profiles are merged and everything recalculated. > How can you do reporting/analytics? This hasn’t been a focus so far outside of our ETL mechanics. You can see who has been imported and exported and with what and when and all of that. Things get more interesting around properties and their values, but we haven’t gotten there yet. We’ve seen some success at pointing tools like Metabase at the Grouparoo database.
  - shakedown1 6y ago
    Cool thanks, just installed it now to have a look. Any timeline for when S3 and Redshift will be available as a source & destination?
soumyadeb 6y ago
Congrats guys from the team @ RudderStack. Glad to see more open-source products in this broad customer data space. Is it fair to say this is more like the Segment personas's product? We see a bunch of use cases for personas (which we don't have in RudderStack) so can point to you guys. Congrats again on the launch.
- bleonard 6y ago
  Yeah, the group-building part and syncing those (for example to Static Lists in Marketo) is like Personas. It was always funny to me that you had to pay quite a bit for Personas for Segment to actually, you know, segment. The gap we saw was around understanding the user and segmenting in a way that could be shared across multiple services and the product. And doing so in a way where you controlled the data and the total cost was managed. It's nice to see there are others open in this space. The trends are certainly in the open direction to provide a lot of value and control in a way that's good for everyone.
jslakro 6y ago
What if I need to store all user data (not limited to profile information) in google sheets and I provide only an interface for an specific usage. I cannot see how good this scenario fits. Use cases are quite simple
- bleonard 6y ago
  Everything in Grouparoo is somehow tied to a user, so if you have a list of "locations" or something, that's not a good fit at this point. We basically need a "foreign key" to a property of the user. If you do have that, we have support for google sheets[1]. You share a sheet with a service account and that allows it to be a source in Grouparoo. From there, you can make groups and send to destinations. We'd be curious about your use cases. Feel free to make an issue[2] with what you are hoping for and we can discuss there. [1] https://www.grouparoo.com/blog/google-sheets-source https://www.grouparoo.com/blog/google-sheets-source [2] https://github.com/grouparoo/grouparoo/issues/new?assignees=&labels=enhancement&template=feature_request.md https://github.com/grouparoo/grouparoo/issues/new?assignees=...
edmundito 6y ago
I've worked with the founders in the past and they're legit! I'm excited to see where they'll go with Grouparoo.
borisjabes 6y ago
Very cool to see. A lot of folks (Notion, Figma, Loom) use our service (https://getcensus.com https://getcensus.com) for this purpose if folks are curious to check out a SaaS version of the concept. We support all the major data warehouses (incl. straight Postgres), connect to a bunch of different applications, and don't store any of your customer data! Bonus: we recently added native support for dbt :-)
bartekr 6y ago
Can’t wait to try this out sometime soon. I’ve worked with all the founders and they’re awesome.
nloui 6y ago
Looking forward to tracking this project. I totally get it.
recur 6y ago
This was such a massive problem at the large company where I used to work. Excited to see the progress!
ramziq 6y ago
Grouparoo made it dead simple to connect a mysql database and generate information. Easy to Install and Integrations make it easy to send the right data to each tool you use. No code is necessary to change what data gets sent.

103 points by kine 6y ago

coob 6y ago

Wow, this is exactly what I started looking for at the start of the day. Unfortunately I can't use it yet as there's no Postgres SSL support - filed a GitHub issue here: https://github.com/grouparoo/grouparoo/issues/734 https://github.com/grouparoo/grouparoo/issues/734

bleonard 6y ago
Thanks! That should be an easy one. What was your goal when you started the day? :-)
- coob 6y ago
  Syncing our customer user data to Zendesk without having to write our own syncing service.
  - bleonard 6y ago
    Sounds perfect for us. We have some questions on that issue. Let's make it happen!
bleonard 6y ago
Postgres connector enhanced! https://github.com/grouparoo/grouparoo/pull/735 https://github.com/grouparoo/grouparoo/pull/735 Let us know on the issue if it's working for you.

kine 6y ago

I worked with the Grouparoo guys years ago at TaskRabbit. They're a super impressive team. Excited to watch their progress with Grouparoo!

bleonard 6y ago

Thanks for checking us out! Co-founder here, happy to answer any questions. There is so much to do in this space, but we’re excited to be getting started. No engineer wakes up in the morning excited to sync data to Marketo, so we started there - `npm install` and so you can get back to building the core product. We make data self-serve for your non-technical colleagues and we handle all the exhausting integration stuff you don’t want to think about (API nuances, rate limiting, retrying, batching, etc).

psadri 6y ago
Does grouparoo need to deal with GDPR? If so, how is it handled?
- bleonard 6y ago
  Good question! GDPR is about giving users control and visibility over their personal data and controlling personally identifiable information. Grouparoo is an open source app that runs in your own infrastructure (AWS, etc) and it does segmentation of users. The effect of that is that less info leaves your world and goes to third parties. For example, you used to send an address and lifetime value to Braze so that you could make a “high value Bay Area customers” group over there. Now you keep that in house. On top of that, whatever information is leaving now has a chokepoint, so you can stop sending a user to Braze (and everywhere else) if that is the requirement or return all the information about them easily if that is the ask.
mythshaker 6y ago
So, what's informed your making this open source?
- bleonard 6y ago
  For us, open source is about who has control of the data and the integrations. I don't think the world needs another SaaS marketing tool. In the past, you needed a large, focused SaaS vendor to be able to store a million users and their properties/events. AWS and friends have caught up and now it's easy. Because of that, we can take that data into your own environment and use it to increase control, customization, privacy, and compliance. The cost is significantly better, too. Open source is a good way to do that because you know what you are running and can fit it to your needs. Engineers tend to like open source and we've seen interest in extending it. There are thousands of things to connect with, both inside your infra and outside with vendors, and open source makes that possible.

desk_minion 6y ago

Oh wow, this reminds of Hightouch (https://hightouch.io https://hightouch.io) and Rudder (https://rudderstack.com https://rudderstack.com). It's interesting that all of these are positioned around data warehouses, which is generally very messy to deal with.

deleted 6y ago
[deleted]
bleonard 6y ago
Thanks for the links. Overall, there's an interesting organizational dynamic that we've seen around data enablement. Marketing and other operational teams need it and it's often locked in the product space. It's usually not a priority for the eng team because they are focused on the core product, but the data is there. The important stuff (ETL copy of the product db) is usually not a huge mess. We are inspired by warehouse tools like Looker that made that accessible to more people, giving them autonomy to be successful. Grouparoo takes that one step further to add on top of the data and make it actionable in all the other places that people want it.
- desk_minion 6y ago
  I see, yeah that makes sense. The integrations on your website kind of clarify your point too. You are moving up the stack, and providing value at that layer. As a general rule that formula works in most industries. I'm a big fan of looker, and I hope to see you guys grow!
- mmaia 6y ago
  In my experience this scenario is very accurate. Add to it the growing usage of tools like Pipedrive, Pipefy, Airtable, that sometimes prod-eng team is not even aware. As a Rudder and Fivetran user, I can see a very complementary use case for Grouparoo. Where the first two are responsible for unifying events and external data in the DW and Grouparoo to sync user data to other tools. Two other tools that I saw in this space (not Open Source): https://www.calixa.io/ https://www.calixa.io/ https://windsor.io/ https://windsor.io/

shaanr 6y ago

I had a demo of Grouparoo a few weeks ago -- super cool platform that solves so many ETL / integration challenges.

shakedown1 6y ago

At a glance, this looks cool - im in the middle of designing an internal Customer Data Platform/Single Customer View to serve as a single data source for various internal & external marketing tools. Looks like this might be worth looking at. Does it have an API to access the user profile and group data ad-hoc? Can you stream data in? Can it trigger a destination sync when the underlying data changes? Can it do profile merging (visitor -> known customer stitching)? How can you do reporting/analytics?

bleonard 6y ago
Great questions, thanks! > Does it have an API to access the user profile and group data ad-hoc? We’ve seen a few approaches here. 1) Yes, there are APIs 2) The Postgres database this runs on is in your data center, so you can read it directly 3) You can write back to your own product database as a “destination” > Can you stream data in? We support events via an API. We’ll store the vents and allow your to create profile properties from them. I’m very interested in creating a Kafka or other message bus sort of integration too that brings in data and/or triggers recalculations. No one has needed that yet, though, so it’s just on our eventual list. > Can it trigger a destination sync when the underlying data changes? We have schedules, table queries, and events to know profile data has changed. When it changes, it then recalculates groups. Then properties or groups change that are being sent to a destination, it automatically exports there. “Hey Mailchimp, the user changed their first name and should now be tagged as VIP.” > Can it do profile merging (visitor -> known customer stitching)? We have the concept of anonymous id before login. When we realize two profiles are the same (usually after logging in from another device or something), the profiles are merged and everything recalculated. > How can you do reporting/analytics? This hasn’t been a focus so far outside of our ETL mechanics. You can see who has been imported and exported and with what and when and all of that. Things get more interesting around properties and their values, but we haven’t gotten there yet. We’ve seen some success at pointing tools like Metabase at the Grouparoo database.
- shakedown1 6y ago
  Cool thanks, just installed it now to have a look. Any timeline for when S3 and Redshift will be available as a source & destination?

soumyadeb 6y ago

Congrats guys from the team @ RudderStack. Glad to see more open-source products in this broad customer data space. Is it fair to say this is more like the Segment personas's product? We see a bunch of use cases for personas (which we don't have in RudderStack) so can point to you guys. Congrats again on the launch.

bleonard 6y ago
Yeah, the group-building part and syncing those (for example to Static Lists in Marketo) is like Personas. It was always funny to me that you had to pay quite a bit for Personas for Segment to actually, you know, segment. The gap we saw was around understanding the user and segmenting in a way that could be shared across multiple services and the product. And doing so in a way where you controlled the data and the total cost was managed. It's nice to see there are others open in this space. The trends are certainly in the open direction to provide a lot of value and control in a way that's good for everyone.

jslakro 6y ago

What if I need to store all user data (not limited to profile information) in google sheets and I provide only an interface for an specific usage. I cannot see how good this scenario fits. Use cases are quite simple

bleonard 6y ago
Everything in Grouparoo is somehow tied to a user, so if you have a list of "locations" or something, that's not a good fit at this point. We basically need a "foreign key" to a property of the user. If you do have that, we have support for google sheets[1]. You share a sheet with a service account and that allows it to be a source in Grouparoo. From there, you can make groups and send to destinations. We'd be curious about your use cases. Feel free to make an issue[2] with what you are hoping for and we can discuss there. [1] https://www.grouparoo.com/blog/google-sheets-source https://www.grouparoo.com/blog/google-sheets-source [2] https://github.com/grouparoo/grouparoo/issues/new?assignees=&labels=enhancement&template=feature_request.md https://github.com/grouparoo/grouparoo/issues/new?assignees=...

edmundito 6y ago

I've worked with the founders in the past and they're legit! I'm excited to see where they'll go with Grouparoo.

borisjabes 6y ago

Very cool to see. A lot of folks (Notion, Figma, Loom) use our service (https://getcensus.com https://getcensus.com) for this purpose if folks are curious to check out a SaaS version of the concept. We support all the major data warehouses (incl. straight Postgres), connect to a bunch of different applications, and don't store any of your customer data! Bonus: we recently added native support for dbt :-)

bartekr 6y ago

Can’t wait to try this out sometime soon. I’ve worked with all the founders and they’re awesome.

nloui 6y ago

Looking forward to tracking this project. I totally get it.

recur 6y ago

This was such a massive problem at the large company where I used to work. Excited to see the progress!

ramziq 6y ago

Grouparoo made it dead simple to connect a mysql database and generate information. Easy to Install and Integrations make it easy to send the right data to each tool you use. No code is necessary to change what data gets sent.