5 ms·

100% agree. And we are not there yet. But I feel like open source is the best answer to 1 (building an ecosystem of integrations), and that it will also allow u

by iFelix 3y ago

100% agree. And we are not there yet. But I feel like open source is the best answer to 1 (building an ecosystem of integrations), and that it will also allow us to build a better platform for developers (2): why learn Aura components/Apex when you already know and can use Typescript/React?

solarkraft 3y ago
A "real" development stack might enable users to build actually good experiences. The custom (proprietary) stacks I've seen so far have always been pretty "meh", lacking features, community and tooling, resulting in not-great DX and making building some features something between very hacky and impossible.
btown 3y ago
An open-source core is only part of it, because different users will need different data models for different business domains. One of the biggest challenges here will be: how can you allow tenants to run arbitrary plugins that (a) execute arbitrary sandboxed server-side code, and (b) can store arbitrary new data models and custom fields within your instance's centralized storage alongside the canonical model types, in a way that allows for indices, foreign key constraints, and derived materialized fields? If you solve this and provide a great developer experience, including free sandbox accounts and a payments stack so that developers can sell plugins without needing to ever operate their own infrastructure, with namespacing to avoid compatibility problems between apps, then the ecosystem will come.
- charlesTwenty 3y ago
  That's a very good point. In the early stage, we were thinking about a single tenant architecture which would make this questions way easier. However, I am a strong believer in multi-tenant architectures as they allow to scale while mutualizing the resources (I personally think it's single tenant at scale is non-sense in term of ecology) and we will invest into maintaining a multi-tenant architecture. So, as we go through the multi-tenant path, what you say is very relevant and will be challenging. Regarding custom entities and custom fields, we plan to introduce a flexible data table backed by a meta-data, quite close to what salesforce is doing ; this article is gold about how they built it: https://architect.salesforce.com/fundamentals/platform-multitenant-architecture https://architect.salesforce.com/fundamentals/platform-multi.... In short, you have a data table (uuid, objectid, tenantid, field1, ..., fiel500) where fields are VARCHARs and you build your own engine on top of that. This comes with a lot of challenges such as performances (indexation), typing (we lose Typescript/GraphQL power obviously as we deal with flexible data modeling) Regarding plugins that we want users to be able to create and to activate on the marketplace without vetting, here is the way we see it right now: 1) Front-end: serve a dedicated JS depending on what workspace you are on. Rebuild this JS when you activate / update a plugin 2) Back-end: we will need to execute the code in a separate environment. We were thinking about serverless lambdas for the cloud version and keep it local on the main server for self-hosting ; kind of allowing two drivers (lambdas + local) to execute plugin code in the codebase but using lambdas only on the the cloud). Would love to chat a bit more about it. We will likely open a Github discussion thread in the upcoming weeks about this specific topic) so we can get the feedback from anyone interested into it
  - kgeist 3y ago
    >However, I am a strong believer in multi-tenant architectures as they allow to scale while mutualizing the resources (I personally think it's single tenant at scale is non-sense in term of ecology) and we will invest into maintaining a multi-tenant architecture. Our products use multi-tenant architecture in the form of 1 database file per company, and a single database server for everyone (by default). It's great for data isolation, as we can't accidentally leak sensitive corporate data from one company's account to another (say, a missing WHERE). It's also great for indexing, as DB queries only touch small subsets of data. And it works well for most businesses (10-100 employees). For large companies (not that many of them), if we detect a lot of activity which stresses the main database server, we have infrastructure in place to migrate them to dedicated servers, transparently to users. It's worked pretty well so far.
    - charlesTwenty 3y ago
      Very interesting, this makes sense, it is definitely a direction we could go into. We are using Postgres and were also considering using 1 schema per tenant.
- primitivesuave 3y ago
  HubSpot recently launched something to do this by providing a Lambda runtime to generate components within the CRM dashboard. https://developers.hubspot.com/docs/cms/data/serverless-functions https://developers.hubspot.com/docs/cms/data/serverless-func...
  - charlesTwenty 3y ago
    +1, thanks for the link
    - bluelightning2k 3y ago
      We built a version of this which goes far further. - Monaco (embedded VScode inside the target app/platform) - Typescript - NPM modules - Linting/autocomplete including custom fields/objects, so you basically know it's going to work before you even run it - breakpoint debugging - Version control with diff view - Magic utilities to call the platform's own APIs in an easy typesafe way The breakpoint debugging provides an amazing experience for the embedding app. It's pretty magic. But because the runtimes like Lambda don't ship the Inspector API we had to create a custom compiler to make it work. We are actively looking to license this stack to other SaaS looking to build platforms. If you want a demo leave your email and I'll reach out.
sb8244 3y ago
The ecosystem is really not about Aura/Apex. I think the API is a bigger piece. Pretty much everything in Salesforce can be controlled via the API. That isn't even the big challenge though. The biggest challenge is getting people to build for your platform. If a sales team uses 10+ integrations (it's honestly probably 20-50), then they will pick a platform that supports 9-10 of their integrations.

by iFelix 3y ago

solarkraft 3y ago

A "real" development stack might enable users to build actually good experiences. The custom (proprietary) stacks I've seen so far have always been pretty "meh", lacking features, community and tooling, resulting in not-great DX and making building some features something between very hacky and impossible.

btown 3y ago

An open-source core is only part of it, because different users will need different data models for different business domains. One of the biggest challenges here will be: how can you allow tenants to run arbitrary plugins that (a) execute arbitrary sandboxed server-side code, and (b) can store arbitrary new data models and custom fields within your instance's centralized storage alongside the canonical model types, in a way that allows for indices, foreign key constraints, and derived materialized fields? If you solve this and provide a great developer experience, including free sandbox accounts and a payments stack so that developers can sell plugins without needing to ever operate their own infrastructure, with namespacing to avoid compatibility problems between apps, then the ecosystem will come.

charlesTwenty 3y ago
That's a very good point. In the early stage, we were thinking about a single tenant architecture which would make this questions way easier. However, I am a strong believer in multi-tenant architectures as they allow to scale while mutualizing the resources (I personally think it's single tenant at scale is non-sense in term of ecology) and we will invest into maintaining a multi-tenant architecture. So, as we go through the multi-tenant path, what you say is very relevant and will be challenging. Regarding custom entities and custom fields, we plan to introduce a flexible data table backed by a meta-data, quite close to what salesforce is doing ; this article is gold about how they built it: https://architect.salesforce.com/fundamentals/platform-multitenant-architecture https://architect.salesforce.com/fundamentals/platform-multi.... In short, you have a data table (uuid, objectid, tenantid, field1, ..., fiel500) where fields are VARCHARs and you build your own engine on top of that. This comes with a lot of challenges such as performances (indexation), typing (we lose Typescript/GraphQL power obviously as we deal with flexible data modeling) Regarding plugins that we want users to be able to create and to activate on the marketplace without vetting, here is the way we see it right now: 1) Front-end: serve a dedicated JS depending on what workspace you are on. Rebuild this JS when you activate / update a plugin 2) Back-end: we will need to execute the code in a separate environment. We were thinking about serverless lambdas for the cloud version and keep it local on the main server for self-hosting ; kind of allowing two drivers (lambdas + local) to execute plugin code in the codebase but using lambdas only on the the cloud). Would love to chat a bit more about it. We will likely open a Github discussion thread in the upcoming weeks about this specific topic) so we can get the feedback from anyone interested into it
- kgeist 3y ago
  >However, I am a strong believer in multi-tenant architectures as they allow to scale while mutualizing the resources (I personally think it's single tenant at scale is non-sense in term of ecology) and we will invest into maintaining a multi-tenant architecture. Our products use multi-tenant architecture in the form of 1 database file per company, and a single database server for everyone (by default). It's great for data isolation, as we can't accidentally leak sensitive corporate data from one company's account to another (say, a missing WHERE). It's also great for indexing, as DB queries only touch small subsets of data. And it works well for most businesses (10-100 employees). For large companies (not that many of them), if we detect a lot of activity which stresses the main database server, we have infrastructure in place to migrate them to dedicated servers, transparently to users. It's worked pretty well so far.
  - charlesTwenty 3y ago
    Very interesting, this makes sense, it is definitely a direction we could go into. We are using Postgres and were also considering using 1 schema per tenant.
primitivesuave 3y ago
HubSpot recently launched something to do this by providing a Lambda runtime to generate components within the CRM dashboard. https://developers.hubspot.com/docs/cms/data/serverless-functions https://developers.hubspot.com/docs/cms/data/serverless-func...
- charlesTwenty 3y ago
  +1, thanks for the link
  - bluelightning2k 3y ago
    We built a version of this which goes far further. - Monaco (embedded VScode inside the target app/platform) - Typescript - NPM modules - Linting/autocomplete including custom fields/objects, so you basically know it's going to work before you even run it - breakpoint debugging - Version control with diff view - Magic utilities to call the platform's own APIs in an easy typesafe way The breakpoint debugging provides an amazing experience for the embedding app. It's pretty magic. But because the runtimes like Lambda don't ship the Inspector API we had to create a custom compiler to make it work. We are actively looking to license this stack to other SaaS looking to build platforms. If you want a demo leave your email and I'll reach out.

sb8244 3y ago

The ecosystem is really not about Aura/Apex. I think the API is a bigger piece. Pretty much everything in Salesforce can be controlled via the API. That isn't even the big challenge though. The biggest challenge is getting people to build for your platform. If a sales team uses 10+ integrations (it's honestly probably 20-50), then they will pick a platform that supports 9-10 of their integrations.