5 ms·

Everywhere I've worked at that had some kind of Salesforce integration, that integration seemed almost incomprehensibly complex or a source of endless problems,

by pxc 2y ago

Everywhere I've worked at that had some kind of Salesforce integration, that integration seemed almost incomprehensibly complex or a source of endless problems, and often both. But I've never been (nor wanted to be) very closely involved with any such integrations. Is Salesforce garbage? Is that just how CRM systems are? Is everybody just doing it wrong? What's the deal?

deleted 2y ago
[deleted]
WgaqPdNr7PGLGVW 2y ago
> Is that just how CRM systems are? Is everybody just doing it wrong? What's the deal? These kinds of tools cover 80% of what you want to do out-of-the-box. For the remaining 20% to build it correctly you need to either hire expensive consultants or hire in-house staff to build. Nobody budgets properly for this, and it isn't in the sales pitch, and so that last 20% is built as horrible spaghetti code by the cheapest possible devs / consultants. Even if you wanted to pay good salaries and hire people in-house how many great engineers want to be limited to programming in Apex on salesforce?
gregw2 2y ago
I've only been involved with such Salesforce integrations at one company, but based on that, I can give you my take. I'd be interested to hear others'. One, the salesforce data changes all occur through APIs (ok) which various enterprise integration tools (Informatica, Mulesoft, etc) support (ok), but those tools typically dont support easy options for retrying a change to a specific row that causes an error. If you are updating 100 Accounts in a single Salesforce Bulk API call and "5" are busy/locked, you have to build a lot of custom error handling to notice and retry just those 5 rows. Its not part of most connectors or python libraries I've seen. Also, 3 of those errors might be fatal and 2 might be retriable but you have to learn which are which or blindly retry them all. In database terms, their API transactional semantics are not statement by statement ACID but row by row within an API request. Second, no API or SOQL operations can pull back or process or update more than 50,000 rows. Given those two things, unless the integration person is skilled about both error handling and testing, some of the object busy/contention failures only show up in production traffic with many jobs going on so a generic integration specialist doesn't know about these Salesforce-specific pitfalls and they are discovered after the integration goes live under strange production access patterns. EDIT: a third issue is that most Salesforce developers are UI-centric in their thinking and training and don't have database or data modeling or data integration experience to draw on so the troubleshooting for data issues on their end tends to suffer.
- rawgabbit 2y ago
  I have been there and done that. In a complex SF org with a lot of triggers, any record update will get blocked. The only solution is to refactor all the apex triggers processes and flows to something more orderly. Technically it is doable. Politically it is almost impossible. SF is an ERP in most companies and touches every department.
  - chasd00 2y ago
    Salesforce definitely makes it easy to screw yourself with customization. It’s a very complex beast.
- chasd00 2y ago
  The regular REST api has request scoped transactions. Only the bulk api has the issues you describe. The bulk api is kind of a special thing annyway and has its quirks. The regular REST api works more like what you’d expect.
  - gregw2 2y ago
    Great point. That said, the remaining problem is that regular API calls dont scale/perform well for integrations between systems involving lots of data syncing. A few records? Fine. Many? Not so much. I would love to be wrong on this.
    - chasd00 2y ago
      Yeah pushing or pulling large, according to salesforce, numbers of records is definitely harder than it should be. On the “push” side you have row lock errors and no way to disable APEX based triggers unless you’ve designed that into the code itself. On the pull side, if you’re trying to extract a large number of records and your soql query times out you’re out of luck. SF is good about creating custom indexes for you through a support case but it takes time. Even then, on the order of millions of records it’s still difficult. My day job is implementing large SF projects. Multi-million record data migrations arent unusual. Even if the data is clean and mapped the migrations take weeks of planning. We go over every inch of the setup to make sure we have the best chance of getting a clean load on try #1. However, we schedule for 3 trial loads and verification before a “go live load” into actual production. Even after all that it’s still an all nighter with contingency plans and c-suite cell numbers on deck.
rawgabbit 2y ago
The actual Salesforce core products are very good. Their documentation is well written. The Salesforce flow low code tool works well. Their overly complex object/row/field permissions is a hot mess. Mulesoft is limited; there is a reason why they tried to buy Informatica. Their marketing and hype machine hurts their credibility imo.
maxfurman 2y ago
Salesforce has the ability for users to define their own bespoke data models. This modeling is nearly always done by sales people who may or may not be good at sales, but are almost never good at data modeling - not an insult, simply stating that data modeling isn't their job - and so the models are almost always a mess. The problems flow from there

by pxc 2y ago

deleted 2y ago

[deleted]

WgaqPdNr7PGLGVW 2y ago

> Is that just how CRM systems are? Is everybody just doing it wrong? What's the deal? These kinds of tools cover 80% of what you want to do out-of-the-box. For the remaining 20% to build it correctly you need to either hire expensive consultants or hire in-house staff to build. Nobody budgets properly for this, and it isn't in the sales pitch, and so that last 20% is built as horrible spaghetti code by the cheapest possible devs / consultants. Even if you wanted to pay good salaries and hire people in-house how many great engineers want to be limited to programming in Apex on salesforce?

gregw2 2y ago

I've only been involved with such Salesforce integrations at one company, but based on that, I can give you my take. I'd be interested to hear others'. One, the salesforce data changes all occur through APIs (ok) which various enterprise integration tools (Informatica, Mulesoft, etc) support (ok), but those tools typically dont support easy options for retrying a change to a specific row that causes an error. If you are updating 100 Accounts in a single Salesforce Bulk API call and "5" are busy/locked, you have to build a lot of custom error handling to notice and retry just those 5 rows. Its not part of most connectors or python libraries I've seen. Also, 3 of those errors might be fatal and 2 might be retriable but you have to learn which are which or blindly retry them all. In database terms, their API transactional semantics are not statement by statement ACID but row by row within an API request. Second, no API or SOQL operations can pull back or process or update more than 50,000 rows. Given those two things, unless the integration person is skilled about both error handling and testing, some of the object busy/contention failures only show up in production traffic with many jobs going on so a generic integration specialist doesn't know about these Salesforce-specific pitfalls and they are discovered after the integration goes live under strange production access patterns. EDIT: a third issue is that most Salesforce developers are UI-centric in their thinking and training and don't have database or data modeling or data integration experience to draw on so the troubleshooting for data issues on their end tends to suffer.

rawgabbit 2y ago
I have been there and done that. In a complex SF org with a lot of triggers, any record update will get blocked. The only solution is to refactor all the apex triggers processes and flows to something more orderly. Technically it is doable. Politically it is almost impossible. SF is an ERP in most companies and touches every department.
- chasd00 2y ago
  Salesforce definitely makes it easy to screw yourself with customization. It’s a very complex beast.
chasd00 2y ago
The regular REST api has request scoped transactions. Only the bulk api has the issues you describe. The bulk api is kind of a special thing annyway and has its quirks. The regular REST api works more like what you’d expect.
- gregw2 2y ago
  Great point. That said, the remaining problem is that regular API calls dont scale/perform well for integrations between systems involving lots of data syncing. A few records? Fine. Many? Not so much. I would love to be wrong on this.
  - chasd00 2y ago
    Yeah pushing or pulling large, according to salesforce, numbers of records is definitely harder than it should be. On the “push” side you have row lock errors and no way to disable APEX based triggers unless you’ve designed that into the code itself. On the pull side, if you’re trying to extract a large number of records and your soql query times out you’re out of luck. SF is good about creating custom indexes for you through a support case but it takes time. Even then, on the order of millions of records it’s still difficult. My day job is implementing large SF projects. Multi-million record data migrations arent unusual. Even if the data is clean and mapped the migrations take weeks of planning. We go over every inch of the setup to make sure we have the best chance of getting a clean load on try #1. However, we schedule for 3 trial loads and verification before a “go live load” into actual production. Even after all that it’s still an all nighter with contingency plans and c-suite cell numbers on deck.

rawgabbit 2y ago

The actual Salesforce core products are very good. Their documentation is well written. The Salesforce flow low code tool works well. Their overly complex object/row/field permissions is a hot mess. Mulesoft is limited; there is a reason why they tried to buy Informatica. Their marketing and hype machine hurts their credibility imo.

maxfurman 2y ago

Salesforce has the ability for users to define their own bespoke data models. This modeling is nearly always done by sales people who may or may not be good at sales, but are almost never good at data modeling - not an insult, simply stating that data modeling isn't their job - and so the models are almost always a mess. The problems flow from there