5 ms·

I just asked Claude to generate some code using the SAP SuccessFactors API. Every single example was completely useless. The code wouldn't compile, it would in

I just asked Claude to generate some code using the SAP SuccessFactors API. Every single example was completely useless. The code wouldn't compile, it would invent methods and variables and the instructions to go along with it were incoherent. All whilst gaslighting along with the way. I have also previously tried using it with some Golang code and it would constantly add weird statements e.g. locking on non-concurrent operations. LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth.

dankwizard 2y ago
"LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth." If you're doing something in a way it's not in the training data set, maybe your way of approaching the problem is wrong?
- threeseed 2y ago
  Sorry but some of us aren't building the ten millionth CRUD app. SuccessFactors is a popular HR platform and I was asking it any question and getting the wrong answer every time.
- johnnyanmac 2y ago
  >If you're doing something in a way it's not in the training data set in my industry, the "training data set" won't get much farther from public code than the barebones, generated doxygen comments we call "documentation". But in a way you're also right. The industry's approach is fundamentally wrong, making 20 solutions to a problem with plenty of room to standardize a proper approach (plenty of room where you need proprietary techniques, but that's getting less true by the month). But an LLM isn't going to fix that cultural issue and will suffer from it.
  - warkdarrior 2y ago
    > The industry's approach is fundamentally wrong, making 20 solutions to a problem with plenty of room to standardize a proper approach [...]. But an LLM isn't going to fix that cultural issue and will suffer from it. LLM-powered development may push the industry towards standardization. "Oh, CoPilot cannot generate proper code for your SDK/API/service? Sorry, all my developers use CoPilot, so we will not integrate with your SDK/API/service until you provide better, CoPilot-friendly docs and examples."
attentive 2y ago
for obscure API or SDK, upload docs and/or examples to Claude projects.
- johnnyanmac 2y ago
  That sounds like you might potentially break some copyright of your tools. Not all our tools are FOSS (and even some FOSS licenses may not allow that).
  - attentive 2y ago
    You either feed your code into API or you don't. If you can't share your code with Anthropic then there is nothing to talk about. Expecting it to know non-public SDK's and docs isn't reasonable. But if you want it to help you with FOSS libs/docs, this is the way.

dankwizard 2y ago

"LLMs are great when you are doing the same things as everyone else. Step outside of that and it's far more trouble than it's worth." If you're doing something in a way it's not in the training data set, maybe your way of approaching the problem is wrong?

threeseed 2y ago
Sorry but some of us aren't building the ten millionth CRUD app. SuccessFactors is a popular HR platform and I was asking it any question and getting the wrong answer every time.
johnnyanmac 2y ago
>If you're doing something in a way it's not in the training data set in my industry, the "training data set" won't get much farther from public code than the barebones, generated doxygen comments we call "documentation". But in a way you're also right. The industry's approach is fundamentally wrong, making 20 solutions to a problem with plenty of room to standardize a proper approach (plenty of room where you need proprietary techniques, but that's getting less true by the month). But an LLM isn't going to fix that cultural issue and will suffer from it.
- warkdarrior 2y ago
  > The industry's approach is fundamentally wrong, making 20 solutions to a problem with plenty of room to standardize a proper approach [...]. But an LLM isn't going to fix that cultural issue and will suffer from it. LLM-powered development may push the industry towards standardization. "Oh, CoPilot cannot generate proper code for your SDK/API/service? Sorry, all my developers use CoPilot, so we will not integrate with your SDK/API/service until you provide better, CoPilot-friendly docs and examples."

attentive 2y ago

for obscure API or SDK, upload docs and/or examples to Claude projects.

johnnyanmac 2y ago
That sounds like you might potentially break some copyright of your tools. Not all our tools are FOSS (and even some FOSS licenses may not allow that).
- attentive 2y ago
  You either feed your code into API or you don't. If you can't share your code with Anthropic then there is nothing to talk about. Expecting it to know non-public SDK's and docs isn't reasonable. But if you want it to help you with FOSS libs/docs, this is the way.