3 ms·

I find it interesting how coding/software development still appears to be the one category that these most popular models release specialised models for. Where'

I find it interesting how coding/software development still appears to be the one category that these most popular models release specialised models for. Where's the finance or legal models from Mistral or Meta or OpenAI? Perhaps it's just confirmation bias, but programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. Compared to other types of work, it's relatively more straight forward to tell if code is "correct" or not.

a2128 2y ago
Coding models solve a clear problem and have a clear integration into a developer's workflow - it's like your own personal StackOverflow and it can autocomplete code for you. It's not as clear when it comes to finance or legal, you wouldn't want to rely on an AI that may hallucinate financial numbers or laws. These other professions are also a lot slower to react to change, compared to software development where people are already used to learning new frameworks every year
MikeKusold 2y ago
Those are regulated industries, where as software development is not. An AI spitting back bad code won't compile. An AI spitting back bad financial/legal advice bankrupts people.
- knicholes 2y ago
  Generally I agree! I saw a guy shamefully admit he didn't read the output carefully enough when using generated code (that ran), but there was a min() instead of a max(), and it messed up a month of his metrics!
troupo 2y ago
The explanation is easier, I think. Consider what data these models are trained on, and who are the immediate developers of these models. The models are trained on a vast set of whatever is available on the internet. They are developed by tech people/programmers who are surprisingly blind to their own biases and interests. There's no surprise that one of the main things they want to try and do is programming, using vast open quantities of Stack Overflow, GitHub and various programming forums. For finance and legal you need to: - think a bit outside the box - be interested in finance and legal - be prepared to carry actual legal liability for the output of your models
- dannyw 2y ago
  > - be prepared to carry actual legal liability for the output of your models Section 230. It's been argued that a response by a LLM, to user input, is "user-generated content" and hence the platform has generally no liability (except CSAM). Nobody has successfully sued.
  - troupo 2y ago
    No one has challenged this. Because LLMs haven't been (widely) used in legal or legally binding contexts
- moffkalast 2y ago
  Then again, we just had this on the front page: https://news.ycombinator.com/item?id=40957990 https://news.ycombinator.com/item?id=40957990 > We first document a significant decline in stock trading volume during ChatGPT outages and find that the effect is stronger for firms with corporate news released immediately before or during the outages. We further document similar declines in the short-run price impact, return variance, and bid-ask spreads, consistent with a reduction in informed trading during the outage periods. Lastly, we use trading volume changes during outages to construct a firm-level measure of the intensity of GAI-assisted trading and provide early evidence of a positive effect of GAI-assisted trading on long-run stock price informativeness. They're being used, but nobody is really saying anything because the stock market is a zero sum game these days and letting anyone else know that this holds water is a recipe for competition. Programming is about the opposite, the more you give, the more you get, so it makes sense to popularize it as a feature.
  - troupo 2y ago
    Stock trading is indistinguishable from gambling :) But true, I forgot that this, too, is part of finance
sakesun 2y ago
Generating code has significant economical benefit. The code once generated can be execute so many times without requiring high computing resources, unlike AI model.
miki123211 2y ago
> Where's the finance or legal models from Mistral or Meta or OpenAI? Programming is "weird" in that it requires both specialized knowledge and specialized languages, and the languages are very different from any language that humans speak. Legal requires specialized knowledge, but legal writing is still just English and it follows English grammar rules, although it's sometimes a very strange "dialect" of English. Finance is weird in its own way, as that requires a lot more boring, highly-precise calculations, and LLMs are notoriously bad at those. I suspect that finance is always going to be some hybrid of an LLM driving an "old school" computer to do the hard math, via a programming language or some other, yet-unenvisioned protocol. > programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. This is true, mostly because of programmers' love of textual languages, textual protocols, CLI interfaces and generally all things text. If we were all coding in Scratch, this would be a lot harder.
- madeofpalk 2y ago
  Yes, it appears to be the clear successful usecase for the technology, in a way that hasn't been replicated for other professions. I remain very sceptical that a chat-like interface is the ideal form for LLMs, yet it seems very optimal for programming specifically, along with Copilot-like interfaces of just outputting text.
sofixa 2y ago
Finance already has their own models and has had them for decades. Market predictions and high frequency trading is literally what all the hedge funds and the like have been doing for a few decades now. Including advanced sources of information like (take with a grain of salt, I've heard it on the internet) using satellite images to measure factory activity and thus predict results. Understandably they're all quite secretive about their tooling because they don't want the competition to have access to the same competitive advantages, and an open source model / third party developing a model doesn't really make sense.
- madeofpalk 2y ago
  I guess finance is not in need of a large language model?
  - Foobar8568 2y ago
    It does but everything is a joke...
317070 2y ago
I work in the field. The reason has not been mentioned yet. It's because (for an unknown reason), having coding and software development in the training mix is really helpful at most other tasks. It improves everything to do with logical thinking by a large margin, and that seems to help with many other downstream tasks. Even if you don't need the programming, you want it in the training mix to get that logical thinking, which is hard to get from other resources. I don't know how much that is true for legal or financial resources.
drewmate 2y ago
It's just easier to iterate and improve on a coding specialist AI when that is also the skill required to iterate on said AI. Products that build on general LLM tech are already being used in other fields. For example, my lawyer friend has started using one by LexisNexis[0] and is duly impressed by how it works. It's only a matter of time before models like that get increasingly specialized for that kind of work, it's just harder for lawyers to drive that kind of change alone. Plus, there's a lot more resistance in 'legacy' professions to any kind of change, much less one that is perceived to threaten the livelihoods of established professionals. Current LLMs are already not bad at a lot of things, but lawyer bots, accountant bots and more are likely coming. [0] https://www.lexisnexis.com/en-us/products/lexis-plus-ai.page https://www.lexisnexis.com/en-us/products/lexis-plus-ai.page

a2128 2y ago

Coding models solve a clear problem and have a clear integration into a developer's workflow - it's like your own personal StackOverflow and it can autocomplete code for you. It's not as clear when it comes to finance or legal, you wouldn't want to rely on an AI that may hallucinate financial numbers or laws. These other professions are also a lot slower to react to change, compared to software development where people are already used to learning new frameworks every year

MikeKusold 2y ago

Those are regulated industries, where as software development is not. An AI spitting back bad code won't compile. An AI spitting back bad financial/legal advice bankrupts people.

knicholes 2y ago
Generally I agree! I saw a guy shamefully admit he didn't read the output carefully enough when using generated code (that ran), but there was a min() instead of a max(), and it messed up a month of his metrics!

troupo 2y ago

The explanation is easier, I think. Consider what data these models are trained on, and who are the immediate developers of these models. The models are trained on a vast set of whatever is available on the internet. They are developed by tech people/programmers who are surprisingly blind to their own biases and interests. There's no surprise that one of the main things they want to try and do is programming, using vast open quantities of Stack Overflow, GitHub and various programming forums. For finance and legal you need to: - think a bit outside the box - be interested in finance and legal - be prepared to carry actual legal liability for the output of your models

dannyw 2y ago
> - be prepared to carry actual legal liability for the output of your models Section 230. It's been argued that a response by a LLM, to user input, is "user-generated content" and hence the platform has generally no liability (except CSAM). Nobody has successfully sued.
- troupo 2y ago
  No one has challenged this. Because LLMs haven't been (widely) used in legal or legally binding contexts
moffkalast 2y ago
Then again, we just had this on the front page: https://news.ycombinator.com/item?id=40957990 https://news.ycombinator.com/item?id=40957990 > We first document a significant decline in stock trading volume during ChatGPT outages and find that the effect is stronger for firms with corporate news released immediately before or during the outages. We further document similar declines in the short-run price impact, return variance, and bid-ask spreads, consistent with a reduction in informed trading during the outage periods. Lastly, we use trading volume changes during outages to construct a firm-level measure of the intensity of GAI-assisted trading and provide early evidence of a positive effect of GAI-assisted trading on long-run stock price informativeness. They're being used, but nobody is really saying anything because the stock market is a zero sum game these days and letting anyone else know that this holds water is a recipe for competition. Programming is about the opposite, the more you give, the more you get, so it makes sense to popularize it as a feature.
- troupo 2y ago
  Stock trading is indistinguishable from gambling :) But true, I forgot that this, too, is part of finance

sakesun 2y ago

Generating code has significant economical benefit. The code once generated can be execute so many times without requiring high computing resources, unlike AI model.

miki123211 2y ago

> Where's the finance or legal models from Mistral or Meta or OpenAI? Programming is "weird" in that it requires both specialized knowledge and specialized languages, and the languages are very different from any language that humans speak. Legal requires specialized knowledge, but legal writing is still just English and it follows English grammar rules, although it's sometimes a very strange "dialect" of English. Finance is weird in its own way, as that requires a lot more boring, highly-precise calculations, and LLMs are notoriously bad at those. I suspect that finance is always going to be some hybrid of an LLM driving an "old school" computer to do the hard math, via a programming language or some other, yet-unenvisioned protocol. > programming really does seem to be the ideal usecase for LLMs in a way that other professions just haven't been able to crack. This is true, mostly because of programmers' love of textual languages, textual protocols, CLI interfaces and generally all things text. If we were all coding in Scratch, this would be a lot harder.

madeofpalk 2y ago
Yes, it appears to be the clear successful usecase for the technology, in a way that hasn't been replicated for other professions. I remain very sceptical that a chat-like interface is the ideal form for LLMs, yet it seems very optimal for programming specifically, along with Copilot-like interfaces of just outputting text.

sofixa 2y ago

Finance already has their own models and has had them for decades. Market predictions and high frequency trading is literally what all the hedge funds and the like have been doing for a few decades now. Including advanced sources of information like (take with a grain of salt, I've heard it on the internet) using satellite images to measure factory activity and thus predict results. Understandably they're all quite secretive about their tooling because they don't want the competition to have access to the same competitive advantages, and an open source model / third party developing a model doesn't really make sense.

madeofpalk 2y ago
I guess finance is not in need of a large language model?
- Foobar8568 2y ago
  It does but everything is a joke...

317070 2y ago

I work in the field. The reason has not been mentioned yet. It's because (for an unknown reason), having coding and software development in the training mix is really helpful at most other tasks. It improves everything to do with logical thinking by a large margin, and that seems to help with many other downstream tasks. Even if you don't need the programming, you want it in the training mix to get that logical thinking, which is hard to get from other resources. I don't know how much that is true for legal or financial resources.

drewmate 2y ago

It's just easier to iterate and improve on a coding specialist AI when that is also the skill required to iterate on said AI. Products that build on general LLM tech are already being used in other fields. For example, my lawyer friend has started using one by LexisNexis[0] and is duly impressed by how it works. It's only a matter of time before models like that get increasingly specialized for that kind of work, it's just harder for lawyers to drive that kind of change alone. Plus, there's a lot more resistance in 'legacy' professions to any kind of change, much less one that is perceived to threaten the livelihoods of established professionals. Current LLMs are already not bad at a lot of things, but lawyer bots, accountant bots and more are likely coming. [0] https://www.lexisnexis.com/en-us/products/lexis-plus-ai.page https://www.lexisnexis.com/en-us/products/lexis-plus-ai.page