3 ms·
Sure. An example is classification. Take suppose a company that provides a platform for other companies but only if their businesses are supportable under a var
by fnordpiglet 3y ago
Sure. An example is classification. Take suppose a company that provides a platform for other companies but only if their businesses are supportable under a variety of regimes, such as industry (no cannabis say), sanctions and export controls, etc. Typically this is done by training up an army of people that poorly review websites and collateral by hand and make some determinations with a very low precision and recall, and a very high expense profile including training, benefits, HR, etc, with high attrition. Building these operations is complex, time consuming, and often fails to produce meaningfully useful results for years.
With LLM at GPT4 level ability and a modest investment in scraping, context, building an agent based tool you can automate the entire thing with a pretty high precision and recall (such as a 0.9 precision with a 0.8 recall, vs 0.4 / 0.5 for humans). Using classical NLP, schema matching, and classification techniques you could triage and improve human performance by as much as 20%. But that requires a lot of effort and provides important but modest improvements. The LLM approach however is almost effortless from an engineering effort (note all of the infrastructure is the same as the NLP/schema matching/classification/etc effort).
This may not seem earth shatteringly important to you, but for an enterprise this is monumental. It dramatically reduces risk, complexity, cost, and all the while improves outcomes by an enormous amount.
Another example is incident management. Having LLM agents monitoring incidents on slack as reduced our MTRR dramatically. They provide summaries for new joiners, broad cast management updates, etc, and take care of all the glue out response teams typically did but with a high degree of quality, low degree of variability, and extremely low latencies compared to humans.
Another example is policy management. Many enterprises are saddled with thousands of pages of external requirements from regulators, and those in turn create tens of thousands of pages of policy, and hundreds of thousands of LoC of control implementations. By indexing them into a Context+RAG LLM you can create a policy oracle, allowing you to create coverage and effectiveness testing, analyze for policy or control gaps, draft responses to auditors and regulators with total policy knowledge, answer legals questions, advise management on the implications of a business driven exception, etc. The state of the art before was a search engine dumping phrase based matches into a grid of thousands of rows of results, which was effectively useless for all of these use cases.
- PH95VuimJjqBqy 3y ago> training up an army of people that poorly review websites and collateral by hand and make some determinations with a very low precision and recall, and a very high expense profile including training, benefits, HR, etc, with high attrition. Building these operations is complex, time consuming, and often fails to produce meaningfully useful results for years. followed by > GPT4 level ability and a modest investment in scraping oh yeah, these speech patterns are those of someone who isn't unnecessarily hyped while trying to downplay other approaches. no bias here at all folks...
- fnordpiglet 3y agoI’ve done the other approaches for decades, specifically using NLP and classifiers to try to do these and things, my friend. It’s been a really grueling effort with little success. But as an example in the real world visible to most people, the product catalog at eBay is the end product of my work in applying NLP and classifiers at building product catalogs, ontologies, and taxonomies automatically from scrapes of e-commerce sites etc. GPT4 would have turned my job into a simple exercise in structuring the classifications. It would have completely revolutionized what I had to do. It’s readily apparent from the pessimistic HN threads they come from people who don’t actually do these things for a living and haven’t had much real world experience. These speech patterns as you call them are the way people who “do things” differentiate from those who can’t or haven’t done things. It’s called “experience.”
- JohnPrine 3y agoThanks for the comments, that was really enlightening. Not sure why people on a tech forum seem so eager to dismiss an emerging technology rather than using their imaginations a little
- PH95VuimJjqBqy 3y agono one is doing that, the issue is the overhyping. When claude shannon first came out with his information theory everyone started trying to apply it to everything, discuss everything in terms of his theories. He himself thought it was ridiculous and basically told everyone to stop it. This is no different.