3 ms·

As I was writing the second part to my Developing Intelligence series, I noticed that In my AI endeavors, I have always drawn a lot of inspiration from the eleg

As I was writing the second part to my Developing Intelligence series, I noticed that In my AI endeavors, I have always drawn a lot of inspiration from the elegant art of Defensive Design and Defensive Programming. This article will (at least attempt to) synthesize and make clear some principles that can be followed in all phases of building AI projects.

jruohonen 3y ago
Defensive programming is a good start. But increasingly, I am starting to think it is a mission impossible due to the too large combinatory space of natural language inputs. If you need to guard output against "malicious" inputs in this context, it would follow that your training data is ill-suited for the task to begin with, no?
- DeadlyPretzel 3y ago
  I was not necessarily talking about malicious input, rather hallucinations I noticed often when using LLMs to generate JSON structures out of (unstructured) data (job postings, resumés, news articles, ...) a lot of language models tend to hallucinate extra properties you didn't ask for There are many valid ways to deal with this I think which do end up giving you satisfactory output 100% of the time (constrained generation etc) It's just important to actually implement these - whereas a lot of the times when I see articles/videos/... of people implementing AI automation, they seem to often just think "Yeah this probably won't work with smaller models, but GPT-4 can do it flawlessly" But of course, no, not even GPT-4 can do it flawlessly 100% of the time, maybe 99% of the time, yes, but nobody is really talking about how to handle or prevent it going wrong. And as a side-effect of handling those things, I found that most of the time, in fact, small models are more than sufficient and significantly cheaper to implement than GPT-4, IF you actually know how to get the most out of them.
  - jruohonen 3y ago
    "I noticed often when using LLMs to generate JSON structures out of (unstructured) data (job postings, resumés, news articles, ...) a lot of language models tend to hallucinate extra properties you didn't ask for." I see; for that kind of thing, including code generation in general, defensive programming indeed seems reasonable. You could probably do post-validation too, as you indicated with assertions. But when you move to "pure" natural language outputs, things get truly hairy. Yet, are you sure a LLM is the right tool to generate JSON structures?
    - DeadlyPretzel 3y ago
      Well it depends what you are generating from, but this does raise a good point/principle. I think "Use traditional code for as long as possible" is a good rule to follow as well. Let's say you are generating JSON from scraped job offerings so you can insert them into a job-finding database. If these job offerings all come from one place and follow a specific format, DEFINITELY use traditional code, LLMs can't do a better job than that! However if you are getting them from multiple places, emails, linkedin, ... and they don't follow a set structure, using traditional code to extract any meaningful data would be utterly hopeless without an LLM So, the only option really is asking the LLM to generate some highly-structured (JSON) data for you based on the unstructured job offer. In this particular use-case though I would go toward using the Guidance library very, very quickly as it doesn't let me down for cases where I need to generate JSON like this. In fact, I am working on an article that goes through this exact use-case and my reasoning step by step, which triggered me to write the Defensive AI Design article first as I noticed I had built up a very specific set of rules/workflow that I almost always follow EDIT: Additionally, I'd say that if you can keep the AI generation to a minimum, and only fill in what you can't with traditional code, then go for it! If you can programmatically extract the job title, date of posting, but you need AI to extract the required skills & education, then that's the best route to take IMHO

jruohonen 3y ago

Defensive programming is a good start. But increasingly, I am starting to think it is a mission impossible due to the too large combinatory space of natural language inputs. If you need to guard output against "malicious" inputs in this context, it would follow that your training data is ill-suited for the task to begin with, no?

DeadlyPretzel 3y ago
I was not necessarily talking about malicious input, rather hallucinations I noticed often when using LLMs to generate JSON structures out of (unstructured) data (job postings, resumés, news articles, ...) a lot of language models tend to hallucinate extra properties you didn't ask for There are many valid ways to deal with this I think which do end up giving you satisfactory output 100% of the time (constrained generation etc) It's just important to actually implement these - whereas a lot of the times when I see articles/videos/... of people implementing AI automation, they seem to often just think "Yeah this probably won't work with smaller models, but GPT-4 can do it flawlessly" But of course, no, not even GPT-4 can do it flawlessly 100% of the time, maybe 99% of the time, yes, but nobody is really talking about how to handle or prevent it going wrong. And as a side-effect of handling those things, I found that most of the time, in fact, small models are more than sufficient and significantly cheaper to implement than GPT-4, IF you actually know how to get the most out of them.
- jruohonen 3y ago
  "I noticed often when using LLMs to generate JSON structures out of (unstructured) data (job postings, resumés, news articles, ...) a lot of language models tend to hallucinate extra properties you didn't ask for." I see; for that kind of thing, including code generation in general, defensive programming indeed seems reasonable. You could probably do post-validation too, as you indicated with assertions. But when you move to "pure" natural language outputs, things get truly hairy. Yet, are you sure a LLM is the right tool to generate JSON structures?
  - DeadlyPretzel 3y ago
    Well it depends what you are generating from, but this does raise a good point/principle. I think "Use traditional code for as long as possible" is a good rule to follow as well. Let's say you are generating JSON from scraped job offerings so you can insert them into a job-finding database. If these job offerings all come from one place and follow a specific format, DEFINITELY use traditional code, LLMs can't do a better job than that! However if you are getting them from multiple places, emails, linkedin, ... and they don't follow a set structure, using traditional code to extract any meaningful data would be utterly hopeless without an LLM So, the only option really is asking the LLM to generate some highly-structured (JSON) data for you based on the unstructured job offer. In this particular use-case though I would go toward using the Guidance library very, very quickly as it doesn't let me down for cases where I need to generate JSON like this. In fact, I am working on an article that goes through this exact use-case and my reasoning step by step, which triggered me to write the Defensive AI Design article first as I noticed I had built up a very specific set of rules/workflow that I almost always follow EDIT: Additionally, I'd say that if you can keep the AI generation to a minimum, and only fill in what you can't with traditional code, then go for it! If you can programmatically extract the job title, date of posting, but you need AI to extract the required skills & education, then that's the best route to take IMHO