6 ms·

I had similar results trying out Amazon Q. I fed it a copy of our employee handbook and had it scrape our corporate website. My idea is that this could be a goo

I had similar results trying out Amazon Q. I fed it a copy of our employee handbook and had it scrape our corporate website. My idea is that this could be a good fit for an HR bot. I asked it a few simple fact based questions, such as who our CEO is. It answered with our previous CEO based on an old press release. I asked it what year it is, and it couldn’t say. A few other specific fact based questions were either wrong or unanswered. So I asked it higher level questions about what the corporation does. It was able to answer those. Finally, I began asking questions about policies in the handbook. When I asked about harassment, it refused to provide any answers. It felt like I was filtered. Filtering will kill the utility of these products. Something like a legal practice is messy business that the filter won’t like.

adastra22 3y ago
Maybe try downloading a LLaMA model and fine tuning it with your corporate handbooks?
- _ink_ 3y ago
  Do you have a link on how to do such a thing?
  - Multiplayer 3y ago
    Here's a fast and easy tool for Mac (& Linux) that I've been using: https://ollama.ai https://ollama.ai
    - adastra22 3y ago
      [flagged]
    - ForkMeOnTinder 3y ago
      ollama is great for off-the-shelf models, but unless I missed the news I don't think it makes fine-tuning on a pile of PDFs any easier.
      - eurekin 3y ago
        I planned trying exactly that on weekend
- s3tt3mbr1n1 3y ago
  It’s not recommended to finetune models on content that was not in the training set. Finetuning should be reserved for desired behavior of the LLM, not to add new knowledge. For that, RAG is better suited.
  - mcfig 3y ago
    Why not? (Is this about effectiveness, training time, forgetting, or something else?)
    - QuantumGood 3y ago
      Finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge is fixed at the point of its last training update. It adjusts how existing knowledge is used, but doesn't add new facts or information post-training. It's more about tweaking responses, biases, and style, rather than updating its factual database. RAG combines a language model like GPT with a real-time search component. This allows the model to pull in information from external sources during its response generation process. Now the ability to access and integrate the most recent information is gained, which the language model alone wouldn't have.
      - adastra22 3y ago
        This is splitting hairs, and pragmatically speaking not wholly accurate. It may even be completely incorrect, depending on your definitions. You can think of an LLM as a set of basis vectors for human knowledge. If I feed in a PR training manual that is not in its dataset, it nevertheless figures out “hey I can make a reasonable approximation of this by combining X, Y, and Z” where X, Y, and Z are things it learned form its training set. In other words it maps the input into a vector representation based on its training data. But in linear algebra two mappings can represent the same vector, just using different basis, so long as the vector space for the two basis are equal (or at least one is a subspace of the other). That's essentially what's going on here. A LLM builds a vector space on top of all human knowledge. If its parameters and training set are large enough, then the basis is in fact sufficient for representing anything you might throw at it. It will represent it in terms of it training set, yes, but that representation is high fidelity enough to represent the document in its entirety. Fine-tuning a model is essentially rebalancing the initial weights of the LLM to pay special attention to certain clusters in its vector space, represented by the fine-tuning data. It's as if I threw random 2D points at a machine learning algorithm and it learned the basis { (1, 0), (0, 1) } representing the x-axis and y-axis. As a consequence of how inference works, it may then end up preferring to generate points when asked which are nearer to one axis or the other. But then I fine-tune it on points that are distributed along the diagonal. This is not representative of the original training data, but NOT "outside" the original data. These points are fully represented by a linear combination of the x- and y- basis vectors. Nevertheless, the fine-tuning trains the model to prefer points which have weights that are multiples of (1, 1) or (1, -1) when represented in the original model. In other words, points along the diagonals. Pragmatically speaking, this is no different from doing a whole new training run on the diagonal points, except that it is much, much cheaper, and has the capacity to reuse whatever knowledge was learned in the first training run.
- thedougd 3y ago
  Thanks. It was more of a proof of concept to see what I could do or not do with Q. The product looks promising to an Enterprise as it has connectors for most of our existing document stores. Executives will ask questions about the technologies they hear about in the news, and it's good to be able to quickly speak to the good and the bad.
bradley13 3y ago
Your comment about filters is important. These companies are so worried that someone will generate "evil" results that they disable whole swaths of potential queries. All sorts of different people may genuinely need those results: Attorneys, doctors and nurses, law enforcement, private investigators. Individuals searching for advice. Doing background checks on other people. Heck, just high school students researching a topic. I'm not sure what to do about the (legitimate) concerns of the AI providers. They don't want a shitstorm, because someone got their AI model to [whatever]. But [whatever] can genuinely be important.
- wredue 3y ago
  They’re not “worried about it”. When filters are removed, one of the first things people do to the bots is make them racists. Time and time again. It happens every time.
  - deleted 3y ago
    [deleted]
  - Mountain_Skies 3y ago
    There are many examples of people testing LLMs on which is worse, extinguishing all of humanity or making a racist comment in a place where no one will hear it. The LLMs keep choosing to extinguish humanity. Just something to consider when using "but it could say something racist" as justification for filtering. Is racism so evil that it's better to kill every human to avoid it ever happening? Maybe the only option in that case is to prohibit the existence of LLMs. We already have corporations in the role of powerful sociopaths preying on society, do we need LLMs looking to kill us all to satisfy its goals too?
    - NoraCodes 3y ago
      > Is racism so evil that it's better to kill every human to avoid it ever happening? The companies that make and operate these systems - like most companies - are more interested in avoiding PR problems than avoiding harm to humans. This is just one somewhat humorous example of that general policy.
Jenk 3y ago
The HR bot use case is very close to my domain - literally one of the teams reporting to me are maintaining the HR bot for a large enterprise - and it is a legal minefield. Whatever the bot provides as a response could be legally binding, according to legal. Examples of the kind of prompts we get in a many-thousands sized org: - What are my p/maternity entitlements? - When will I be promoted? - What will the bonus pool be this year? - What will happen to me if my boss finds out I have been slacking off? and many more examples of prompts that HR/legal simply will not accept a vague, or worse: incorrect, response to. Each of those types of questions require specific and considerate knowledge that LLMs simply cannot produce.
- deleted 3y ago
  [deleted]
- thedougd 3y ago
  You've sufficiently talked me out of it.
- dfxm12 3y ago
  Whatever the bot provides as a response could be legally binding, according to legal. I wonder if it could be legally binding according to any actual law though.
  - lesuorac 3y ago
    I mean if you ask HR (bot) how many weeks of paternity leave you have and it says 20 and so you disappear for 20 weeks and get fired after 8 because it actually ran out are you really in the wrong? While there are laws about minimum time in some states; the state is probably happy to enforce whatever extra time you grant. So if you tell your employees 20 they're probably going to be happy to hold you to 20.
    - jandrese 3y ago
      Or the bot responds with 2 weeks which is below the legal minimum for your state and the employee sues.
throwup238 3y ago
> Filtering will kill the utility of these products. Something like a legal practice is messy business that the filter won’t like. I’ve been trying to apply all the big name LLMs to analyze court opinions from the Caselaw project and holy mother of god, it’s all but useless in the most ridiculous way. Criminal cases are almost universally guaranteed to trip up at least one of their safety filters. There’s some truly awful stuff in there… which is kinda the point of having an LLM read this stuff.
- qingcharles 3y ago
  Yeah, almost every criminal case law causes it to think you are about to commit the crime itself and instead gives you another boring moral lecture about what an awful human you are. That and it hallucinates me a bunch of new case cites :D
ren_engineer 3y ago
>I asked it a few simple fact based questions, such as who our CEO is. It answered with our previous CEO based on an old press release. this is the problem with basic RAG implementations that most products are currently using, simple vector search isn't able to handle queries like this. The solution is enhancing the base query and also using structured data to filter on metadata here's a good article covering some strategies - https://jxnl.github.io/instructor/blog/2023/09/17/rag-is-more-than-just-embedding-search/ https://jxnl.github.io/instructor/blog/2023/09/17/rag-is-mor...
qingcharles 3y ago
Try Claude? I've been through a lot of LLMs and Claude is really the best with larger documents.
gentleman11 3y ago
Same when using it as a writing aid: the censorship prevents it from answering two thirds the questions you want to ask about, making it faster to just search for the information yourself in Ddg or brave
hfhdjdks 3y ago
I see a lot of people asking "what year it is" or some other question that tests knowledge lf current events. To me it sounds like a misunderstanding of what an LLM + retrieval augmentation does. If the LLM is only fed the HR documents that have multiple dates, why would you expect it to answer that correctly? Why is it relevant that it answered it correctly.

adastra22 3y ago

Maybe try downloading a LLaMA model and fine tuning it with your corporate handbooks?

_ink_ 3y ago
Do you have a link on how to do such a thing?
- Multiplayer 3y ago
  Here's a fast and easy tool for Mac (& Linux) that I've been using: https://ollama.ai https://ollama.ai
  - adastra22 3y ago
    [flagged]
  - ForkMeOnTinder 3y ago
    ollama is great for off-the-shelf models, but unless I missed the news I don't think it makes fine-tuning on a pile of PDFs any easier.
    - eurekin 3y ago
      I planned trying exactly that on weekend
s3tt3mbr1n1 3y ago
It’s not recommended to finetune models on content that was not in the training set. Finetuning should be reserved for desired behavior of the LLM, not to add new knowledge. For that, RAG is better suited.
- mcfig 3y ago
  Why not? (Is this about effectiveness, training time, forgetting, or something else?)
  - QuantumGood 3y ago
    Finetuning means adjusting parameters based on a smaller, specific dataset to tailor the LLM's responses, but the model's underlying knowledge is fixed at the point of its last training update. It adjusts how existing knowledge is used, but doesn't add new facts or information post-training. It's more about tweaking responses, biases, and style, rather than updating its factual database. RAG combines a language model like GPT with a real-time search component. This allows the model to pull in information from external sources during its response generation process. Now the ability to access and integrate the most recent information is gained, which the language model alone wouldn't have.
    - adastra22 3y ago
      This is splitting hairs, and pragmatically speaking not wholly accurate. It may even be completely incorrect, depending on your definitions. You can think of an LLM as a set of basis vectors for human knowledge. If I feed in a PR training manual that is not in its dataset, it nevertheless figures out “hey I can make a reasonable approximation of this by combining X, Y, and Z” where X, Y, and Z are things it learned form its training set. In other words it maps the input into a vector representation based on its training data. But in linear algebra two mappings can represent the same vector, just using different basis, so long as the vector space for the two basis are equal (or at least one is a subspace of the other). That's essentially what's going on here. A LLM builds a vector space on top of all human knowledge. If its parameters and training set are large enough, then the basis is in fact sufficient for representing anything you might throw at it. It will represent it in terms of it training set, yes, but that representation is high fidelity enough to represent the document in its entirety. Fine-tuning a model is essentially rebalancing the initial weights of the LLM to pay special attention to certain clusters in its vector space, represented by the fine-tuning data. It's as if I threw random 2D points at a machine learning algorithm and it learned the basis { (1, 0), (0, 1) } representing the x-axis and y-axis. As a consequence of how inference works, it may then end up preferring to generate points when asked which are nearer to one axis or the other. But then I fine-tune it on points that are distributed along the diagonal. This is not representative of the original training data, but NOT "outside" the original data. These points are fully represented by a linear combination of the x- and y- basis vectors. Nevertheless, the fine-tuning trains the model to prefer points which have weights that are multiples of (1, 1) or (1, -1) when represented in the original model. In other words, points along the diagonals. Pragmatically speaking, this is no different from doing a whole new training run on the diagonal points, except that it is much, much cheaper, and has the capacity to reuse whatever knowledge was learned in the first training run.
thedougd 3y ago
Thanks. It was more of a proof of concept to see what I could do or not do with Q. The product looks promising to an Enterprise as it has connectors for most of our existing document stores. Executives will ask questions about the technologies they hear about in the news, and it's good to be able to quickly speak to the good and the bad.

bradley13 3y ago

Your comment about filters is important. These companies are so worried that someone will generate "evil" results that they disable whole swaths of potential queries. All sorts of different people may genuinely need those results: Attorneys, doctors and nurses, law enforcement, private investigators. Individuals searching for advice. Doing background checks on other people. Heck, just high school students researching a topic. I'm not sure what to do about the (legitimate) concerns of the AI providers. They don't want a shitstorm, because someone got their AI model to [whatever]. But [whatever] can genuinely be important.

wredue 3y ago
They’re not “worried about it”. When filters are removed, one of the first things people do to the bots is make them racists. Time and time again. It happens every time.
- deleted 3y ago
  [deleted]
- Mountain_Skies 3y ago
  There are many examples of people testing LLMs on which is worse, extinguishing all of humanity or making a racist comment in a place where no one will hear it. The LLMs keep choosing to extinguish humanity. Just something to consider when using "but it could say something racist" as justification for filtering. Is racism so evil that it's better to kill every human to avoid it ever happening? Maybe the only option in that case is to prohibit the existence of LLMs. We already have corporations in the role of powerful sociopaths preying on society, do we need LLMs looking to kill us all to satisfy its goals too?
  - NoraCodes 3y ago
    > Is racism so evil that it's better to kill every human to avoid it ever happening? The companies that make and operate these systems - like most companies - are more interested in avoiding PR problems than avoiding harm to humans. This is just one somewhat humorous example of that general policy.

Jenk 3y ago

The HR bot use case is very close to my domain - literally one of the teams reporting to me are maintaining the HR bot for a large enterprise - and it is a legal minefield. Whatever the bot provides as a response could be legally binding, according to legal. Examples of the kind of prompts we get in a many-thousands sized org: - What are my p/maternity entitlements? - When will I be promoted? - What will the bonus pool be this year? - What will happen to me if my boss finds out I have been slacking off? and many more examples of prompts that HR/legal simply will not accept a vague, or worse: incorrect, response to. Each of those types of questions require specific and considerate knowledge that LLMs simply cannot produce.

deleted 3y ago
[deleted]
thedougd 3y ago
You've sufficiently talked me out of it.
dfxm12 3y ago
Whatever the bot provides as a response could be legally binding, according to legal. I wonder if it could be legally binding according to any actual law though.
- lesuorac 3y ago
  I mean if you ask HR (bot) how many weeks of paternity leave you have and it says 20 and so you disappear for 20 weeks and get fired after 8 because it actually ran out are you really in the wrong? While there are laws about minimum time in some states; the state is probably happy to enforce whatever extra time you grant. So if you tell your employees 20 they're probably going to be happy to hold you to 20.
  - jandrese 3y ago
    Or the bot responds with 2 weeks which is below the legal minimum for your state and the employee sues.

throwup238 3y ago

> Filtering will kill the utility of these products. Something like a legal practice is messy business that the filter won’t like. I’ve been trying to apply all the big name LLMs to analyze court opinions from the Caselaw project and holy mother of god, it’s all but useless in the most ridiculous way. Criminal cases are almost universally guaranteed to trip up at least one of their safety filters. There’s some truly awful stuff in there… which is kinda the point of having an LLM read this stuff.

qingcharles 3y ago
Yeah, almost every criminal case law causes it to think you are about to commit the crime itself and instead gives you another boring moral lecture about what an awful human you are. That and it hallucinates me a bunch of new case cites :D

ren_engineer 3y ago

>I asked it a few simple fact based questions, such as who our CEO is. It answered with our previous CEO based on an old press release. this is the problem with basic RAG implementations that most products are currently using, simple vector search isn't able to handle queries like this. The solution is enhancing the base query and also using structured data to filter on metadata here's a good article covering some strategies - https://jxnl.github.io/instructor/blog/2023/09/17/rag-is-more-than-just-embedding-search/ https://jxnl.github.io/instructor/blog/2023/09/17/rag-is-mor...

qingcharles 3y ago

Try Claude? I've been through a lot of LLMs and Claude is really the best with larger documents.

gentleman11 3y ago

Same when using it as a writing aid: the censorship prevents it from answering two thirds the questions you want to ask about, making it faster to just search for the information yourself in Ddg or brave

hfhdjdks 3y ago

I see a lot of people asking "what year it is" or some other question that tests knowledge lf current events. To me it sounds like a misunderstanding of what an LLM + retrieval augmentation does. If the LLM is only fed the HR documents that have multiple dates, why would you expect it to answer that correctly? Why is it relevant that it answered it correctly.