4 ms·

According to ChatGPT: > 1. Start by training a machine learning model to identify features of natural language. You can use supervised learning techniques with

According to ChatGPT: > 1. Start by training a machine learning model to identify features of natural language. You can use supervised learning techniques with a corpus of labeled data consisting of texts written by both humans and AI systems. > 2. Use natural language processing techniques to extract features from the text such as word frequencies, grammar, and syntax. > 3. Implement a scoring system to assign a numerical score to each text based on the features identified by the model. > 4. Create a threshold score which will be used to differentiate between texts written by humans and AI systems. > 5. Test the model on a separate dataset to ensure accuracy. > 6. Deploy the model in a production environment to detect AI-written texts. This is from the prompt: "How would you build a tool to detect texts written by artificial intelligence systems?" It seems like an okay plan, but doesn't offer any actual insight into differences between human text and AI text. For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4".

amanzi 4y ago
I also asked ChatGPT a similar question to which it started its response with: "To train an AI to detect text written about another AI, you would need to use a technique called supervised learning. In supervised learning, you would provide the AI with a large dataset of text, some of which is written about AI and some of which is not. The AI would then learn to identify the characteristics of text written about AI and use those characteristics to make predictions on new, unseen text." So I followed up with another question: "Did an AI write the following paragraph?" and then I pasted the above paragraph of text to follow. ChatGPT responded with: "No, a human wrote the paragraph you provided. It is possible that an AI could be trained to generate text that is similar to the paragraph you provided, but it did not write the paragraph itself." Oh oh...
joshuamcginnis 4y ago
I suspect that in time, this will only accelerate the degree to which AI and human-authored text are indistinguishable from each other.
- hxugufjfjf 4y ago
  I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself. I asked OpenAI and it said: It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be unethical. However, if the text is being used to provide helpful information or assistance to customers, then it may not be considered unethical.
  - bilsbie 4y ago
    People have been coping and pasting text into emails for millennia without explicitly telling the recipient. This shouldn’t be any different.
    - hxugufjfjf 4y ago
      I guess its not any worse than the millions of lines of code in production mindlessly copypasted from SO.
    - xdennis 4y ago
      But this isn't about copying. It's about sending texts authored by non-people. It's a recent thing which does require a different approach.
      - 9wzYQbTYsAIc 4y ago
        The business sending the email would already be liable for anything actionable that arose from the generated content, so traditional human-in-the-loop verification and validation might be all that is needed.
        9wzYQbTYsAIc 4y ago
        According to ChatGPT: “Whether or not it is ethical to use an AI to assist with your communications depends on several factors, including the specific context in which you are using the AI and the potential consequences of using it. In general, it is important to be transparent about the fact that you are using an AI to assist with your communications, and to make sure that the AI is not being used to deceive or harm others. It is also important to consider the potential impact on the people you are communicating with, and to ensure that the AI is not being used in a way that infringes on their rights or autonomy. Ultimately, the ethics of using an AI to assist with your communications will depend on the specific circumstances and how you use the technology.”
ad404b8a372f2b9 4y ago
That's a very verbose way of saying TF-IDF + SVM.
nonameking2026 4y ago
I think you should begin with specifying the purpose of such a detector, apart from the obvious one that you want to know that you are talking to a real human being.
FridgeSeal 4y ago
Whilst technically a valid approach, I feel like so much content from these systems is just...vacuously correct? Like, yep, that's a process to train a machine-learning model. Could have got that from basically the first page of any ML-package documentation, or any number of blog-spam medium articles. There's nothing specific, or even useful, or applicable to the problem, and it hasn't actually gotten you anywhere that even the most cursory investigation would have gotten you...
jokethrowaway 4y ago
> For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4". That's exactly how I approach anything and I'm 80% sure I'm not an AI (at least, not so unsophisticated)

amanzi 4y ago

I also asked ChatGPT a similar question to which it started its response with: "To train an AI to detect text written about another AI, you would need to use a technique called supervised learning. In supervised learning, you would provide the AI with a large dataset of text, some of which is written about AI and some of which is not. The AI would then learn to identify the characteristics of text written about AI and use those characteristics to make predictions on new, unseen text." So I followed up with another question: "Did an AI write the following paragraph?" and then I pasted the above paragraph of text to follow. ChatGPT responded with: "No, a human wrote the paragraph you provided. It is possible that an AI could be trained to generate text that is similar to the paragraph you provided, but it did not write the paragraph itself." Oh oh...

joshuamcginnis 4y ago

I suspect that in time, this will only accelerate the degree to which AI and human-authored text are indistinguishable from each other.

hxugufjfjf 4y ago
I've already sent text to customers 100% written by AI. Ethically dubious in a commercial setting perhaps, but higher quality text than I would be able to produce myself. I asked OpenAI and it said: It is not necessarily unethical to send customers text generated by AI, but it depends on the context and the specific situation. For example, if the text is being used to deceive or mislead customers, then it would be unethical. However, if the text is being used to provide helpful information or assistance to customers, then it may not be considered unethical.
- bilsbie 4y ago
  People have been coping and pasting text into emails for millennia without explicitly telling the recipient. This shouldn’t be any different.
  - hxugufjfjf 4y ago
    I guess its not any worse than the millions of lines of code in production mindlessly copypasted from SO.
  - xdennis 4y ago
    But this isn't about copying. It's about sending texts authored by non-people. It's a recent thing which does require a different approach.
    - 9wzYQbTYsAIc 4y ago
      The business sending the email would already be liable for anything actionable that arose from the generated content, so traditional human-in-the-loop verification and validation might be all that is needed.
      - 9wzYQbTYsAIc 4y ago
        According to ChatGPT: “Whether or not it is ethical to use an AI to assist with your communications depends on several factors, including the specific context in which you are using the AI and the potential consequences of using it. In general, it is important to be transparent about the fact that you are using an AI to assist with your communications, and to make sure that the AI is not being used to deceive or harm others. It is also important to consider the potential impact on the people you are communicating with, and to ensure that the AI is not being used in a way that infringes on their rights or autonomy. Ultimately, the ethics of using an AI to assist with your communications will depend on the specific circumstances and how you use the technology.”

ad404b8a372f2b9 4y ago

That's a very verbose way of saying TF-IDF + SVM.

nonameking2026 4y ago

I think you should begin with specifying the purpose of such a detector, apart from the obvious one that you want to know that you are talking to a real human being.

FridgeSeal 4y ago

Whilst technically a valid approach, I feel like so much content from these systems is just...vacuously correct? Like, yep, that's a process to train a machine-learning model. Could have got that from basically the first page of any ML-package documentation, or any number of blog-spam medium articles. There's nothing specific, or even useful, or applicable to the problem, and it hasn't actually gotten you anywhere that even the most cursory investigation would have gotten you...

jokethrowaway 4y ago

> For example, a frequent feature I've observed (of course, I'm not alone) is that it tends to generate lists of things and tie them together with "Firstly, item1. Additionally, item2. More so, item3. In conclusion, item4". That's exactly how I approach anything and I'm 80% sure I'm not an AI (at least, not so unsophisticated)