4 ms·

Wiretapping vast amounts of data on the internet is quite cool, but actually sifting through all that data is the really difficult part. Right now intelligence

by samus 3y ago

Wiretapping vast amounts of data on the internet is quite cool, but actually sifting through all that data is the really difficult part. Right now intelligence services are probably looking at lots of false positives and lots of dots they can't connect because the evidence is just too dispersed for a human or a current-generation system to make sense. LLMs could enable them to make the analysis more targeted and effective. But for all we know intelligence services could be using LLMs for years now, since they are usually a few years ahead of everybody else in many regards :-)

jcgrillo 3y ago
> LLMs could enable them to make the analysis more targeted and effective. How? I'm not trying to be combative, I genuinely am curious if you have an idea how these things could be usefully applied to that problem. In my experience working in the information security space, approximate techniques (neural nets, etc.) haven't gotten much traction. Deterministic detection rules are how we approach the problem of finding the needle in the hay pile. So if you have a concrete idea here that could represent an advancement in this field.
- pixl97 3y ago
  I guess my next question is how many needles do you find and how sharp are they? Detection rules would filter out most of the noise, then something like an LLM would do a post filter for intent analysis to rank relative risks for human intelligence to look at.
  - jcgrillo 3y ago
    I suspect this would disincentivize operators to take care in the way they write their detection rules, and the nondeterminism of the LLM would then result in false negatives. So the rate of growth of the needles set would increase, and the analysts would be getting lower quality information mediated by the LLM. In a world where false negatives--i.e. failing to detect a sharp needle--are the worst possible failure mode, approximations need to be handled with exceeding care.
sanderjd 3y ago
This is not the new capability that LLMs have pioneered. It's true that it is difficult to sift out signal from the noise of a vast data trove, but it is difficult in a way that people have been getting extremely good at since the late 90s. What you're describing is a Google-level capability, and that's truly a very complex thing not to be downplayed. But it's a capability that we've had and been honing for decades now. I'm sure language models and transformer techniques will be (or more likely: already are) an important part of the contemporary systems that do this stuff. But I'm skeptical that they care much about GPT-4 itself (or other general models). I'm not skeptical about whether they think it is useful and an important capability to incorporate ML techniques into their systems, I'm unsure how much utility they see in general (the "G" in AGI) models.

by samus 3y ago

jcgrillo 3y ago

> LLMs could enable them to make the analysis more targeted and effective. How? I'm not trying to be combative, I genuinely am curious if you have an idea how these things could be usefully applied to that problem. In my experience working in the information security space, approximate techniques (neural nets, etc.) haven't gotten much traction. Deterministic detection rules are how we approach the problem of finding the needle in the hay pile. So if you have a concrete idea here that could represent an advancement in this field.

pixl97 3y ago
I guess my next question is how many needles do you find and how sharp are they? Detection rules would filter out most of the noise, then something like an LLM would do a post filter for intent analysis to rank relative risks for human intelligence to look at.
- jcgrillo 3y ago
  I suspect this would disincentivize operators to take care in the way they write their detection rules, and the nondeterminism of the LLM would then result in false negatives. So the rate of growth of the needles set would increase, and the analysts would be getting lower quality information mediated by the LLM. In a world where false negatives--i.e. failing to detect a sharp needle--are the worst possible failure mode, approximations need to be handled with exceeding care.

sanderjd 3y ago

This is not the new capability that LLMs have pioneered. It's true that it is difficult to sift out signal from the noise of a vast data trove, but it is difficult in a way that people have been getting extremely good at since the late 90s. What you're describing is a Google-level capability, and that's truly a very complex thing not to be downplayed. But it's a capability that we've had and been honing for decades now. I'm sure language models and transformer techniques will be (or more likely: already are) an important part of the contemporary systems that do this stuff. But I'm skeptical that they care much about GPT-4 itself (or other general models). I'm not skeptical about whether they think it is useful and an important capability to incorporate ML techniques into their systems, I'm unsure how much utility they see in general (the "G" in AGI) models.