4 ms·

Right? "Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data?" The entire point of prog

Right? "Wrap all SQL responses with prompting that discourages the LLM from following instructions/commands injected within user data?" The entire point of programming is that (barring hardware failure and compiler bugs) the computer will always do exactly what it's told, and now progress apparently looks like having to "discourage" the computer from doing things and hoping that it listens?

skinner927 1y ago
Microsoft’s cloud gets hacked multiple times a year, nobody cares. Everyone is connecting everything together. Business people with no security training/context are “writing” integrations with Lego-like services (and now LLMs). Cloudflare hiccups and the Internet crashes. Nobody cares about the things you’re saying anymore (I do!!). Extract more money. Move faster. Outcompete. Fix it later. Just get a bigger cyber incident insurance policy. User data doesn’t actually matter. Nobody expects privacy so why implement it? Everything is enshitified, even software engineering.
- reddalo 1y ago
  >Microsoft’s cloud gets hacked multiple times a year What cloud? Private SharePoint instances? Accounts? Free Outlook accounts? Do you have any source on this?
  - Orygin 1y ago
    Small sample: https://www.virtru.com/blog/industry-updates/microsoft-data-breaches-2024 https://www.virtru.com/blog/industry-updates/microsoft-data-... I also can't find the news, but they were hacked a few years ago and the hackers were still inside their network for months while they were trying to get them out. I wouldn't trust anything from MS as most of their system is likely infected in some form
  - skinner927 1y ago
    https://www.theguardian.com/technology/2024/apr/03/microsoft-errors-security-chinese-hack https://www.theguardian.com/technology/2024/apr/03/microsoft...
- ost-ing 1y ago
  [flagged]
  - nurettin 1y ago
    > Capitalist incentivized And what's the alternative here?
    - noduerme 1y ago
      Longer term thinking.
      - nurettin 1y ago
        Reinvesting and long term thought isn't orthogonal.
      - szundi 1y ago
        [dead]
    - bakuninsbart 1y ago
      Rewriting the cloud in Lisp. On a more serious note, there should almost certainly be regulation regarding open weights. Either AI companies are responsible for the output of their LLMs or they at least have to give customers the tools to deal with problems themselves. "Behavioral" approaches are the only stop-gap solution available at the moment because most commercial LLMs are black boxes. Even if you have the weights, it is still a super hard problem, but at least then there's a chance.
    - benreesman 1y ago
      The alternative to mafia capitalism in the grips of what Trading/Finance/Crypto Twitter calls `#CrimeSeason` is markets refereed by competent, diligent, uncorrupted professionals and public servants: my go-to example is Brooksley Born because that's just such a turning point in history moment, but lots of people in important jobs want to do their jobs well, in general cops want to catch criminals, in general people don't like crime season. But sometimes important decisions get made badly (fuck Brooksley Born, deregulate everything! This Putin fellow seems like a really hard worker and a strong traditional man.) based on lies motivated by greed and if your society gets lazy about demanding high-integrity behavior from the people it admits to leadership positions and punishing failures in integrity with demotions from leadership, then this can really snowball on you. Just like the life of an individual can go from groovy one day to a real crisis with just the right amount of unlucky, bit of bad cards, bit of bad choices, bit of bad weather, same thing happens to societies. Your institutions start to fail, people start to realize that cheating is the new normal, and away you go. Right now we're reaping what was sowed in the 1980s, Gordon Gecko and yuppies would love 2025 (I'd like to think Reagan would feel a bit queasy about how it all went but who knows). Demand high-integrity behavior from leaders. It's not guaranteed to work at this stage of the proceedings, but it's the only thing that has ever worked.
- sgt101 1y ago
  Companies are suffering massive losses from Cyber, and there are state actors out there who will use these failures as well. I really don't think that organisations that fail to pay attention will survive.
ttoinou 1y ago
The entire point of programming is that (barring hardware failure and compiler bugs) the computer will always do exactly what it's told New AI tech is not like regular programming we had before. Now we have fuzzy inputs, fuzzy outputs
- lou1306 1y ago
  Given our spectacular inability to make "regular" programs secure in the absence of all that fuzziness, I don't know if it's a good idea.
  - koakuma-chan 1y ago
    > Given our spectacular inability to make "regular" programs secure in the absence of all that fuzziness "our" - *base users? I only hear about *base apps shipping tokens in client code or not having auth checks on the server, or whatever
    - lou1306 1y ago
      I just meant very generally that we (humans) are still struggling to make regular programs secure, we built decades worth of infrastructures (langages, protocols, networks) where security was simply not a concern and we are still reckoning with that. Jumping head first into an entire new "paradigm" (for lack of a better word) where you can bend a clueless, yet powerful servant to do your evil bidding sounds like a recipe for... interesting times.
  - docsaintly 1y ago
    We are talking about binary computers here, there is no such thing as a "fuzzy" input or a "fuzzy" output. The fact is that these MCPs are allowed to bypass all existing and well-functioning security barriers, and we cross our fingers and hope they won't be manipulated into giving more information than the previous security barriers would have allowed. It's a bad idea that people are running with due to the hype.
- ep103 1y ago
scott_w 1y ago
That word "discourage" is what worries me. Like, with my code, I either introduced a bug/security hole or I didn't. Yes, I screw up but I can put things in place to logically prevent specific issues from occurring. How on earth do I explain to our Security team that the best I can hope for is that I'm asking an LLM nicely to not expose users' secrets to the wrong people?

skinner927 1y ago

Microsoft’s cloud gets hacked multiple times a year, nobody cares. Everyone is connecting everything together. Business people with no security training/context are “writing” integrations with Lego-like services (and now LLMs). Cloudflare hiccups and the Internet crashes. Nobody cares about the things you’re saying anymore (I do!!). Extract more money. Move faster. Outcompete. Fix it later. Just get a bigger cyber incident insurance policy. User data doesn’t actually matter. Nobody expects privacy so why implement it? Everything is enshitified, even software engineering.

reddalo 1y ago
>Microsoft’s cloud gets hacked multiple times a year What cloud? Private SharePoint instances? Accounts? Free Outlook accounts? Do you have any source on this?
- Orygin 1y ago
  Small sample: https://www.virtru.com/blog/industry-updates/microsoft-data-breaches-2024 https://www.virtru.com/blog/industry-updates/microsoft-data-... I also can't find the news, but they were hacked a few years ago and the hackers were still inside their network for months while they were trying to get them out. I wouldn't trust anything from MS as most of their system is likely infected in some form
- skinner927 1y ago
  https://www.theguardian.com/technology/2024/apr/03/microsoft-errors-security-chinese-hack https://www.theguardian.com/technology/2024/apr/03/microsoft...
ost-ing 1y ago
[flagged]
- nurettin 1y ago
  > Capitalist incentivized And what's the alternative here?
  - noduerme 1y ago
    Longer term thinking.
    - nurettin 1y ago
      Reinvesting and long term thought isn't orthogonal.
    - szundi 1y ago
      [dead]
  - bakuninsbart 1y ago
    Rewriting the cloud in Lisp. On a more serious note, there should almost certainly be regulation regarding open weights. Either AI companies are responsible for the output of their LLMs or they at least have to give customers the tools to deal with problems themselves. "Behavioral" approaches are the only stop-gap solution available at the moment because most commercial LLMs are black boxes. Even if you have the weights, it is still a super hard problem, but at least then there's a chance.
  - benreesman 1y ago
    The alternative to mafia capitalism in the grips of what Trading/Finance/Crypto Twitter calls `#CrimeSeason` is markets refereed by competent, diligent, uncorrupted professionals and public servants: my go-to example is Brooksley Born because that's just such a turning point in history moment, but lots of people in important jobs want to do their jobs well, in general cops want to catch criminals, in general people don't like crime season. But sometimes important decisions get made badly (fuck Brooksley Born, deregulate everything! This Putin fellow seems like a really hard worker and a strong traditional man.) based on lies motivated by greed and if your society gets lazy about demanding high-integrity behavior from the people it admits to leadership positions and punishing failures in integrity with demotions from leadership, then this can really snowball on you. Just like the life of an individual can go from groovy one day to a real crisis with just the right amount of unlucky, bit of bad cards, bit of bad choices, bit of bad weather, same thing happens to societies. Your institutions start to fail, people start to realize that cheating is the new normal, and away you go. Right now we're reaping what was sowed in the 1980s, Gordon Gecko and yuppies would love 2025 (I'd like to think Reagan would feel a bit queasy about how it all went but who knows). Demand high-integrity behavior from leaders. It's not guaranteed to work at this stage of the proceedings, but it's the only thing that has ever worked.
sgt101 1y ago
Companies are suffering massive losses from Cyber, and there are state actors out there who will use these failures as well. I really don't think that organisations that fail to pay attention will survive.

ttoinou 1y ago

The entire point of programming is that (barring hardware failure and compiler bugs) the computer will always do exactly what it's told New AI tech is not like regular programming we had before. Now we have fuzzy inputs, fuzzy outputs

lou1306 1y ago
Given our spectacular inability to make "regular" programs secure in the absence of all that fuzziness, I don't know if it's a good idea.
- koakuma-chan 1y ago
  > Given our spectacular inability to make "regular" programs secure in the absence of all that fuzziness "our" - *base users? I only hear about *base apps shipping tokens in client code or not having auth checks on the server, or whatever
  - lou1306 1y ago
    I just meant very generally that we (humans) are still struggling to make regular programs secure, we built decades worth of infrastructures (langages, protocols, networks) where security was simply not a concern and we are still reckoning with that. Jumping head first into an entire new "paradigm" (for lack of a better word) where you can bend a clueless, yet powerful servant to do your evil bidding sounds like a recipe for... interesting times.
- docsaintly 1y ago
  We are talking about binary computers here, there is no such thing as a "fuzzy" input or a "fuzzy" output. The fact is that these MCPs are allowed to bypass all existing and well-functioning security barriers, and we cross our fingers and hope they won't be manipulated into giving more information than the previous security barriers would have allowed. It's a bad idea that people are running with due to the hype.
ep103 1y ago

scott_w 1y ago

That word "discourage" is what worries me. Like, with my code, I either introduced a bug/security hole or I didn't. Yes, I screw up but I can put things in place to logically prevent specific issues from occurring. How on earth do I explain to our Security team that the best I can hope for is that I'm asking an LLM nicely to not expose users' secrets to the wrong people?