6 ms·

I'm really confused what is going on here with people being so upset. It doesn't do anything unless you provide an OpenAI key. Nothing is sent anywhere unless y

by mdeeks 2y ago

I'm really confused what is going on here with people being so upset. It doesn't do anything unless you provide an OpenAI key. Nothing is sent anywhere unless you actually run the "Engage Artificial Intelligence" command. It's roughly the same thing as asking ChatGPT, but its right in your terminal app. It isn't always on or anything. It's actually kind of hard to find and use. Is there maybe something I'm missing? Or is this a general backlash against AI?

tflol 2y ago
yeah youre missing that this suggests all usage is suddenly being used for training. the incentive for the company is avoiding suffering the opportunity cost of not using user data for training. the aggression of this post is weirder than the aggression of the opposition. edit: several rephrases of the middle sentence
- mdeeks 2y ago
  If it does do that then yeah, there should be backlash. But I don't see that stated anywhere in the notes nor do I see a hint of that in the app. The only two AI interfaces I see are the "Engage Artificial Intelligence" menu item where you ask it how to form a command, and then a second interface in the Toolbelt called Codecierge where it tries to complete a task you tell it to do. Both of which are completely optional and require an OpenAI key. If you do use them you're sending your request to OpenAI but as far as I can tell it doesn't get any of your command history or anything like that.
  - tflol 2y ago
    > If it does do that then yeah, there should be backlash. It's a threat. Disrespectful to users. Maybe it does, maybe it doesn't! If it doesn't maybe it will someday! When just the right person gets hired into exec.
  - upon_drumhead 2y ago
    The API key is unvalidated and so any entry that isn't exactly a blank string is treated like a valid key and data is sent out. There's a risk there that a future version may handle the blank key incorrectly or differently and start transmitting away. For some people, the possibility of a string handling bug or a filesystem corruption or what not is enough of a risk to avoid the software.
    - lurkersince2013 2y ago
      If you're so paranoid about this possibility then just block it at the network level - heck, even whitelist IP ranges that you want iTerm2 to be able to connect out to and default deny the rest. Let's not blame iTerm2, which is a highly useful, amazing piece of software that many of us have relied on on every day for years and have never paid anything to use. If you really are so worried about the AI feature being part of iTerm2 then go fork it and strip the AI functionality yourself: https://github.com/gnachman/iTerm2 https://github.com/gnachman/iTerm2
stalfosknight 2y ago
I think people are just exhausted with the non-stop hype/frothing over AI and how hallucinating stochastic parrots have to now be stuffed into anything and everything, fuck all if it makes sense or if it’s even safe to do so. It just feels like every company / developer is racing tripping over themselves to scream “ME TOO!” with AI. I prefer Apple’s built-in Terminal but even I’m wincing at this new “feature” being included in iTerm2.
- mdeeks 2y ago
  I agree about the trend to stuff AI in everything. This one feels legitimately useful though? One hot key away from asking how to write a command in plain english is kinda nice. Honestly iTerm has always been loaded with features that of dubious usefulness. They all tend to be useful to someone though. This is just another one on the list. I think the biggest failure here was the way it was communicated. Extremely ambiguous about WHAT it actually does and any privacy concerns.
  - stalfosknight 2y ago
    I’m not anti-AI in general. I think there it has immense potential for benefiting humankind just as there is immense potential for harm and misuse, much like nuclear power. I just can’t recall any other product or tech that has so many executive types and techbros frothing so intensely at the mouth and desperate to shoehorn it into everything like this.
elicksaur 2y ago
Some of the comments note that just the existence of the feature means they can’t use the app at their workplace. If this was their preferred terminal, that would be pretty frustrating.
kasey_junk 2y ago
There are organizations that have very hard lines about the systems they allow. One of those lines that has recently been added in many places is “can’t integrate with ai systems”. Whether that implies it’s ok to have a toggle or not is frequently not up to technologists it’s up to compliance folk who may have no interest or incentive to look deeper. And not for nothing it’s a weird coupling. I can’t think of a single reason I’d integrate with an ai agent in this way over a more unixy approach.
- mdeeks 2y ago
  > And not for nothing it’s a weird coupling. I can’t think of a single reason I’d integrate with an ai agent in this way over a more unixy approach. The AI Agent part is a bit weird, but the "Engage AI" feature is pretty nice. Just hit "cmd+y" and type something you want to do like "Clone without blobs", or "get the last two fields from a CSV", and it shows you the command which you can then run with "shift+enter"
- 542458 2y ago
  > can’t integrate with ai systems Do these locations have no devices running Windows, OSX, or iOS?
  - upon_drumhead 2y ago
    No, our production datacenters don't have Windows, OSX or iOS. We have security ensuring no external devices are brought inside by techs.
  - rsynnott 2y ago
    If you’re thinking of Siri and friends, those would generally be force-disabled, and those operating systems (at least MacOS and iOS, who knows with Windows these days) offer fairly robust mechanisms to do that.
404mm 2y ago
I think there should be a non-AI build so companies can allow this specific flavor and not ban iTerm2 completely. There is always going to be somebody using it when they shouldn’t.
- 542458 2y ago
  You don’t need a separate build. This has the exact same risk profile as somebody just navigating to the openAI website, and can be blocked in the exact same way, with network policy. I could see the argument that the iterm needs a more clear way to explicitly disable these features (rather than not configuring them) but I don’t think a separate build solves anything.
- mullingitover 2y ago
  You can make calls to OpenAI's API with curl, too. Should we ask for linux distros without curl, to prevent employees from accessing AI that way? If companies are that worried about rogue employee access to forbidden AI APIs, the network layer seems like a more appropriate place for those blocks.
upon_drumhead 2y ago
FWIW, it being un-configured is a dealbreaker in my company. We just received word that iTerm is now unapproved for usage, and that impacts all historic versions as well. It's overkill, but that's the stance legal is taking on it. People are having to migrate over to Apple Terminal and losing a lot of the environment that made them productive. It's really impacting us. While they're allowed to do whatever they want, and I'm happy for the years of wonderfully useful software they've provided, 3rd party AI features are radioactive for some organizations and the suddenness of this is jarring to impacted folks.
- 542458 2y ago
  So I’m confused to the logic here. You’re running Mac OS, which can call out to all kinds of external services, even from the terminal. Siri is debatably a third party AI feature itself. Why is that OK but this is toxic?
  - upon_drumhead 2y ago
    My terminal contents are not able to be sent out. We do have Siri disabled as part of our MDM profile, and legal feels that our protections are adequate for the os layer. We're extremely limited to what software we can install, and we have os level protections on what endpoints our devices can even communicate with on the wider internet. Yes, it's overkill, but legal's stance is that anything that can possibly talk to a third party AI service is forbidden regardless of the other technical solutions in place preventing information leakage. FWIW, anything that has telemetry to third parties (crash reports, usage stats) are also forbidden, with extremely limited exceptions that legal has approved.
    - rkwasny 2y ago
      Everything you type into spotlight search is send to apple
    - smrtinsert 2y ago
      This must be the fastest legal team in the industry
kelnos 2y ago
I wouldn't want my terminal to even have code that can try to access the network. It doesn't need it, it's superfluous, and bugs in it (or even in the code that determines whether or not it should try to access the network) could open me up to security issues. And I'm not even like other commenters who have strict security needs around their work where things that can access the network need to be vetted before being approved for use. I think some of it is backlash against AI though; you're probably right about that.

by mdeeks 2y ago

tflol 2y ago

yeah youre missing that this suggests all usage is suddenly being used for training. the incentive for the company is avoiding suffering the opportunity cost of not using user data for training. the aggression of this post is weirder than the aggression of the opposition. edit: several rephrases of the middle sentence

mdeeks 2y ago
If it does do that then yeah, there should be backlash. But I don't see that stated anywhere in the notes nor do I see a hint of that in the app. The only two AI interfaces I see are the "Engage Artificial Intelligence" menu item where you ask it how to form a command, and then a second interface in the Toolbelt called Codecierge where it tries to complete a task you tell it to do. Both of which are completely optional and require an OpenAI key. If you do use them you're sending your request to OpenAI but as far as I can tell it doesn't get any of your command history or anything like that.
- tflol 2y ago
  > If it does do that then yeah, there should be backlash. It's a threat. Disrespectful to users. Maybe it does, maybe it doesn't! If it doesn't maybe it will someday! When just the right person gets hired into exec.
- upon_drumhead 2y ago
  The API key is unvalidated and so any entry that isn't exactly a blank string is treated like a valid key and data is sent out. There's a risk there that a future version may handle the blank key incorrectly or differently and start transmitting away. For some people, the possibility of a string handling bug or a filesystem corruption or what not is enough of a risk to avoid the software.
  - lurkersince2013 2y ago
    If you're so paranoid about this possibility then just block it at the network level - heck, even whitelist IP ranges that you want iTerm2 to be able to connect out to and default deny the rest. Let's not blame iTerm2, which is a highly useful, amazing piece of software that many of us have relied on on every day for years and have never paid anything to use. If you really are so worried about the AI feature being part of iTerm2 then go fork it and strip the AI functionality yourself: https://github.com/gnachman/iTerm2 https://github.com/gnachman/iTerm2

stalfosknight 2y ago

I think people are just exhausted with the non-stop hype/frothing over AI and how hallucinating stochastic parrots have to now be stuffed into anything and everything, fuck all if it makes sense or if it’s even safe to do so. It just feels like every company / developer is racing tripping over themselves to scream “ME TOO!” with AI. I prefer Apple’s built-in Terminal but even I’m wincing at this new “feature” being included in iTerm2.

mdeeks 2y ago
I agree about the trend to stuff AI in everything. This one feels legitimately useful though? One hot key away from asking how to write a command in plain english is kinda nice. Honestly iTerm has always been loaded with features that of dubious usefulness. They all tend to be useful to someone though. This is just another one on the list. I think the biggest failure here was the way it was communicated. Extremely ambiguous about WHAT it actually does and any privacy concerns.
- stalfosknight 2y ago
  I’m not anti-AI in general. I think there it has immense potential for benefiting humankind just as there is immense potential for harm and misuse, much like nuclear power. I just can’t recall any other product or tech that has so many executive types and techbros frothing so intensely at the mouth and desperate to shoehorn it into everything like this.

elicksaur 2y ago

Some of the comments note that just the existence of the feature means they can’t use the app at their workplace. If this was their preferred terminal, that would be pretty frustrating.

kasey_junk 2y ago

There are organizations that have very hard lines about the systems they allow. One of those lines that has recently been added in many places is “can’t integrate with ai systems”. Whether that implies it’s ok to have a toggle or not is frequently not up to technologists it’s up to compliance folk who may have no interest or incentive to look deeper. And not for nothing it’s a weird coupling. I can’t think of a single reason I’d integrate with an ai agent in this way over a more unixy approach.

mdeeks 2y ago
> And not for nothing it’s a weird coupling. I can’t think of a single reason I’d integrate with an ai agent in this way over a more unixy approach. The AI Agent part is a bit weird, but the "Engage AI" feature is pretty nice. Just hit "cmd+y" and type something you want to do like "Clone without blobs", or "get the last two fields from a CSV", and it shows you the command which you can then run with "shift+enter"
542458 2y ago
> can’t integrate with ai systems Do these locations have no devices running Windows, OSX, or iOS?
- upon_drumhead 2y ago
  No, our production datacenters don't have Windows, OSX or iOS. We have security ensuring no external devices are brought inside by techs.
- rsynnott 2y ago
  If you’re thinking of Siri and friends, those would generally be force-disabled, and those operating systems (at least MacOS and iOS, who knows with Windows these days) offer fairly robust mechanisms to do that.

404mm 2y ago

I think there should be a non-AI build so companies can allow this specific flavor and not ban iTerm2 completely. There is always going to be somebody using it when they shouldn’t.

542458 2y ago
You don’t need a separate build. This has the exact same risk profile as somebody just navigating to the openAI website, and can be blocked in the exact same way, with network policy. I could see the argument that the iterm needs a more clear way to explicitly disable these features (rather than not configuring them) but I don’t think a separate build solves anything.
mullingitover 2y ago
You can make calls to OpenAI's API with curl, too. Should we ask for linux distros without curl, to prevent employees from accessing AI that way? If companies are that worried about rogue employee access to forbidden AI APIs, the network layer seems like a more appropriate place for those blocks.

upon_drumhead 2y ago

FWIW, it being un-configured is a dealbreaker in my company. We just received word that iTerm is now unapproved for usage, and that impacts all historic versions as well. It's overkill, but that's the stance legal is taking on it. People are having to migrate over to Apple Terminal and losing a lot of the environment that made them productive. It's really impacting us. While they're allowed to do whatever they want, and I'm happy for the years of wonderfully useful software they've provided, 3rd party AI features are radioactive for some organizations and the suddenness of this is jarring to impacted folks.

542458 2y ago
So I’m confused to the logic here. You’re running Mac OS, which can call out to all kinds of external services, even from the terminal. Siri is debatably a third party AI feature itself. Why is that OK but this is toxic?
- upon_drumhead 2y ago
  My terminal contents are not able to be sent out. We do have Siri disabled as part of our MDM profile, and legal feels that our protections are adequate for the os layer. We're extremely limited to what software we can install, and we have os level protections on what endpoints our devices can even communicate with on the wider internet. Yes, it's overkill, but legal's stance is that anything that can possibly talk to a third party AI service is forbidden regardless of the other technical solutions in place preventing information leakage. FWIW, anything that has telemetry to third parties (crash reports, usage stats) are also forbidden, with extremely limited exceptions that legal has approved.
  - rkwasny 2y ago
    Everything you type into spotlight search is send to apple
  - smrtinsert 2y ago
    This must be the fastest legal team in the industry

kelnos 2y ago

I wouldn't want my terminal to even have code that can try to access the network. It doesn't need it, it's superfluous, and bugs in it (or even in the code that determines whether or not it should try to access the network) could open me up to security issues. And I'm not even like other commenters who have strict security needs around their work where things that can access the network need to be vetted before being approved for use. I think some of it is backlash against AI though; you're probably right about that.