I think one of the things that scares me most about API-accessed LLMs is how powerful they are as data collection tools in their own right. OpenAI, for example,

I think one of the things that scares me most about API-accessed LLMs is how powerful they are as data collection tools in their own right. OpenAI, for example, recently updated their terms of use to be more vague about how they work with the gov and I have no doubts that giving the NSA access to conversational feeds is absolutely a requirement to their continued operation as an entity, a la lavabit. In fact, part of me thinks that the Sam/Ilya drama and sam's god complex are at least partially rooted in this, alleged, collaboration. Imagine the questions you could pose to a GPT trained on all the conversations had with users that's been enriched with their biographical data. These conversations are often intimate and curiosity driven in a way that seeking the truth could easily be framed as self-radicalization.