Typically, when people say it lacks real understanding, they seem to mean it lacks an understanding of any object’s semantic meaning. In the above example, the

Typically, when people say it lacks real understanding, they seem to mean it lacks an understanding of any object’s semantic meaning. In the above example, the person asked for an example of X API solving problem Y, and X lacked the functionality needed to solve Y. If the AI understood the formal specifications of the API, it would be able to know that functionality doesn’t exist and reply as such. However, the AI is just a next token predictor here, it only knows certain words tend to follow others. Thus if you never told it that X API can’t do Y in training, it’s unlikely to reply as such. Instead it might give you a fictional function call in the style of X API to solve your query.