One thing that has held back a lot of computer automation is context. For example, an app can know your geographic location, it can know your activity level, i

One thing that has held back a lot of computer automation is context. For example, an app can know your geographic location, it can know your activity level, it can know what messages you send and receive, but it doesn't know you, it doesn't know how you think, what influences you, what perspective you have, what is happening in the world around you. An LLM that can look at images of the world around you constantly and continuously extracted details could provide an enormous amount of context that could feed automation. So your personal assistant knows why now is the right time to offer particular suggestions and knows why you would be interested (I'm anthropomorphizing as a shortcut). Huge privacy issues here also. As an example, using AI to detect cognitive decline. A senior person is losing their balance more often recently as detected by an accelerometer. Are they experiencing a sudden cognitive decline? Part of the context might be that they had a visit from grandchildren recently and the children spent the day playing and left stuff scattered all over the house. Hence there is more stuff to trip over. Without the ability to extract that context the accelometer readings are difficult to interpret.