I get annoyed by a lot of the discourse about LLMs, such debates about how much they are going to revolutionize industry, or whether they are truly intelligent. There are things that we know for sure about their limitations, and things that are not going to change: in particular, they are trained on textual input, and this is not the same as a world model. To put it bluntly, ChatGPT does not know where your keys are.

Sure, you can still ask ChatGPT where your keys are, and it can give you advice about looking for keys, but it doesn’t actually know where they are, it can just give outputs that are statistically likely based on its training input. If you want an LLM to be able to tell you where your keys are, you can tell it where you last put it, and as long as it’s in the context window it can spit that information back at you. But if you move your keys and don’t tell the model that, it won’t know the new location. Perhaps you could put a location tracker on your keys and feed the new location into an LLM, but at that point it’s redundant: you can just look at the location tracking app on your phone.

A lot of the debates about the economic effects of LLMs follow these same patterns: they’re the same broad debates about automation in general, not about LLMs in particular.