I completely understand what you're saying, but I think the problem you run into when trying to make too hard a dividing line is that it's very hard to describe

I completely understand what you're saying, but I think the problem you run into when trying to make too hard a dividing line is that it's very hard to describe precisely where "language" stops and where "culture" begins. For example, in the sentence "I think you should do that", with an emphasis on the "you", what is the grammatical role of that emphasis? It's not necessarily something you'll find explicitly in an English textbook, but it is conveying important meaning here. It's also cultural - I will use different forms of emphasis of I'm from the US or from the UK. In the US I might speak louder for emphasis, in the UK I might speak more slowly for emphasis, or use a different word choice. But what happens if I'm culturally British, in the UK, a native English speaker, but speaking a sign language like BSL? Well then I'm going to use a different form of emphasis again, because BSL has its own ways of indicating emphasis. So clearly emphasis is also related to the language one speaks - my culture hasn't changed, after all! This is the issue with splitting language to into a cultural part and a linguistic part - too much of language straddles both sides. You might say that the core of a language is its grammar, but Scots English and AAVE are both grammatically different from Standard British English, and yet the differences there are as much cultural as purely linguistic. In principle, I get what you mean - you can learn all the words and the grammar of a language from a textbook without knowing anything about the culture behind that language. But if you can't use the language to communicate fluently with native speakers of that language, have you really learned it? Your time traveling Roman asking for the portal rather than the door has basically just invented his own dialect, not learned English to any meaningful degree.