For your first question, the LLM someone built in Minecraft can handle simple conversations with 5 million weights, mostly 8 bits. I doubt it would be able to

For your first question, the LLM someone built in Minecraft can handle simple conversations with 5 million weights, mostly 8 bits. I doubt it would be able to make good use of a large context window, though.