Yes, every token is expanded into a vector that can be many thousand of dimensions. The vectors are stored for every token and every layer. You absolutely can

Yes, every token is expanded into a vector that can be many thousand of dimensions. The vectors are stored for every token and every layer. You absolutely can not fill even a single research paper in 2 GB much less an entire book.