Not on a windows box right now, so might be slightly off, but iirc, this should be: Control panel > Folder options > Advanced tab > there should be an option c

Not on a windows box right now, so might be slightly off, but iirc, this should be: Control panel > Folder options > Advanced tab > there should be an option called "disable caching thumbnails" or some such.