4 ms·
Greater context size means more computational resources means more energy. Dedicating a portion of the context to telling the LLM not to refer to goblins then h
by boomlinde 6mo ago
Greater context size means more computational resources means more energy. Dedicating a portion of the context to telling the LLM not to refer to goblins then has a non-zero energy cost every time you prompt the model.