It's worth mentioning that there are embeddings inside the model (e.g. what vector does each token correspond to / nn.Embedding) and the "embedding" of an input

It's worth mentioning that there are embeddings inside the model (e.g. what vector does each token correspond to / nn.Embedding) and the "embedding" of an input, which are used in the same context. They might not even live in the same space (so are not comparable) but the former are learned during training, and the latter are computed during inference. For example, take a sentence like "The quick brown fox jumps over the lazy dog". A tokenizer turns that into "[The][<S>][qui][ck][<S>][br][own][<S>]..." which corresponds to a set of indices in a lookup table: "12,0,653,34,0,...". Those are then looked up in the model embedding (creating e.g. 300-dimensional vectors for each token). The model is run over that output, creating a 500-dimensional vector for the sentence (for example, if the model is an nn.Linear(300, 500)). Why would we want vector databases for this? Suppose the sentence is actually a help text question. The final 500-dimensional vector for the question and the correspond text of the answer could be stored in the database. When a new user asks a new question, you can run the model to create a new 500-dimensional vector, find the nearest answered-question vector, then return the corresponding answer text. So to answer your questions: (1) Yes, the model embeddings (300 dim above), but not all "embeddings" (500 above). (2) Yes (in both cases). (3) Yes, people usually define a similarity measure (e.g. cosine similarity as mentioned by siblings) and train the model such that if texts X, Y are similar (X ~ Y) as defined by humans, then f(X) ~ f(Y) (where f is the neural network).