Usage
Most of your text is short, less than around 400 words, and your primary goal is to perform similarity or semantic searches on the text
Methodology
The token average strategy takes the first n tokens of the text, extracts their individual embeddings and averages them to make a single embedding for the input text.