Usage
Most of your text is longer than around 400 words, and your primary goal is to use the embeddings for retrieval.
Methodology
The chunk and special token average strategy takes the entire text and breaks it into chunks of n tokens, includes the model-specific special tokens for each chunk, averages the embeddings of the tokens in each chunk and then further averages those final embeddings.