| Model Name | Longformer |
| Description | A text transformer model for text up to 4096 tokens, about 3000 words |
| Use For | Long text |
| Limitations | Text longer than 4096 tokens will be truncated |
| Graft Default | No |
Reference information
| Source | allenai/longformer-base-4096 - hugging face |
| Trained on | English Wikipedia and “BookCorpus” |
| Paper | 2004.05150 |
| Embedding Dimension | 768 |