The Chat settings menu allows the user to determine the settings applied to any chat within the project.
Chat settings are found under the Chat mode side menu, accessed in the bottom left hand corner of the UI
|
RESPONSE STRATEGY determines whether Chat will generate SQL queries or a text search. The user may select either option or set to Auto where Graft will attempt to determine the best strategy Note: Some response strategies may be unavailable based on the Entities that have been selected.
TEXT SEARCH STRATEGY (when Text Search is selected) Provides the option to switch between keyword (lexical) and semantic search. Auto allows Graft to determine the best option based on the available data |
|
|
|
|
|
CHAT MODEL Allows the switching between available LLMs. by Default Graft uses Chat GPT 3.5 Turbo with a larger token count (to allow for the use of larger documents). A user may switch to another model. The only models listed are those the user has API keys for, adding new API keys under Security will display further models. |
|
|
RELEVANCE THRESHOLD When Graft determines that a text search should be conducted (or when a text search is manually enforced) we using text reranking to determine the relevance of retrieved citations to the user message. The relevance threshold is the threshold below which we deem citations “irrelevant” and should therefore not be passed to the LLM for response synthesis. 0 (Zero) allows all results to be sent to the LLM |
|
|
CITATION PROMPT When text search brings back a number of relevant citations we have to send them to the LLM along with the user message and ask for an appropriate response. This is handled by the citation prompt, our default citation prompt is showing the screenshot There are two required substrings for the citation prompt
|
|