Persian support¶
Persian is a first-class language in llm-search-eval, next to English.
- Arabic
يandكare unified with Persianیandک; Persian and Arabic-Indic digits become ASCII digits. - Diacritics, tatweel, and directional marks are removed, and the zero width non-joiner is treated as a word boundary.
- Query language is detected from the script when
langis missing, and reports break down the primary metric by language. - The LLM judge prompt states the query language and counts translations and transliterations between Persian and English as matches.