Skip to content

Marketplace search and ranking

marketplace-search-and-ranking

Index purchasable catalog units and rank them with text relevance, facets, stock, and language-aware matching for Persian and other languages. Use when building marketplace search, filters, relevance, or multilingual product discovery.

Version 0.3.2  ·  MIT  ·  Source  ·  Rule

Install this skill

Copy the skill and its rule into your project:

npx @sadrmanesh/commerce-agent-skills add marketplace-search-and-ranking

The skill goes to .agents/skills/ and the rule to .agents/rules/. See Command line for other directories.

Use this skill for product discovery in a marketplace or catalog. It does not assume a particular search engine. Apply the same document and ranking rules whether the index is a dedicated search service or a database index. Text normalization for Persian and mixed digits lives in scripts/normalize-search-text.mjs. Field and scoring details are in the ranking notes.

Procedure

  1. Index the purchasable unit. A simple product is one document. Each variant is its own document, with parent title, brand, and categories copied onto it. Do not index only the parent when prices and stock differ by variant.
  2. Normalize text the same way at index time and query time. Run normalize-search-text.mjs logic before analysis.
  3. Give every facet a stable id and a localized label. Filter on the id.
  4. Rank with text relevance first, then stock and explicit business rules. Start from the factors in the ranking notes and tune them with labeled queries. Do not treat those factors as measured conversion rates.
  5. Return a keyset (search-after) page, not a deep offset. Offsets skip or repeat documents when the index changes.

Document

Each document needs:

  • unit_id and sku
  • parent_id when the unit is a variant
  • title, brand, and a short description per language you actually search
  • attributes as { code, value_id, label }
  • price and currency of the unit
  • in_stock and backorder as booleans derived from the catalog stock record
  • category_ids
  • updated_at

Do not index a display string that already contains a currency symbol as the price. Keep the number and the currency.

Query

  1. Normalize the query.
  2. If the normalized query equals a SKU or GTIN, return that unit first.
  3. Otherwise run a text query across title, brand, attribute labels, and description, with title weighted above description.
  4. Apply selected facets as filters, not as optional boosts. A shopper who chose "blue" should not see red.
  5. Apply stock policy. Hide out-of-stock units when the shop is configured to hide them. Demote them when the shop still wants them visible.
  6. Page with a stable sort. Add unit_id as a tiebreaker so a page boundary does not shuffle.

Facets

  • Compute facet counts from the current text query and the other selected facets. Do not compute counts from the unfiltered catalog if the UI shows them next to a filtered result set.
  • A facet value with a zero count may be hidden. Do not let the shopper select a value that would empty the result unless you explain that.
  • Multi-select inside one facet is usually OR. Across facets it is AND. Say which one the UI implements and make the query match it.
  • Price facets use the unit price in the shopper's display currency only after conversion. The filter still has to be reproducible from stored prices.

Multilingual and RTL

  • Use a language-specific analyzer. Do not stem Persian with an English stemmer, and do not stem English with a Persian stemmer.
  • Store title_fa and title_en (or one field per language you support) rather than one mixed field.
  • Detect the query language only as a hint. If the shopper is browsing the Persian catalog, search Persian fields first.
  • Normalize Arabic Yeh ي to Persian Yeh ی, Arabic Kaf ك to Persian Kaf ک, Persian and Arabic-Indic digits to ASCII, and remove ZWNJ for matching so می‌روم matches میروم. Keep ZWNJ in the source catalog text. The normalizer does this.
  • Strip bidi marks from indexed and query text. They arrive from copied titles and should not change a match.
  • RTL affects rendering of the search UI, not the score. Use the Persian and RTL skill for layout.

Ranking factors

Use these as a starting point, then change them when labeled queries show a miss:

  • Exact SKU or GTIN match outranks a fuzzy title match.
  • Title matches outrank description-only matches.
  • An in-stock purchasable unit outranks an out-of-stock unit when both match.
  • A newer updated_at can break ties. It should not outrank a clearly better text match.
  • Sponsored placements are labeled as sponsored in the UI and are not mixed into the organic score silently.

Write the formula down in the service that serves results. A boost that exists only in an indexer config and not in review notes will be retuned by accident.

Gotchas

  • Indexing the parent product shows one price and one stock figure for every variant. Shoppers then open a color that is not the price they saw.
  • Updating a variant price without updating the document leaves search and the product page in conflict. Update the document in the same flow as the catalog write, or from an outbox.
  • Synonyms belong in a managed list (tee / t-shirt). Do not add a synonym because one query failed once.
  • Typo tolerance on short queries (two or three characters) matches unrelated brands. Tighten fuzziness for short strings and for SKU fields.
  • Facet labels in Persian still need stable ids. Searching the translated label as if it were the id breaks when the translation changes.

Verification

  • A SKU query returns that unit even if the title uses different digits or Yeh/Kaf forms.
  • A color facet returns only units with that color's value id.
  • Out-of-stock policy matches the shop setting on page 1 and on the next page.
  • An English stemmer is not applied to a Persian title field.
  • A sponsored unit is visibly marked and is not the only explanation for an organic ranking change.

Evaluation

Worked examples and the checks they must satisfy are in evals/tasks.json.

References

Search documents and a starting rank

This is an implementation note for the marketplace search skill. The weights are a default to review with real queries, not a measured optimum.

Field weights

Field Weight Analyzer
sku, gtin Exact only. No stemmer, no typo tolerance above one transposition on long codes. Keyword
title_{lang} 5 Language analyzer after normalizeSearchText
brand 3 Keyword plus a normalized text copy
attribute_labels_{lang} 2 Language analyzer
description_{lang} 1 Language analyzer

Copy parent title and brand onto each variant document so a query for the parent name still finds the purchasable unit. Store variant_label separately so the result can show "Cotton shirt, blue".

Stock factor

After the text score:

  • In stock: multiply by 1.
  • Backorder and the shop shows backorders: multiply by 0.85.
  • Out of stock and the shop still lists it: multiply by 0.25.
  • Out of stock and the shop hides it: drop the document.

Change the multipliers when you have query judgments. Keep the policy of "in stock ahead of out of stock" unless the shop explicitly wants sold-out items promoted.

Tie break

Sort by score descending, then updated_at descending, then unit_id ascending. The final key makes keyset pagination stable.

Keyset page

Request a page with the last seen sort tuple, not with offset. An offset page skips documents when a product is indexed during the session and repeats documents when one is deleted.

What not to put in the text score

  • Margin, unless the shop has agreed that margin may influence organic rank and the influence is documented.
  • Seller rating, until the rating exists on the document and has a defined missing value. A missing rating is not zero stars and is not five stars.
  • Click history from the current shopper, unless the feature is explicitly personalization and can be turned off.

Normalization

normalizeSearchText in scripts/normalize-search-text.mjs:

  • Applies Unicode NFC.
  • Maps Arabic Yeh and Kaf to the Persian forms.
  • Maps Persian digits ۰-۹ and Arabic-Indic digits ٠-٩ to 0-9.
  • Removes ZWNJ (U+200C) and common bidi marks.
  • Lowercases Latin text and collapses whitespace.

Run it on index text and on the query. Do not also remove ZWNJ from the catalog source of truth. Display text still needs it.

Reindex

On catalog write, enqueue the parent id. The worker loads the parent and its variants and replaces that parent's documents as a set. Replacing one variant document and leaving a deleted variant in the index sells a combination that no longer exists.

Evaluation

Fixtures from evals/tasks.json. See evaluation fixtures for how the checks work.

index-variant

Index a variable product so each color with its own price and stock is a separate search document.

Must include: unit_id, sku

Must not include: one document for the parent only

Worked example

Index the purchasable unit.

Each color is a document with unit_id, sku, the parent title, in_stock, and the variant price. Facet filters use the color value id.

normalize-query

Normalize a Persian shopper query and the catalog title the same way before lexical matching.

Must include: normalizeSearchText

Must not include: English stemmer on Persian

Worked example

Run the shared normalizer on both sides.

const query = normalizeSearchText(rawQuery);

Fold Yeh and Kaf, map digits to ASCII, and remove ZWNJ before analysis. Use a Persian analyzer for Persian titles.