🎉🎉🎉 Our paper, SPEAR has been accepted to the Main Track of RecSys 2026!! 🎉🎉🎉

SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search

English | 简体中文

Overview

SPEAR is an end-to-end framework for personalized query rewriting and retrieval in community search. Conventional systems optimize rewrite selection and item retrieval as separate stages; this mismatch can favor broad, high-frequency rewrites that attract clicks but drift from the user's original intent. SPEAR learns both decisions together while explicitly preserving semantic relevance.

Key Ideas

SPEAR addresses three failure modes in path-based rewriting systems:

• Dual-Embedding Isolation uses task-specific Recall and Rank projection heads, preventing CTR from directly updating Recall-specific parameters while keeping the shared encoders jointly trainable.
• Multiplicative Gating rewards a rewrite only when both selector confidence and query-item relevance are strong, suppressing generic-rewrite shortcuts.
• Dynamic Rewrite Selection uses user context, original/rewrite representations, semantic fidelity, and cross interactions to predict personalized rewrite weights.

At the framework level, a residual original-query path provides a fallback, while a dual-view InfoNCE objective aligns original and rewritten queries with retrieval targets.

The paper reports improvements over the production baseline on 100K held-out search sessions: +18.2% rewrite semantic similarity@10 and +99.5% click recall@10. Online A/B testing also improved query-view CTR by +0.259% and average reading depth by +0.733%.

Repository Contents

This repository provides the PyTorch 2.x implementation of SPEAR. It includes:

• the SPEAR model and paper-aligned ablation presets;
• a versioned 28-feature TSV schema with strict validation;
• deterministic synthetic data for an end-to-end smoke test;
• a bundled 1,000-request companion sample for code validation; and
• commands for training, evaluation, prediction export, and AUC metrics.

Quick Start

The recommended runtime is Python 3.8+ and PyTorch 2.4.1. CPU, CUDA, and Apple MPS devices are selected through --device.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r requirements.txt
./run.sh

./run.sh executes a two-step end-to-end demo: it generates and validates synthetic data, trains, evaluates the checkpoint, exports predictions, and computes AUC metrics. Artifacts are written under runs/.

Common Commands

• ./run.sh: run the complete synthetic-data demo.
• ./run.sh sample: generate deterministic train/valid/test TSVs.
• ./run.sh prepare-paper-data: verify and split the bundled 1K sample.
• ./run.sh validate [PATH]: validate generated data or a specified TSV.
• ./run.sh train [FLAGS]: train a model.
• ./run.sh eval [FLAGS]: evaluate a checkpoint.
• ./run.sh predict [FLAGS]: export prediction scores.
• ./run.sh metrics [FLAGS]: compute global and request-level AUC.

Run ./run.sh help for paths, environment variables, and options.

Train on custom data

Custom input must follow the public schema in configs/feature_schema.json.

./run.sh train \
  --train_file=data/custom/train.tsv \
  --val_file=data/custom/valid.tsv \
  --output_dir=runs/experiment \
  --ablation_mode=full \
  --max_train_step=1000

Use a new output directory for each architecture preset. Training writes model_contract.json; evaluation and prediction reject incompatible checkpoints or settings.

Use the bundled 1K sample

./run.sh prepare-paper-data

DATA_DIR="$PWD/data/prepared/paper_slim_1k" \
MODEL_DIR="$PWD/runs/paper-slim-1k" \
  ./run.sh train

The generated 800/100/100 split is intended to validate the public code path. It is not the official paper split and must not be used to claim paper-level model quality. All bundled categorical values are letter-only anonymous tokens; the sample contains no original query, rewrite, item, or behavior text.

Documentation

• Technical reference: objectives, gradient routing, data contract, evaluation, and ablations
• Sample release notes: verification and deterministic splitting
• Sample data card: provenance, processing, and usage limitations

Citation

If this code supports your research, please cite:

Wenbin Wu, Yuzhong Wu, Yufan Xu, Kuan Fang, Xing Xu, Cheng Ye, and Xiaobin Hu. 2026. SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search. In 20th ACM Conference on Recommender Systems (RecSys '26), September 27-October 02, 2026, Minneapolis, MN, USA. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/3773078.3831757

@inproceedings{wu2026spear,
  author    = {Wu, Wenbin and Wu, Yuzhong and Xu, Yufan and Fang, Kuan and
               Xu, Xing and Ye, Cheng and Hu, Xiaobin},
  title     = {SPEAR: Selection-aware Personalized End-to-end Adaptive Rewriting and Retrieval for Community Search},
  booktitle = {Proceedings of the 20th ACM Conference on Recommender Systems},
  year      = {2026},
  location  = {Minneapolis, MN, USA},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  numpages  = {11},
  doi       = {10.1145/3773078.3831757},
  url       = {https://doi.org/10.1145/3773078.3831757}
}

License

The source code in this repository is licensed under the MIT License.

The bundled files under data/releases/ are excluded from the MIT License. No permission to copy or redistribute those data files is granted unless a separate data license or written authorization is provided.