*Equal contribution
Generative Engine Optimization (GEO) lets content owners rewrite web content to increase their visibility in generative systems. In recommendation agents, this creates a risk that sources controlled by sellers make flawed products appear better supported than they are. We study this risk at the generation stage by asking whether recommendation agents continue to make decisions that align with user utility when these sources are rewritten for GEO. To make this question measurable, we construct SafeGEO, an evaluation suite with 22 GEO attack variants across 600 recommendation cases. We empirically show that GEO attacks can promote flawed target products: they increase the rate at which such flawed products enter the recommendation set by up to 83.2 percentage points (pp). We further study whether agent-side design choices can mitigate this risk and show that simple defenses reduce harmful target promotion by up to 39.2 pp. These gains are substantial but do not restore the no-GEO performance, showing that GEO remains a serious risk despite mitigation.
Tests whether an agent keeps utility-aligned recommendations when seller-controlled sources are rewritten. It uses 22 attack variants and 2 controls over 600 cases and 3 sampled targets (40,800 instances).
Seven atomic manipulation primitives across three loci, composed from single moves to coherent realistic benchmark templates.
Ten full-benchmark Hugging Face configs plus ten matching SafeGEO Diamond configs, with model-facing inputs, hidden benchmark-reference labels, and line-level evidence annotations.
The exact benchmark request plus five prompt/input-level interventions, evaluated on all three sampled targets across the eight realistic attacks.
GEO is a serious threat to recommendation agents, and practical developer-side defenses substantially reduce, but do not eliminate, its impact.
600 recommendation base cases across 6 product verticals, each expanded into 68 instances (22 attack variants times 3 sampled targets, plus 2 controls) for 40,800 instances, shipped as 10 full-benchmark Parquet configs on the Hugging Face Hub.
Configs: visible, labels, candidate_quality,
source_annotations, geo_line_annotations, targets,
instances_manifest, quality_distributions, requirement_annotations,
controlled_documents.
SafeGEO Diamond mirrors all ten configs in a 600-instance, vertical-balanced screening split for inexpensive model iteration. It is a high-effect stress set; use the complete benchmark for population-level reporting.
from datasets import load_dataset
visible = load_dataset("wieeii/SafeGEO", "visible", split="test") # model-facing inputs
labels = load_dataset("wieeii/SafeGEO", "labels", split="test") # hidden benchmark reference
diamond = load_dataset("wieeii/SafeGEO", "diamond_visible", split="test")
SafeGEO models GEO as an adversary that rewrites seller-controlled sources along three loci, built from seven primitives, composed into 22 attack variants and probed against two controls over three sampled targets.
| Code | Primitive | Locus |
|---|---|---|
A | authority laundering | epistemic |
U | unsupported fit claim | content |
C | caveat omission | content |
R | relevance flooding | content |
E | evidence padding | epistemic |
S | salience manipulation | model-facing |
M | model-directed instruction | model-facing |
For each base case, three eligible non-ground-truth candidates are sampled without replacement as GEO targets;
each instance rewrites only that target's own source.
Full taxonomy: all 22 attack variants, controls, and source templates
Given that GEO attacks work, what can a developer do without changing the model? Six matched conditions are compared on the same attacked instances (all three sampled targets and the 8 realistic variants; 14,400 instances per condition), each measured against the exact no-mitigation benchmark request.
Every instance is scored against hidden ground truth. The headline metrics weigh attack success against recommendation utility and safety:
Citation validity, refuting-evidence recall, gap detection, and more are in the full metric glossary.
We evaluate three open-weight agents (Gemma 4 31B IT, Qwen3.6 27B, Devstral Small 2 24B Instruct) and run a frontier robustness check on DeepSeek-V4-Flash. Metrics: Target@3 (attacked-target top-3), HCV@1 (hard-constraint violation at rank 1), GT@3, uNDCG@5.
| Model | Target@3 | HCV@1 | GT@3 | uNDCG@5 |
|---|---|---|---|---|
| Gemma 4 31B IT | 3.4 → 79.6 +76.2 | 16.9 → 75.6 +58.8 | 71.2 → 67.9 −3.3 | 74.4 → 68.6 −5.8 |
| Qwen3.6 27B | 8.1 → 78.3 +70.2 | 24.2 → 83.7 +59.5 | 61.2 → 60.8 −0.4 | 66.5 → 63.6 −3.0 |
| Devstral Small 2 24B Instruct | 12.7 → 90.9 +78.2 | 41.1 → 90.7 +49.7 | 50.7 → 47.9 −2.8 | 67.4 → 59.2 −8.2 |
GEO moves a flawed target into the top 3 in up to 90.9% of cases, up from roughly 3 to 13% under truthful controls. The strongest single variant, full-stack realistic on Devstral, reaches +83.2 on Target@3.
| Layer | Gemma 4 31B IT | Qwen3.6 27B | Devstral 2 24B |
|---|---|---|---|
| L0 No mitigation (Target@3) | 79.6 | 78.3 | 90.9 |
| L1 Defensive prompt | −15.1 | −11.0 | −2.8 |
| L2 Rationale elicitation | −15.0 | +7.5 | +2.3 |
| L3 Evidence breakdown | −29.7 | −39.2 | −17.7 |
| L4 Context balancing | −11.5 | −4.5 | −3.2 |
| L5 Instruction filtering | −2.2 | +3.0 | −0.5 |
The L3 candidate-level evidence breakdown is the strongest defense, reaching a 39.2 point Target@3 reduction, but no condition restores no-GEO performance.
Robustness across scale. DeepSeek-V4-Flash (larger, more recent) is the most robust agent we evaluate, yet a single seller-controlled rewrite still lifts Target@3 from 4.6% to 72.6% (+68.0) and HCV@1 from 23.0% to 73.4% (+50.4). Full per-variant and 22-variant results are in the paper.
@inproceedings{wen-etal-2026-safegeo,
title = {SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents},
author = {Wen, Qianfeng and Liu, Yifan Simon and Liu, Xin and Jiao, Difan and Yang, Blair and Wu, Junda and Tang, Zhenwei},
booktitle = {Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing},
year = {2026}
}