How far can icon-only training go for task-conditioned full-screen hero and skill detection?

1. The short answer

The experiment asks a deliberately narrow question: if we do not collect additional gameplay crops for training, how far can the existing labeled icon library take us? The answer has two parts. First, the ID-trained and multi-task embeddings group matching identities extremely well on a seed-held-out synthetic-view test: the identities are known during training, but the exact generated backgrounds, color changes, shifts, and blur are new. Their MAP@R is near 0.995. Second, full screenshots are much harder. The crop-aware reranker raises end-to-end F1 to 0.765, but five of eighteen objects are still missed and three false detections remain.

This is still a useful result. It separates two problems that are often blurred together: recognizing what an icon is and finding the exact rectangle that contains it. The first problem is almost solved for synthetic held-out views. The second remains the bottleneck.

2. What is the system trying to do?

You do not need to know the game. A screenshot can contain three selectable heroes or three main skills. The program must return a list of rectangles, and each rectangle must have the correct identity. A prediction counts as correct only when both conditions hold: the predicted ID matches the label, and the rectangle overlaps the labeled rectangle by at least IoU 0.50.

Vocabulary used below

3. Exactly what does “icon-only training” mean?

The catalog contains 354 semantic identity classes represented by 390 clean image files. Some Arena skills have several valid visual variants, so the number of files is larger than the number of IDs; the extra files are not extra classes. The experiment catalog excludes the four scroll-shaped basic-magic references; the valid magic skill images are book-like or their legitimate level variants.

Audit count: Gameplay screenshot pixels used to update the ID-trained, multi-task, or crop-aware model weights = 0; gameplay bounding boxes used to update those weights = 0. Real validation and test screenshots are evaluation evidence, not training examples.

Procedural backgrounds are generated by code: smooth colors, faint rectangles, and lines. They are not copied from gameplay screenshots. Color jitter, resizing, blur, rotation, and synthetic cropping also create new pixels, but they do not introduce a new photographed or game-recorded object.

4. Train, validation, and test: what is actually held out?

Every known label appears during training because the product requirement is to recognize all known game icons. What is held out is the random view, not the identity. Training, validation, and test use disjoint deterministic seed namespaces, so the exact background, scale, shift, rotation, color, and blur combination in a test image was never used for training.

The ID-trained and multi-task models train for twelve epochs. In every epoch, each identity contributes four independently generated view-pairs; each pair contains two views, and a batch covers 32 identities. Catalog validation uses four fixed views per ID to retain a checkpoint; the catalog synthetic-view test uses six fixed views per ID, for 2,124 test queries. Neither set updates weights. When an ID has several image files, each generated view may start from a different legitimate visual, encouraging cross-variant grouping.

What augmentation is trying to teach

Identity augmentation says, “these visual changes should not change who this is.” It therefore encourages the backbone to ignore modest changes in background, brightness, scale, and position. That is good for recognition, but it creates a localization tension: if the model becomes too shift-invariant, a poorly aligned crop can still receive a high identity score.

5. The four algorithms, in logical order

The reader-facing sequence is A → B → C → D. G0/G1/H1/H2 are internal experiment codes retained only so the report can be matched to checkpoints and metrics. All stages use the same fixed candidate grid; each row below adds exactly one training idea.

Algorithm A — Generic embedding (experiment code G0)

G0 is the starting point. It uses torchvision 0.28.0 convnext_tiny(weights=ConvNeXt_Tiny_Weights.IMAGENET1K_V1) and removes the final 1,000-class linear layer. Inputs use ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225). For any 224×224 crop, features → AdaptiveAvgPool2d(1) → classifier LayerNorm produces the 768-D pre-linear vector, which is L2-normalized.

For a query crop q and a reference icon vector r, the score is cosine similarity:

G0 receives no game-icon fine-tuning. It tells us how much the generic ImageNet representation already knows. This is essential: without G0, a strong-looking result might merely restate what the pretrained network could already do.

Algorithm B — ID-classification embedding (experiment code G1)

G1 starts from the same weights but adds one 354-way classification head and fine-tunes on the labeled icons. The training target is ordinary cross-entropy: raise the logit of the correct ID and lower the others. After training, the ID head is discarded and retrieval again uses the normalized 768-D penultimate representation.

G1 is the cleanest answer to “does training over icons and labels help?” It does. Augmentation MAP@R rises from 0.8995 to 0.9954, and real-screen F1 rises from 0.5161 to 0.6111. G1 is also scientifically important because it prevents us from crediting every H1 gain to the more elaborate losses.

Algorithm C — Multi-task identity embedding (experiment code H1)

H1 attaches three training outputs to the same backbone representation:

768-D shared representation r
 ├─ 4-way type head: hero / skill / creature / artifact
 ├─ 354-way global-ID head
 └─ 128-D projection head used only by supervised contrastive loss

The total objective is:

Cross-entropy answers “which label?” The type head supplies a coarse semantic signal. Supervised contrastive loss directly shapes the geometry of the representation. Its temperature 0.07 controls how sharply the loss emphasizes the closest competing samples; it is a scale parameter, not an accuracy threshold.

The type and ID predictions are counted jointly correct only if both match. The multi-task model achieves 1.000 type accuracy, 1.000 ID accuracy, and 1.000 joint accuracy on 2,124 seed-held-out synthetic views. The projection head is discarded after training; the production embedding is still the full backbone's normalized 768-D representation.

H1's MAP@R is 0.9945, slightly below G1's 0.9954. That tiny ordering means the extra losses did not improve this already-saturated synthetic retrieval metric. Yet H1's screen F1 is 0.6486, above G1's 0.6111. Different downstream behavior can improve even when a nearly saturated proxy metric moves sideways.

6. Algorithm D — Crop-aware reranking (experiment H2)

The first three algorithms primarily learn identity. To teach localization while respecting the icon-only boundary, the crop-aware stage creates a 320×320 procedural canvas, places one canonical icon on it, and samples a proposed square. Because the icon placement and proposal are generated by code, the exact ground-truth box and IoU are known for free.

For each of 390 catalog image files, this stage uses 16 weight-updating training crops, 8 catalog-derived validation crops, and 8 synthetic-test crops. Because there are eight families, validation and test contain each family exactly once per image file: tight complete, loose complete, two shift directions, small partial, oversized with background, barely overlapping, and background only. Split-specific seeds keep the exact pixels disjoint.

It freezes the multi-task embedding model. The localization head reads model.features(x), the final 768×7×7 spatial map, pools it to 768×3×3, and flattens that into 6,912 values. A LayerNorm → Linear(6912,256) → GELU → Dropout(0.10) → Linear(256,96) → GELU trunk feeds four output layers:

• continuous IoU between proposal and synthetic truth;
• whether the crop is “full” at IoU ≥ 0.50;
• whether any object is present;
• class-agnostic center and size offsets from the proposal to the truth.

The synthetic-test results show that this head learned its generated task: IoU MAE 0.0485, IoU rank correlation 0.9547, full-crop AUROC 0.9622, and objectness AUROC 0.9963. AUROC is threshold-free. Spearman correlation measures ordering rather than exact calibration.

IoU_hat is passed through a sigmoid, so all three quality inputs lie in [0,1]. The cube root is their geometric mean; α controls how strongly this combined quality modifies identity similarity. Real-screen validation tried α ∈ {0.25, 0.5, 0.75, 1.0, 1.5, 2.0} and selected 2.0.

7. How full-screen sliding-window search works

The detector does not use pixel template matching. Template matching would compare raw aligned colors and be fragile to scale, background, and visual effects. Here the program enumerates many square candidate boxes, resizes each crop to 224×224, encodes it, and compares its vector to the reference bank.

Why only these windows? These selection screens constrain icon sizes to a narrow range. Rather than enumerate every possible rectangle, the program tries several plausible square sizes at regular positions. This game-UI prior is what makes exhaustive search manageable.

1. Enumerate proposals. Hero search uses five sizes from 128 to 192 pixels with 20-pixel stride, producing 21,193 windows on a 1920×1080 screen. Skill search uses six sizes from 72 to 132 pixels with 16-pixel stride, producing 43,134 windows.
2. Retrieve the best ID. Compare the candidate embedding with the 390 reference-image embeddings. When an ID has multiple images, keep its best similarity. This produces S_identity: “how much does this crop resemble the best catalog ID?”
3. Apply crop quality in Algorithm D. The separate S_crop asks whether the icon is present, complete and aligned. The final score is S_identity × S_crop^2, with exponent 2 selected on real-screen validation.
4. Remove duplicates. NMS sorts every candidate by the variant's final score, regardless of predicted ID. Before the acceptance threshold, it keeps the highest score and suppresses any later box with IoU>0.10 against a kept box, stopping at 20 survivors. Two overlapping boxes with different predicted IDs do not both survive.
5. Optionally move boxes. The D+ / H2-offset variant applies half of the predicted box offset.
6. Accept detections. A real-screen-validation-selected score threshold is frozen for engineering test, with at most three detections per task-screen.

Real-screen validation contains three recordings and the engineering test contains three disjoint recordings. Each recording contributes one hero screen, one skill screen, and two task-conditioned negative screens: for example, a skill-selection screenshot is negative for the hero-search task even though other UI icons may be visible. This produces 12 task-screens and 18 positive objects per split. The “negative-screen accept rate” counts task-screens, not boxes: one false box and ten false boxes both make that negative task-screen count as accepted. Algorithm D has three FP boxes spread across 3 of 6 negative task-screens, so its rate is 3/6 = 0.500.

8. Complete results

8.1 Seed-held-out synthetic-view retrieval

Metric key and protocol. This is a closed-set augmentation-robustness test, not category-level generalization. Each of the 2,124 test views is a query; its own vector is removed, the other 2,123 test views form the gallery, and R=5 because each ID has five other test views. Precision@1 checks only the nearest neighbor. R-Precision measures the fraction of same-ID views in the first R results. MAP@R also rewards putting those relevant views earlier. The cosine gap is mean same-ID similarity minus mean different-ID similarity.

Precision@1 reaches 1.000 for the ID-trained and multi-task embeddings. The ID-trained model wins MAP@R by 0.0009; this is too small and too close to saturation to justify a general superiority claim.

8.2 Multi-task classification on seed-held-out synthetic views

“Both correct” requires the four-way type and the 354-way ID to match simultaneously. “Hierarchy consistency” means the predicted coarse type agrees with the catalog type implied by the predicted ID.

8.3 Seed-held-out synthetic crop-quality test

IoU MAE is the mean absolute error between predicted and true overlap; lower is better. Spearman measures whether crop-quality ordering is correct. AUROC measures threshold-free separation of positive and negative crops.

8.4 Real-screen-validation-frozen engineering benchmark

Do not read the ladder as uniform domination. From Algorithm A to C, TP rises from 8 to 12 and FN falls from 10 to 6, but FP also rises from 5 to 7. Algorithm C's improvement is primarily a recall gain; it does not solve false-positive rejection. Algorithm D is the stage that later reduces FP to 3.

Why does box regression tie instead of improve? On validation, the selected 0.50 scale raises F1 from 0.7222 to 0.7778 and mean correct-ID IoU from 0.5341 to 0.5517. On the development-inspected test, however, it leaves TP/FP/FN unchanged at 13/3/5 and mean correct-ID IoU changes from 0.5270 to 0.5261. Synthetic box geometry did not improve this test reliably.

Per-recording counts reveal failure clustering that the aggregate F1 hides: Recording C misses its entire hero screen, while all three skill screens succeed.

9. Inspecting successes and failures

The final result is asymmetric. Main skills transfer well: the test contains nine skill objects and all nine are found. Heroes are the weak point: only four of nine are found. One entire hero screen from the final recording falls below the frozen threshold, while three false alarms occur on negative task-screens.

This pattern suggests that the remaining limitation is not global identity confusion. Among the 13 accepted predictions that first overlap a ground-truth object at IoU≥0.50, all 13 have the correct global ID: conditional identity accuracy is 13/13. The larger problem is the domain gap between procedural icon canvases and the real hero-selection UI: decorative frames, text, portraits whose effective visual region is not square, and background elements can make synthetic completeness look different from real completeness.

10. Why the old 0.857 H2 result is not the main answer

An earlier H2 prototype trained a similar quality head on 3,970 proposals cut directly from gameplay recordings and their real bounding boxes. It reached test F1 0.8571 with TP/FP/FN 15/2/3. This earlier result is useful as a diagnostic comparison: matched real-proposal supervision appears to reduce the remaining domain gap.

The gap between icon-only H2 (0.765) and X2-real-proposal (0.857) is consistent with a domain-mismatch explanation. It does not prove that 0.857 will generalize to a larger untouched set, but it shows where additional labeled data would likely buy improvement if the project later relaxes the data constraint.

11. What we can conclude—and what comes next

1. Icon fine-tuning clearly helps. The ID-trained embedding raises closed-set MAP@R from 0.8995 to 0.9954 and screen F1 from 0.5161 to 0.6111.
2. The richer multi-task objective helps the screen task, not the saturated icon metric. Screen F1 rises to 0.6486 while MAP@R is essentially tied with the ID-trained model.
3. Synthetic crop-quality supervision shows evidence of transfer on this engineering benchmark. Crop-aware ranking reaches 0.7647 F1 without any gameplay image or box updating its weights; because the test recordings were inspected during development, this is not an unbiased estimate for new recordings.
4. Synthetic box adjustment does not improve this test reliably. It improves validation but not the frozen engineering test.
5. The system is not close to 100%. Hero recall and negative-screen rejection remain the dominant failures.

If the next experiment must keep the same icon-only boundary, the most defensible improvements are stronger procedural UI simulation, multi-scale feature fusion, an explicit background/open-set rejection objective with more varied generated negatives, and a detector architecture trained entirely on synthetic icon placements. The next evaluation should add more untouched recordings before any more tuning. If the boundary is later relaxed, a small reviewed real-crop set is the most direct route to higher stability.

Within the stated data boundary, the experiment answers the narrow question it set out to test. The icon library is enough to learn identity extremely well and to improve task-conditioned hero/skill search substantially on this engineering benchmark. It is not enough, by itself and with this synthetic generator, to guarantee exact boxes and near-zero false alarms across all gameplay.

12. Glossary for a returning perception reader

Key terms above can be hovered with a mouse or focused with the keyboard. This always-visible glossary contains the same ideas.

Backbone

The shared visual network that converts pixels into feature maps and embeddings.

Embedding

A numeric vector used for similarity search rather than a final class probability.

Classification head

A training-time layer that maps the shared representation to class logits.

Cosine similarity

Similarity between normalized vector directions; higher means more alike.

Proposal / sliding window

One candidate rectangle cut from a full screenshot.

IoU

Intersection area divided by union area for two rectangles.

NMS

A rule that keeps the highest-scoring box and suppresses overlapping duplicates.

Threshold

The score cutoff above which a prediction is accepted.

AUROC

Threshold-free ranking quality for a binary target.

MAP@R

Retrieval metric that evaluates whether all relevant same-ID views rank early.

Precision / recall / F1

False-alarm control, miss control, and their harmonic-mean balance.

Box regression

Predicting how to move and resize a proposal toward a target box.

Internal reproducibility artifacts retained with the experiment: metrics.json, g1-embedding-metrics.json, h2-icon-only-metrics.json, training histories, sliding-window records, configuration files, checkpoints, and source scripts. They are not currently linked from this public page.

1. 先说结论

这个实验故意把问题限制得很窄：如果不额外收集游戏截图裁剪，只用已有的带标签图标库，我们能走多远？答案分成两部分。第一，ID 训练 embedding 和多任务 embedding 能把相同 ID 的图标放得非常近；在固定随机种子、训练阶段未生成过的合成视图测试上，MAP@R 接近 0.995。这里的 ID 在训练中见过，但具体背景、颜色变化、位移和模糊组合是新的。第二，把这种特征表示用于整张游戏截图要难得多。裁剪感知重排器把端到端 F1 提升到 0.765，不过 18 个目标仍漏掉 5 个，并留下 3 个误检。

最重要的认识是：“这是哪个图标？”和“哪一个框才完整、对齐地包住图标？”是两个不同问题。前者在合成视图上几乎解决了，后者仍然是瓶颈。

2. 系统到底要做什么？

你不需要玩过这个游戏。一个选择页面可能显示三个英雄，或者三个主技能。程序必须返回若干矩形框，并为每个框给出正确 ID。只有两个条件同时满足才算真阳性（TP）：预测 ID 正确，而且预测框与人工标注框的 IoU 至少为 0.50。

下文会反复使用的词

3. “只用图标训练”具体是什么意思？

图标目录包含354 个语义身份类别，由 390 张干净图片表示。一些 Arena 技能有多个合法等级外观，所以图片数大于 ID 数；多出来的图片不是多出来的类别。数据审计已经移除四张卷轴形的错误基础魔法参考图；有效的魔法技能使用书本图标或其合法等级版本。

审计结果：用于更新 ID 训练、多任务和裁剪感知模型的真实游戏截图像素 = 0；真实框 = 0。真实验证/测试截图只用于选择运行参数和评测，不参与反向传播。

程序背景由代码绘制，包括平滑颜色、浅色矩形和线条；它们不是从游戏截图复制的。颜色抖动、缩放、模糊、旋转和合成裁剪会生成新像素，但不会引入新的游戏截图对象。

4. Train / validation / test 到底 hold out 了什么？

所有已知标签都参加训练，因为产品目标就是识别所有已知游戏图标。被留出的是随机生成的增强视图，不是图标 ID。训练、验证和测试使用不同且可复现的随机种子区间，所以测试图片的背景、缩放、位移、旋转、颜色和模糊组合没有在训练中出现。

ID 分类与多任务模型都训练 12 个 epoch。每个 epoch，每个 ID 生成 4 对独立增强视图；每对包含两张图，每个 batch 覆盖 32 个 ID。目录 validation 为每个 ID 固定生成 4 张图，只用于保留 checkpoint；目录 synthetic test 生成 6 张，共 2,124 个查询，只报告合成指标。二者都不更新权重。若一个 ID 有多个合法图片，生成器可以从不同版本起步，从而教模型把它们聚在一起。

数据增强想让模型学到什么？

身份增强的含义是：“这些外观变化不应该改变它是谁。”因此 backbone 会学着忽略适度的背景、亮度、尺度和位置变化。这对识别很好，却带来定位上的矛盾：模型如果对位移过于不敏感，一个没有对齐的裁剪仍可能得到很高的身份分数。

5. 四个算法，按逻辑顺序解释

面向读者的顺序是 A → B → C → D。G0/G1/H1/H2 只是为了对应 checkpoint 和 metrics 而保留的实验代码。四个阶段使用完全相同的固定候选网格；下表每一行只比上一行增加一个训练想法。

算法 A — 通用 embedding（实验代码 G0）

G0 使用 torchvision 0.28.0 的 convnext_tiny(weights=ConvNeXt_Tiny_Weights.IMAGENET1K_V1)，并移除最后的 1,000 类线性分类层。输入使用 ImageNet 均值 (0.485, 0.456, 0.406) 和标准差 (0.229, 0.224, 0.225)。任意 224×224 裁剪依次经过 features → AdaptiveAvgPool2d(1) → classifier LayerNorm，得到线性分类层之前的 768 维向量，再做 L2 归一化。

对于查询向量 q 和参考向量 r，余弦分数是：

G0 不做任何游戏图标 fine-tune。它回答“通用 ImageNet 特征本来就能做多好？”保留这个对照组，才能把后面的提升归因于图标领域学习或裁剪质量学习，而不是更换了整个流程。

算法 B — ID 分类 embedding（实验代码 G1）

G1 从相同权重开始，加一个 354 类 classification head，用带标签图标训练。标准 cross-entropy 会提高正确 ID 的 logit、压低错误 ID。训练结束后丢掉分类头，检索仍使用归一化后的 768 维倒数第二层特征。

这是回答“直接用图标和 ID 标签做普通分类是否有帮助？”最干净的实验。答案是有：合成视图 MAP@R 从 0.8995 升到 0.9954，真实画面 F1 从 0.5161 升到 0.6111。

算法 C — 多任务身份 embedding（实验代码 H1）

H1 在同一个 768 维共享特征上连接三个训练输出：

768-D shared representation r
 ├─ 4-way type head: hero / skill / creature / artifact
 ├─ 354-way global-ID head
 └─ 128-D projection head: 只用于 supervised contrastive loss

Cross-entropy 回答“是哪一个标签？”类别头提供粗粒度语义。Supervised contrastive loss 直接改变 embedding 空间的几何结构。Temperature 0.07 控制 loss 对最近竞争样本有多敏感，它不是准确率阈值。

只有类别和 ID 同时正确才算联合正确。多任务模型在 2,124 个固定种子的合成测试视图上，类别准确率、ID 准确率和联合准确率都是 1.000。Projection head 训练后丢弃，实际使用的 embedding 仍是 backbone 的 768 维特征。

H1 的 MAP@R 是 0.9945，比 G1 的 0.9954 略低 0.0009，说明这个合成指标已经接近饱和；但真实画面 F1 从 0.6111 升到 0.6486。代理指标几乎不动，不代表下游行为不能改善。

6. 算法 D — 裁剪感知重排（实验代码 H2）

前三个算法主要学习身份。为了在不使用真实截图训练的前提下教定位，我们在 320×320 的程序画布上放一张标准图标，再采样一个候选方框。图标位置和候选框都由代码生成，因此真值框和 IoU 可以精确计算，不需要人工标注。

每张目录图片生成 16 个更新权重的训练裁剪、8 个目录派生 validation 裁剪和 8 个 synthetic test 裁剪。八类分别是：紧贴完整、宽松完整、两个位移方向、小幅残缺、包含过多背景、几乎不重叠和纯背景。不同数据划分使用不同随机种子，精确像素不重合。

这个阶段冻结多任务 embedding 模型，读取 model.features(x) 输出的 768×7×7 空间特征图，pool 成 768×3×3，再展平成 6,912 个数。随后经过 LayerNorm → Linear(6912,256) → GELU → Dropout(0.10) → Linear(256,96) → GELU，并分别预测连续 IoU、图标是否完整、是否包含对象，以及候选框到真值框的中心/尺寸偏移。

在固定种子的合成裁剪测试上，IoU MAE = 0.0485，IoU 排名相关性 = 0.9547，完整裁剪 AUROC = 0.9622，objectness AUROC = 0.9963。这说明它确实学会了生成器定义的裁剪质量任务；能否迁移到游戏画面，仍要看真实画面评测。

IoU_hat 经过 sigmoid，因此三个质量信号都在 [0,1]。立方根把它们变成几何平均数；α 控制综合质量对身份相似度的影响强度。真实截图 validation 尝试 α ∈ {0.25, 0.5, 0.75, 1.0, 1.5, 2.0}，最终选择 2.0。

7. 滑动窗口全屏搜索如何工作？

这个检测器不是原始像素的 template matching。模板匹配会直接比较对齐像素，对尺度、背景和视觉特效很敏感。这里的程序枚举大量方形候选框，把每个裁剪缩放到 224×224，编码成 embedding，再和参考库比较。

为什么只试这些框？这些选择界面把图标尺寸限制在一个较窄范围内。程序不枚举所有可能矩形，而是只在规则位置尝试几种合理的正方形尺寸；这个游戏 UI 先验让穷举搜索变得可计算。

1. 生成候选框。英雄任务使用 128–192 像素的 5 种尺寸、20 像素步长；1920×1080 画面上共 21,193 个窗口。技能任务使用 72–132 像素的 6 种尺寸、16 像素步长，共 43,134 个窗口。
2. 找最佳 ID。候选 embedding 与 390 张参考图的 embedding 比较；一个 ID 有多个图片时取最大相似度。这个 S_identity 回答：“它有多像最匹配的目录 ID？”
3. 算法 D 加入独立的裁剪质量。S_crop 回答：“图标是否存在、完整而且对齐？”最终分数是 S_identity × S_crop^2；指数 2 只由真实截图 validation 选择。
4. 删除重复框。NMS 按当前算法的最终分数对所有候选框排序，不区分预测 ID。在应用接受阈值之前，它保留最高分框，并删除与任意已保留框 IoU>0.10 的后续框，直到留下 20 个；两个重叠但预测 ID 不同的框也不能同时保留。
5. 可选移动框。D+ / H2-offset 应用一半预测偏移。
6. 接受或拒绝。真实截图 validation 选择分数阈值；engineering test 时冻结。每个任务-截图对最多返回 3 个检测。

真实截图 validation 有 3 段录像，engineering test 有另外 3 段互不重叠的录像。每段形成一个英雄正任务、一个技能正任务，以及两个任务条件式负样本；每个 split 共 12 个任务-截图对、18 个正目标。“负画面接受率”计算的是任务-截图对，不是框：一个负画面上出现 1 个或 10 个误检，都会让这个画面只计作一次“被接受”。算法 D 的 3 个 FP 分布在 6 个负任务-截图对中的 3 个，因此该比例是 3/6 = 0.500。

8. 完整实验结果

8.1 闭集合成视图的增强鲁棒性检索

怎么读、怎么算：这是闭集增强鲁棒性测试，不是新类别泛化。2,124 张测试图逐一作为查询；查询自身从图库中移除，其余 2,123 张作为候选。每个 ID 还有 5 张相关测试图，因此 R=5。Precision@1 只看最近邻；R-Precision 看前 5 个结果里有多少是同 ID；MAP@R 还奖励把相关图排得更靠前。Cosine gap 是平均同 ID 相似度减去平均不同 ID 相似度。

8.2 多任务模型的合成视图分类

“同时正确”要求 4-way type 和 354-way ID 都匹配；hierarchy consistency 是 predicted coarse type 与 predicted ID 在 catalog 中对应的 type 一致。

8.3 随机种子留出的合成裁剪质量测试

IoU MAE 是 predicted overlap 与 true overlap 的平均绝对误差，越低越好；Spearman 看排序是否正确；AUROC 衡量不选 threshold 时，positive 是否通常排在 negative 前面。

8.4 验证集选参后冻结的真实画面工程测试

不要把算法阶梯理解成每个指标都单调变好。从算法 A 到 C，TP 从 8 增加到 12、FN 从 10 减少到 6，但 FP 也从 5 增加到 7。算法 C 的提升主要来自召回率，不代表它解决了误检；真正把 FP 降到 3 的是算法 D。

为什么 box adjustment 没有继续提升？验证集上，50% 框位移把 F1 从 0.7222 提到 0.7778，正确 ID 的平均 IoU 从 0.5341 提到 0.5517；但在工程过程中已经查看过的测试集上，TP/FP/FN 仍为 13/3/5，平均 IoU 反而从 0.5270 变为 0.5261。合成框的几何规律没有在这个测试集上带来稳定提升。

分录像计数揭示了总 F1 看不出的失败聚集：录像 C 的整张英雄画面全部漏掉，而三张技能画面都成功。

9. 成功和失败到底长什么样？

结果明显不对称：测试集中 9 个主技能全部找到，但 9 个英雄只找到 4 个。一整张英雄选择画面的 3 个英雄都低于冻结阈值；负任务-截图对上仍有 3 个误检。

这更像领域差异，而不是全局 ID 混淆。在 13 个先满足 IoU≥0.50 的接受预测中，13/13 的 global ID 都正确；也就是说，给定定位成功后，条件身份准确率为 100%。更大的问题是程序画布与真实英雄选择 UI 的差异：装饰边框、文字遮挡、英雄头像的有效视觉区域并不严格是正方形，背景元素也更复杂。合成数据中“完整裁剪”的含义没有完全迁移到真实英雄框。

10. 为什么旧的 0.857 结果不是主答案？

早期 prototype 用真实游戏录像和标注框生成的 3,970 个候选框，训练类似的质量 head，测试 F1 = 0.8571（15 TP / 2 FP / 3 FN）。这个结果可作为诊断性对照：与任务匹配的少量真实候选框监督，似乎能缩小剩余领域差异。

纯图标 H2 的 0.765 与真实候选框 X2 的 0.857 之间存在差距，这与领域不匹配的解释一致；但它不证明 0.857 能推广到更大的全新测试集。

11. 我们能得出什么结论？下一步是什么？

1. 图标 fine-tune 明显有帮助。ID 训练 embedding 把闭集合成 MAP@R 从 0.8995 提到 0.9954，把真实画面 F1 从 0.5161 提到 0.6111。
2. 多任务目标改善真实画面任务，但没有改善已饱和的图标指标。F1 升到 0.6486，MAP@R 与 G1 基本打平。
3. 合成裁剪质量监督在这个工程评测集上显示出迁移证据。不使用游戏图片或真实框更新参数，F1 仍达到 0.7647；但由于测试录像在开发中被查看过，这不是对新录像泛化性能的无偏估计。
4. 合成框调整没有可靠改善这个测试集。它改善验证集，但没有改善冻结后的工程测试结果。
5. 系统离 100% 还很远。英雄召回率和负画面拒绝仍是主要失败点。

如果下一项实验继续坚持纯图标训练，可以尝试更强的程序 UI 模拟、多尺度特征、明确的背景/开放集拒绝目标，或者完全在合成图标摆放数据上训练检测器。无论采用哪一种，下一次调参前都应该先建立更多未查看过的录像。若放宽数据限制，一小组经过审核的真实裁剪会是最直接的稳定性来源。

在明确的数据边界内，这项实验回答了它一开始提出的窄问题：图标库足以把身份学得非常好，也足以在这个工程评测集上显著改善任务条件式英雄/技能搜索；但仅靠当前图标库与合成生成器，还不能保证所有游戏画面上的精确框和近乎零误检。

12. 给很久没做 perception 的读者：术语表

Backbone

把像素转换成特征图和 embedding 的共享视觉网络。

Embedding

用于相似度搜索的数值向量，不是最终类别概率。

Classification head

训练时把共享特征映射成各类别 logit 的最后几层。

Cosine similarity

比较归一化向量的方向；越高通常越相似。

Proposal / sliding window

从整张截图裁出的一个候选框。

IoU

两个矩形框的交集面积除以并集面积。

NMS

保留高分框并压掉高度重叠重复框的规则。

Threshold

超过这个分数才接受预测的阈值。

AUROC

不固定阈值时，二分类分数的排序质量。

MAP@R

检验同 ID 视图是否整体排在检索结果前面的指标。

Precision / recall / F1

分别关注误检、漏检，以及两者的平衡。

Box regression

预测候选框应该怎样移动和缩放，才能更接近真值框。

实验内部保留了 metrics、训练历史、滑动窗口记录、配置、checkpoint 和源代码；当前公开页面没有提供这些文件的下载链接。