Comparing OCR Pipelines for Small Game UI Titles

In Heroes of Might and Magic: Olden Era, a selection screen offers three creature cards. Each card has a small Chinese title above its artwork. The annotation system should read that title, resolve it to a bilingual game catalog, and attach the name to the neighboring image crop. A wrong reading can send the crop to a weak visual fallback and create a misleading annotation suggestion; if accepted during review, that error could contaminate training data.

This experiment asks a practical engineering question: which available optical character recognition (OCR) path is both accurate enough and fast enough for a resident video-processing service? It reruns six known readable failures and five rejection controls through three distinct OCR engine/model stacks. The historical pipeline is retained as a fourth reproducible path.

PP-OCRv6 medium transcribed all six readable crops correctly from the original pixels and processed 15.24 source crops per second under the resident recipe timing. Windows OCR and EasyOCR each recovered five. The rejection controls expose a separate problem: every engine correctly read a creature name attached to one invalid image crop, so OCR plus a dictionary would still accept the wrong training pair.

B024 therefore tests two questions separately:

1. Did OCR transcribe the visible title exactly?
2. Was this region a valid Creature-selection title-to-image pair in the first place?

1. What the program must return

The input to the wider perception system is a complete video frame and its timestamp. Once the scene and card layout are known, B024 receives only a fixed title strip: nine of the eleven strips are 412×47 pixels and two are 309×35. Its immediate output is recognized text plus an optional model score. A catalog resolver then maps exact Chinese or English names and aliases to a stable creature ID.

A production result needs one more output: accept, reject, or retry. A title may be hidden by a tooltip, the frame may belong to a spell page, or the neighboring image may be an unusable crop. Those cases should not acquire a creature identity even if OCR returns a valid creature name.

graph LR
    A["Scene and layout"] --> B["Crop quality"]
    B --> C["Fixed title crop"]
    C --> D["Three OCR engines"]
    D --> E["Creature catalog"]
    E --> F["Identity evidence"]
    B --> G["Reject or retry"]
    E --> G

Figure 1. OCR is one step inside the decision. Scene, visibility, and crop quality decide whether a correct word is allowed to identify an image.

2. How the experiment was designed

The dataset contains eleven title crops from nine complete source frames. It has three roles:

Subset	Cases	What is measured
Readable failure-recovery set	6	Normalized exact transcription; normalization is defined below
Occlusion and scene-misroute controls	4	Whether OCR plus the Creature catalog rejects text that cannot identify a Creature crop
User-confirmed invalid title-image pair	1	Whether correct title text is incorrectly allowed to validate a bad neighboring image

This is an exploratory diagnostic failure set, not a random or prospectively held-out test set. It was assembled from known historical failures and explicit invalid-input controls before the B024 comparison run, and was used to choose among engines. It therefore provides development evidence, not an untouched final test. Five readable crops came from the earlier OCR-problem queue; the sixth had already produced a rejected annotation. The six readable crops contain only four distinct visible strings: 日矛圣骑兵 appears three times, including two samples from one recording. A 6/6 result measures recovery on these examples; it does not estimate general production accuracy.

No OCR model was trained or fine-tuned on these images. All current engines received every crop. Each engine used a fixed list of image variants, and its selected result followed one rule: prefer a normalized exact Creature name or alias; otherwise keep the highest engine score, with a fixed variant order for ties. Expected text was consulted only after inference for scoring.

The experiment reports four groups of measurements:

1. Normalized exact match: original-only, enhanced-only, production-selected, and any-variant counts for the six readable crops.
2. Creature-route text+catalog rejection: how many of five invalid inputs were rejected without a scene, visibility, or image-quality gate.
3. Startup-to-completed-batch time: preprocessing from original crops plus a fresh worker, imports, engine creation, warmup, inference, IPC, and JSON output; subsequent catalog selection is excluded.
4. Resident recipe time: parent preprocessing plus the worker’s measured post-warmup batch. This estimates steady-state OCR work; it is not a directly timed persistent-service request.

Normalization applies Unicode NFKC, case folding, removes non-alphanumeric characters, and uses Windows zh-CN Traditional-to-Simplified conversion. Raw OCR strings remain in the receipt. Every current engine was run in a fresh process three times on the same AMD Ryzen 9 9900X / NVIDIA RTX 5080 host. The article reports medians. Engine-specific preprocessing means this is a comparison of the complete programs actually available to the project, rather than an equal-pixel comparison of bare neural networks.

3. Every source frame and OCR input

The cards below show all nine complete source frames. Directly under each frame is the exact original title strip used by B024. The six readable failures come first.

The remaining three source frames create five negative cases. The spell page contributes three fixed regions—绝望, 狂暴, and 顺风—because an upstream scene error routed all three through the Creature cropper.

The next grid isolates all eleven original title strips so that small glyph differences remain inspectable at full size.

The diagnostic set contains seven title-input or quality cases: six readable historical OCR failures and one title that must be rejected because tooltip text covers it. It also contains three misrouted slot negatives and one user-confirmed invalid image target. The three spell regions and the invalid target are excluded from Creature identity training.

Case 6 is the readable 虔信者 title shown in the reviewed-wrong screenshot. Case 7 is a different frame: its saved title region contains tooltip prose beginning with 生物先攻值提升…, so the real creature title is unavailable.

The first five cases came from the pending OCR-problem queue. The sixth readable failure, 虔信者, had already been reviewed: OCR returned 信者, exact lookup failed, and a low-confidence image fallback proposed 骷髅兵. The reviewer rejected that image prediction. After this comparison, all six readable results were reopened as PP-OCRv6-backed identity drafts and the covered title was reopened as a crop-quality draft. All seven now wait in Needs Review; none is human-confirmed training truth yet.

4. Four paths backed by three engines

What “old title OCR” actually means

The historical title OCR column is not a fourth neural network. It is the path then numbered B08 and old B021, now documented by B023, that produced the stored results in two-level-corpus-b08-titles-2026-09-03-ocr:

1. The scene and title-box coordinates were already known.
2. The program cropped each title, enlarged it 3× with cubic interpolation, and added a 24-pixel border whose color came from the crop median.
3. It called the installed Simplified-Chinese Windows.Media.Ocr engine.
4. If the color crop did not exactly match a game name or alias, it retried a grayscale CLAHE contrast crop.
5. Normalized text was matched against the bilingual game catalog. Fuzzy matches remained suggestions rather than truth.

The B024 Windows OCR result uses the same operating-system engine with a stronger experiment wrapper: original pixels, 4× Lanczos, 4× CLAHE, and 4× Otsu variants. This distinction matters. 剑士 failed in the old 3× path but succeeded on B024’s CLAHE variant, while 虔信者 still lost its first character.

The repository records these as four reproducible paths, backed by three distinct OCR engine/model stacks. perception/configs/ocr-pipelines-v1.json points to every implementation and assigns the current roles: PP-OCRv6 medium is the accuracy-first bounded-title primary, the stronger Windows recipe is a low-cost fallback candidate, EasyOCR is a regression comparator, and the old B08/B021 recipe remains frozen historical evidence for the workflow now numbered B023.

Path	Text detection and recognition	Decoding and lookup	Variants used here	Runtime
Historical B08 / old-B021 title OCR	Opaque Windows system OCR, zh-Hans-CN	Windows result → normalized game-name lookup	3× color; grayscale CLAHE retry	Windows local engine
B024 Windows baseline	Same Windows system OCR	Prefer an exact catalog name; otherwise fixed variant order	original, 4× Lanczos, 4× CLAHE, 4× Otsu	Windows local engine
PP-OCRv6 medium	PPLCNetV4 + RepLKFPN detector; PPLCNetV4 + LightSVTR recognizer	CTC/NRTR multi-head output → exact catalog lookup	original and 3× CLAHE	Paddle GPU
EasyOCR 1.7.2	CRAFT detector + Chinese generation-2 CRNN	beam search, width 10 → exact catalog lookup	original and 4× CLAHE	PyTorch CPU

Windows OCR is a useful system baseline

Windows.Media.Ocr is the local OCR API built into Windows. It returns recognized words and positions. B024 ran Simplified Chinese recognition on the original crop plus three deterministic variants: 4× Lanczos enlargement, 4× CLAHE contrast enhancement, and a 4× Otsu binary image.

Windows OCR does not expose a comparable character confidence in this bridge, so selection follows a fixed rule: prefer an output that exactly matches a Creature name or alias, then keep the earliest prepared variant. This recovered 剑士 only from the CLAHE image.

PP-OCRv6 medium is the accuracy-first candidate

PaddleOCR’s PP-OCRv6 separates text detection from text recognition. The run used PaddleOCR 3.7.0 with PP-OCRv6_medium_det and PP-OCRv6_medium_rec on the GPU. Each crop ran once as saved and once after a 3× CLAHE enhancement.

This is the largest accuracy-oriented OCR configuration registered in this project. It returned model scores and read all six failure titles exactly on both the original and enhanced passes.

EasyOCR provides a different model family

EasyOCR uses CRAFT to find text and a Chinese CRNN recognizer to turn image features into a character sequence. B024 used EasyOCR 1.7.2, beam-search decoding, and original plus 4× CLAHE crops. This isolated run used CPU.

EasyOCR is valuable here because it is structurally different from PP-OCRv6. Agreement provides useful cross-model corroboration, although their errors are not proven independent. It recovered 虔信者 but did not recover 剑士.

5. How fast were the actual programs?

The measured inputs are fixed title strips, not full screenshots. Nine strips are 412×47 pixels and two are 309×35. A normal strip contains one line and 2–5 normalized title characters; the covered tooltip produces an 11-character prose fragment. All nine complete source frames are shown above: seven are 2560×1440 and two are 1920×1078, but none of those full frames enters the B024 OCR call.

The three configurations perform different amounts of work:

B024 path	Variants per source crop	OCR calls for 11 crops	Pixels processed per batch
Windows OCR	original + three 4× variants	44	9,599,394
PP-OCRv6 medium	original + 3× CLAHE	22	1,959,060
EasyOCR 1.7.2	original + 4× CLAHE	22	3,330,402

I reran all 11 source crops three times per engine on an AMD Ryzen 9 9900X and NVIDIA RTX 5080. Every repetition starts from the original title files. For Windows and EasyOCR, the controller decodes the originals, creates the configured variants, encodes them, writes them, and then starts a fresh worker. Paddle makes its enhanced input inside the worker. The startup end-to-end column includes that complete path through JSON output. The resident recipe column adds each repeat’s parent preprocessing to the worker’s measured post-warmup batch. This is a reconstructed sum, not a directly timed persistent-service round trip. It excludes final catalog selection, response serialization and service dispatch/IPC overhead. Startup includes the controller-to-JSON path, but also excludes the later catalog selection and evaluation. Fresh-process startup uses cached model files; it does not mean flushed operating-system or disk caches.

B024 path	Parent preprocessing median	Startup end-to-end median	Resident worker median	Resident recipe median	Resident recipe source crops/s	Warm OCR p50 / p95 per attempt
Windows OCR	0.198 s	1.397 s	0.771 s	0.968 s	11.36	5.17 / 9.55 ms · n=132
PP-OCRv6 medium · GPU	inside worker	6.214 s	0.722 s	0.722 s	15.24	23.16 / 54.02 ms · n=66
EasyOCR · CPU	0.115 s	9.876 s	4.225 s	4.341 s	2.53	144.85 / 358.22 ms · n=66

These percentiles describe individual OCR attempts, not complete source-crop latency. The p50/p95 samples pool every configured OCR attempt from all three measured repetitions; warmup calls are excluded. Windows contributes 11 crops × 4 variants × 3 repeats = 132 observations. Paddle and EasyOCR each contribute 11 × 2 × 3 = 66. The directly observed median startup phases were 106.9 ms PowerShell setup + 7.7 ms Windows engine creation + 97.7 ms warmup; 2.589 s Paddle imports + 1.209 s model creation + 0.468 s warmup; and 3.165 s EasyOCR imports + 1.486 s reader creation + 0.108 s warmup. Phase medians come from separate repetitions, so they do not add exactly to the end-to-end median. The remainder includes process launch, IPC, request/output I/O, and serialization.

These timings compare the programs and configurations actually used in B024; they are not an intrinsic framework ranking. Windows makes four OCR attempts per source while Paddle makes two, so Windows has lower per-attempt latency but lower source-level throughput in this measured recipe. Paddle pays a much larger startup cost, then gives the highest resident recipe throughput. EasyOCR used CPU-only PyTorch, so its number is not an equal-hardware comparison with Paddle.

Does runtime depend on image size or how much text is present?

Both can matter, but image dimensions and variant count dominate this small diagnostic. EasyOCR’s median OCR call rose from 53.78 ms on the 412×47 original strips to 330.58 ms on their 1648×188 enlarged variants. Windows rose from 1.03 to 6.12 ms. PP-OCRv6 rose from 21.74 ms at 412×47 to 24.96 ms at 1236×141.

After detection, more text boxes and longer sequences can add recognition and decoding work. B024 cannot isolate that effect: almost every crop contains one short line, and input dimensions change at the same time. Grouped by recognized characters, PP-OCRv6 p50 was 23.11, 24.09, 22.59, and 42.00 ms for 2, 3, 5, and 11 characters. EasyOCR was 190.17, 191.93, 144.85, and 153.58 ms. The non-monotonic results mean this sample cannot support a “milliseconds per character” claim. A controlled benchmark must hold pixels, font, and preprocessing fixed while varying line and character counts.

B021 is a different workload: scene understanding sends a complete 2560×1440 frame to OCR. Its original Windows run used eight fresh 40-frame workers per 320-frame recording and took 43.370 and 43.510 seconds including layout processing, about 135.5 and 136.0 ms per frame. That v1 receipt is marked partial because rotated word rectangles failed later geometry projection; the OCR calls themselves were the input to the corrected v2 pass. V2 reused the raw OCR and yielded 28.1 and 28.7 derived text elements per frame, so its later 5.9-second projection time is not OCR speed. The PP-OCRv6 accuracy path ran every full frame twice, original plus 1.5× CLAHE, and took 210.494 and 191.235 seconds for the two 320-frame sessions, or 657.8 and 597.6 ms per source frame. The second session produced more fused text elements yet completed faster, another sign that text count is not the only cost driver.

The older B08/B021 fixed-title run provides another operational measurement: 877 frames, 2,782 title targets, and 318 contrast retries completed in 97.55 seconds, or about 28.5 title targets/s end to end. That receipt measured coverage, not accuracy, and did not separate cold and warm latency.

Are there public benchmark numbers?

There are published numbers, but no common benchmark here is directly comparable across all three packaged engines.

• Paddle’s official OCR pipeline documentation reports 86.2 detection Hmean for PP-OCRv6_medium_det and 83.2 average recognition accuracy for PP-OCRv6_medium_rec. Paddle marks both as results on an internal multi-scene evaluation set, so they are publicly reported metrics rather than independently reproducible public-test-set scores.
• Paddle’s PP-OCRv6 end-to-end speed table uses 200 general and document images and includes image loading, preprocessing, inference, and postprocessing. It reports 0.29 s/image on an NVIDIA A100, 0.72 s/image on a V100, and 2.05 s/image on an Intel Xeon 8350C with the PaddlePaddle backend for the medium model. Those full-image inputs are much larger than our title strips.
• The EasyOCR repository documents the packaged pipeline and model-loading behavior but does not publish one standardized accuracy-and-speed table for EasyOCR 1.7.2’s Simplified-Chinese CRAFT + CRNN pipeline. Its CRAFT detector and CRNN recognizer have paper benchmarks, but those component results are not a benchmark of EasyOCR’s current Chinese weights, preprocessing, and beam-search configuration.
• Microsoft’s Windows.Media.Ocr API documentation specifies the interface, installed-language model selection, and image limits. It does not publish an accuracy or latency benchmark for the installed Simplified-Chinese engine.

The production decision should therefore use a project-owned, video-separated benchmark: exact title accuracy, character error rate, invalid-title false acceptance, cold and warm p50/p95 latency, and throughput with the model kept resident. B024 is a failure-recovery diagnostic; six readable titles are too few to estimate general accuracy.

6. Which reader was most accurate?

The pipeline’s production selector prefers any variant whose normalized text exactly matches a Creature name or alias. That catalog lookup is useful system behavior, but it means the selected score is not a bare-model accuracy. The table therefore separates what each engine read from the original pixels, what its enhanced branch could supply, and what the catalog-assisted production selector returned.

Engine	Original-only normalized exact	Enhanced-variant selected exact	Production-selected exact	Any variant exact
Windows OCR	4 / 6	5 / 6	5 / 6	5 / 6
PP-OCRv6 medium	6 / 6	6 / 6	6 / 6	6 / 6
EasyOCR	5 / 6	5 / 6	5 / 6	5 / 6

Original-only is the least externally assisted comparison: each engine receives the saved crop without a B024-generated enhancement. Engines still apply their internal preprocessing, and outputs use the common scoring normalization. Enhanced-variant selected applies the same catalog-assisted selector after excluding the original attempt; Paddle and EasyOCR each have one enhanced variant, while Windows has three. Production-selected chooses across all configured variants. Any variant is an oracle diagnostic: it asks whether at least one recorded attempt equals the expected normalized string. PP-OCRv6’s 6/6 holds even in the original-only column, so its result does not depend on the catalog choosing among competing variants.

Visible title	Historical B08 / old-B021 Windows pipeline	B024 Windows OCR	PP-OCRv6	EasyOCR
弩兵	no text	弩兵 ✓	弩兵 ✓	弩兵 ✓
日矛圣骑兵	日矛圣马奇兵	日矛圣骑兵 ✓	日矛圣骑兵 ✓	日矛圣骑兵 ✓
剑士	no text	剑士 ✓	剑士 ✓	剑 ✗
日矛圣骑兵	日矛圣马奇兵	日矛圣骑兵 ✓	日矛圣骑兵 ✓	日矛圣骑兵 ✓
日矛圣骑兵	日矛圣马奇兵	日矛圣骑兵 ✓	日矛圣骑兵 ✓	日矛圣骑兵 ✓
虔信者	信者	信者 ✗	虔信者 ✓	虔信者 ✓

Engine	Production-selected normalized exact match	Remaining error
PP-OCRv6 medium	6 / 6	none
Windows OCR with fixed preprocessing	5 / 6	虔信者 → 信者
EasyOCR with beam search	5 / 6	剑士 → 剑; its original pass returned 剑土

These six crops were selected because the old system failed on them, and they represent only four distinct strings. The table measures recovery on an exploratory diagnostic set, not general OCR accuracy. It supports one local decision: bounded title reading should use PP-OCRv6 medium as the primary recognizer. Windows remains a low-cost, structurally different fallback candidate; this set contains no case where Windows rescues a Paddle miss, so incremental fallback recall is still unmeasured.

All three current engines produced a Creature-route text+catalog rejection on four of the five negative controls. Each failed on the same fifth case: 深渊监管者 is valid Creature text, but the paired image is not a valid target. This rejection count and readable-title exact match remain separate because they measure different tasks.

Exact text is then resolved through the current game catalog rather than copied directly into an entity field. This matters for two of the repaired cases: the rendered title 日矛圣骑兵 is an alias of 阳矛骑兵 / Sunspear Cavalry, not the old visual fallback 贵族骑兵 / Noble Cavalry; 虔信者 is an alias of 誓信徒 / Votary. The catalog resolver also maps the public name 剑士 to its stable internal creature:esquire identifier.

Figure 2. The earlier queue mixed OCR failure with downstream image suggestions. The new comparison scores the title transcription before considering image identity.

7. Why correct text can still be invalid

The tooltip crop contains prose instead of a creature name. All three engines read some of that prose and correctly failed to match the Creature catalog. The three spell-slot negatives tell a subtler story:

Region sent to the Creature route	Windows OCR	PP-OCRv6 score	EasyOCR score	Creature-route decision
绝望 spell region	绝望 · no score	绝望 · 0.9999	绝望 · 0.8931	reject
狂暴 spell region	no text · no score	狂暴 · 1.0000	狂暴 · 0.6904	reject
顺风 spell region	顺风 · no score	顺风 · 0.9999	顺风 · 0.9732	reject

These three rows are OCR successes. The rejection means that a spell region was sent to the Creature-title route; it does not mean that 绝望, 狂暴, or 顺风 was transcribed incorrectly. The text is useful because it reveals the upstream scene-to-layout routing mistake.

Paddle and EasyOCR expose recognition confidence scores, but these values are not calibrated probabilities that the text is correct. A value of 0.99 does not establish a 99% real-world correctness rate. Scores depend on the model, decoder, text length and preprocessing, and scores from the two engines are not directly comparable. The Windows bridge used here returns words and positions without a confidence value.

Figure 3. Reading 绝望, 狂暴, and 顺风 correctly should redirect or reject the frame, not generate Creature labels.

The final negative is even more important. Every engine reads 深渊监管者 exactly, and the Creature catalog recognizes it as an alias. PP-OCRv6 scores the selected reading at 0.9998 and EasyOCR at 0.9945. The reviewer nevertheless confirmed that the paired image crop is not a valid target. An OCR-plus-catalog gate would accept this item under every tested engine. The covered tooltip is another warning: PP-OCRv6 gives its prose reading a high 0.9830 score, even though that crop cannot reveal the hidden Creature title.

This is why B024 keeps two decisions. Text exactness measures the recognizer. Current frame and target acceptance combine scene compatibility, expected layout, title visibility, crop quality, an exact catalog match, title-to-image geometry and visual identity evidence. A future sequence layer may add temporal consistency after it is independently evaluated. With OCR and a Creature catalog alone, four of the five negative cases are rejected; the fifth is a false acceptance.

Figure 4. When the expected title region contains tooltip prose, waiting for a later frame is safer than forcing a Creature identity.

8. The production decision

For the current project, PP-OCRv6 medium is our provisional primary bounded-title reader while the broader production benchmark is pending. It is the only current path that recovered all six readable failures from the original pixels, and its 15.24 source crops/s resident recipe throughput was the best of the three measured recipes. Windows starts much faster and uses a distinct system engine, so it remains a low-cost fallback candidate. This diagnostic does not show an actual Paddle failure rescued by Windows; that value must be measured on a larger video-separated benchmark. EasyOCR remains an offline regression comparator and was much slower in this CPU-only run.

The production title path should use these steps:

1. Confirm that the frame’s scene and local layout permit Creature slots.
2. Reject or defer crops that are faded, covered, or not valid image targets.
3. Run bounded PP-OCRv6 medium on the original and enhanced title crop.
4. Preserve raw text and model scores, then resolve only exact names or aliases.
5. Evaluate Windows OCR as a fallback when Paddle is unresolved, while measuring whether it adds recall rather than assuming it does.
6. Treat the image classifier as supporting evidence with its own validity and confidence gates.
7. Retry a later video frame when title or image evidence is blocked.

A future end-to-end gate should abstain unless the available signals form a compatible explanation. A high OCR score can strengthen a text hypothesis, but it cannot override a scene mismatch, an occluded title or an invalid neighboring image crop. Temporal consistency should remain a separate signal until the sequence-level tracking layer is independently evaluated.

The reproducible experiment lives in perception/experiments/b024_bounded-title-ocr-benchmark/. The compact JSON receipt records every source filename, engine and package version, image variant, raw output, score, catalog match, and final comparison. Full generated variants and raw engine files stay in ignored project_data/; the small title fixtures and compact evidence stay in Git.

The next decision-grade experiment should prospectively freeze 100–300 title opportunities from recordings separated from development. It should include ordinary readable titles, low-contrast and partially obscured titles, wrong-scene routes, invalid image targets, and temporal retries. The report should show raw OCR, production-selected OCR plus preprocessing, final validity decisions, normalized exact match, character error rate, false accepts and rejects, and end-to-end warm p50/p95 with every preprocessing step charged to its recipe.