ACM SIGKDD 2026 · Jeju Island, Korea

SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark with Graph-Enhanced Data Mining

Yi Cui, Zilin Wang, Yijie Xu, Qianyi Cai, Huizai Yao, Shuai Jiang, Bingzhuo Zhong, Hui Xiong

A construction-safety benchmark mined from real inspection archives, designed to evaluate multimodal LLMs under temporal and site variation instead of static curated imagery.

Paper PDF arXiv ACM DOI Dataset Code Poster BibTeX

Paper Overview

Construction safety is data-rich, but information-poor.

Construction-safety models must handle concrete deployment risks, such as workers near scaffold edges without guardrails, rather than only recognize common objects in curated images. Real inspection archives are redundant, long-tailed, and collected across changing sites and months.

SafeBuild-Bench is mined from 100K+ industrial image-text records and contains 3,314 task instances, each backed by its own expert-verified image. It covers multiple-choice hazard identification and free-form hazard description, while retaining temporal and site metadata for stratified robustness analysis.

The benchmark spans five collection months, July to November 2025, and every instance keeps its capture month. Month-stratified scores can therefore be reported alongside the aggregate number. Fixed models vary materially across those slices, which is the argument for reporting them; the released metadata supports that analysis rather than a strict forward-chaining or held-out-site protocol.

Illustration of common low-risk records and rare long-tail construction hazards — Large inspection archives contain many repeated low-risk records, while safety-critical hazards form a sparse long tail that deployment systems must still understand.

Curation Pipeline

GEMS mines informative candidates before expert verification.

GEMS combines proxy-model confusion with graph-based diversity. The objective is not to auto-label hazards, but to surface a compact, non-duplicate review queue from redundant industrial streams.

Graph Construction

Fuse image-text records into multimodal embeddings and connect nearby scenes through a sparse k-NN graph.

Uncertainty Signal

Use a proxy MLLM's normalized likelihood as a confusion score for potentially informative samples.

Lazy Greedy Selection

Apply a CELF-based submodular objective to balance high-value hazards against redundancy saturation.

Overview of the GEMS graph-enhanced multimodal selection framework — GEMS selects an informational core for scalable benchmark construction.

Benchmark

Two tasks, one deployment-oriented safety setting.

The benchmark keeps month and site metadata so evaluation can be reported as aggregate scores and as deployment-relevant slices.

2,200 instances

Hazard Identification

Multiple-choice questions ask models to select the primary hazard from visually and semantically similar categories.

Metrics: Accuracy and Macro-Recall
Stress: Fine-grained category discrimination

1,114 instances

Hazard Description

Free-form prompts evaluate whether models detect hazards and provide specific, grounded safety descriptions.

Metrics: Hazard Detection Rate and Description Quality
Scoring: Fixed GPT-4o rubric-based judge with audit evidence

SafeBuild-Bench category distribution across five safety domains and nineteen fine-grained categories — SafeBuild-Bench spans five safety domains and nineteen fine-grained hazard categories.

Examples

Samples expose subtle safety-rule failures.

Orderly construction site scene with no primary hazard

MCQ

No Hazard

The site is orderly, with fire extinguishers properly stored and accessible.

MCQ

Safety Management

The scene tests whether models notice missing warning signs and covered equipment risks.

Description

Partial Match

A model can detect hazards but miss key reference objects needed for a high-quality explanation.

Construction hazard scene where a model missed the hazard

Description

Missed Hazard

The model incorrectly reports no visible hazard despite gaps in safety netting.

Evaluation

Current MLLMs remain far from reliable construction-safety understanding.

The best overall score is near 60, and strong models differ sharply across identification, description, and month-level slices.

Main SafeBuild-Bench results as published. The two description columns were scored before the reference correction described under Release; identification reproduces exactly on the released data.
Model	Identification Acc.	Macro-Recall	HDR	Desc. Quality	Overall
Kimi-K2.5	48.7	67.7	72.6	53.7	60.7
Gemini-3-Flash-Preview	55.8	65.8	67.6	48.4	59.4
Qwen3-VL-Plus	40.3	53.7	79.9	62.5	59.1
Claude-4.5-Sonnet	48.3	56.2	64.4	50.6	54.9
GPT-4o	34.3	43.8	66.5	50.7	48.8

Radar chart comparing category-wise hazard identification accuracy for representative models — Category-wise identification accuracy varies substantially across hazards and model families.

Release

The benchmark, the mining code, and the evaluation protocol are public.

The dataset is released under CC BY 4.0. Instance IDs in the release match the evaluated sets behind the paper's tables exactly.

Benchmark Data

3,314 images with hazard categories, reference descriptions, judge criteria, capture month, and split labels. Hugging Face imagefolder layout.

GEMS Codebase

Graph-enhanced mining for candidate selection from redundant multimodal inspection streams, plus the MCQ and description evaluation runners.

Reference Correction

196 November description references were regenerated from their Chinese expert annotations after publication. The released data is corrected; the paper's description scores are not, and will not reproduce. Four instances remain unscorable and are listed in the patch file.

Intended Use

Safety-focused evaluation and improvement. Not for biometric identification, worker surveillance, or automated disciplinary decisions.

Citation

BibTeX

@inproceedings{cui2026safebuild,
  author    = {Cui, Yi and Wang, Zilin and Xu, Yijie and Cai, Qianyi and
               Yao, Huizai and Jiang, Shuai and Zhong, Bingzhuo and Xiong, Hui},
  title     = {SafeBuild-Bench: A Temporal-Robust Construction Safety Benchmark
               with Graph-Enhanced Data Mining},
  year      = {2026},
  isbn      = {979-8-4007-2259-2},
  publisher = {Association for Computing Machinery},
  address   = {New York, NY, USA},
  url       = {https://doi.org/10.1145/3770855.3817581},
  doi       = {10.1145/3770855.3817581},
  booktitle = {Proceedings of the 32nd ACM SIGKDD Conference on Knowledge
               Discovery and Data Mining V.2},
  numpages  = {12},
  location  = {Jeju Island, Republic of Korea},
  series    = {KDD 2026}
}

An open-access preprint is available at arXiv:2608.00068.