Multi-view context learning for human-object interaction detection

Understanding how humans interact with objects in complex visual scenes is fundamental to human-centric artificial intelligence. Yet, current Human-Object Interaction (HOI) detection frameworks often falter when faced with visual occlusions, vague interaction semantics, and scenarios that extend beyond predefined categories. We present Multi-View Context-Aware Disentangled Interaction Network (MCD-Inet), a unified framework designed to robustly detect HOIs under these challenging, open-world conditions. Our approach integrates a novel Multi-View Context Fusion mechanism that synthesizes fine grained spatial cues and broad semantic information across instance, interaction, and global levels. To decouple the learning of object localization from interaction semantics, we introduce a Disentangled Interaction Decoder, enabling clearer task specific representations. Complementing this, our Bilateral Knowledge Adaptation module bridges high-resolution spatial features from the visual backbone with abstract semantics from a pre-trained vision language model. To overcome the challenge of occluded interactions, we design an Occlusion-Robust Enhancement module, which extrapolates missing regions and mines contextual cues to recover interaction evidence. Finally, MCD-INet leverages multi-modal prompts incorporating both textual templates and visual exemplars to extend its detection capabilities to unseen HOI triplets. Extensive evaluations on HICO-DET, V-COCO, and the HICO-Ambiguous datasets confirm the superiority of our approach, achieving a new state-of-the-art mAP of 40.7% and demonstrating significant resilience in occluded and zero-shot scenarios. MCD-INet offers a powerful step toward generalizable, context-aware HOI detection in real-world visual environments.

Hali tarjima qilinmagan