Diagnosing Deep Self-localization Network: Accelerated Model-free Subimage-level Diagnosis via Image Synthesis and Relevance Feedback

Abstract

Deep convolutional neural network (DCN) has become a common approach in visual robot self-localization. In a typical self-localization system, a DCN is trained as a visual place classifier from past visual experiences in the target environment. However, its classification performance can be deteriorated when it is tested in a different domain (e.g., times of day, weathers, seasons), due to domain shifts. Therefore, an efficient domain-adaptation (DA) approach to suppress per-domain DA cost would be desired. In this study, we address this issue with a novel ``domain-shift localization (DSL)" technique that diagnosis the DCN classifier with the goal of localizing which region of the robot workspace is significantly affected by domain-shifts. In our approach, the DSL task is formulated as a fault-diagnosis (FD) problem, in which the deterioration of DCN-based self-localization for a given query image is viewed as an indicator of domain-shifts at the imaged region. In our contributions, we address the following non-trivial issues: (1) We address a subimage-level fine-grained DSL task given a typical coarse image-level DCN classifier, in which the target DCN system is queried with a region-of-interest (RoI) masked synthesized query image to diagnosis the RoI region; (2) We extend the DSL task to a relevance feedback (RF) framework, to perform a further query and return improved diagnosis results; and (3) We implement the proposed framework on 3D point cloud imagery -based self-localization and experimentally demonstrate the effectiveness of the proposed algorithm.