Robot Perception for Fully Automated Industrial Systems - A Review of Sensing Modalities, Learning Methods, and Open Challenges

Abstract

Complete automation of industrial processes, often described as “lights-out” manufacturing, requires robots that can sense, interpret, and act upon unstructured and changing environments without human supervision. Perception has therefore become the principal bottleneck separating today’s largely preprogrammed industrial robots from truly autonomous production cells. This paper reviews the state of the art in robot perception for industrial automation. We first survey the sensing modalities in use, including 2D machine vision, active and passive 3D sensing, force/torque and tactile sensing, and their fusion. We then examine the core perception tasks that enable autonomy: object detection and segmentation, 6D pose estimation, grasp detection and bin picking, visual inspection and anomaly detection, contact-rich assembly, perception for safe human–robot collaboration, calibration, and localization for mobile manipulators. We further discuss the learning paradigms that underpin recent progress, namely simulation-to-reality transfer, synthetic data, self-supervision, and vision–language foundation models, together with system integration through ROS 2 and digital twins. Finally, we identify open challenges, including transparent and reflective objects, deformable materials, data scarcity, uncertainty quantification, certification of learned components, and benchmark gaps, and outline research directions toward reliable, fully autonomous industrial robotic systems.