PointProgrammerNet: An additive, coordinate-addressed state for point-cloud classification and segmentation

Abstract

A point cloud is rarely delivered all at once: two devices see different parts of a scene, one of them later turns out to be mis-calibrated, and a region is chosen for labelling after the observations are over. This paper asks what a network should keep instead of the points, and answers it for part segmentation and object classification. PointProgrammerNet stores a sum: each point is mapped to a distribution over learned spatial addresses and contributes a second-order value as an outer product, and three fixed 128 × 170 matrices hold the accumulated products. Nothing that couples one point to another enters the write, so normalization across accumulated observations and nonlinear interpretation of the accumulated statistics are deferred to the read, and what is stored admits arithmetic that a pooled or normalized descriptor does not. Two states add exactly, so several devices can encode locally, exchange a fixed-size matrix, and recover the centralized result. A state can be subtracted, so an unreliable source is withdrawn without re-encoding the rest, and scaling by a retention factor weights evidence by age. Because the sum carries spatial addresses instead of collapsing into one vector, the state also defines a segmentation field over the whole coordinate space: when read at labelled coordinates it never wrote, it remains within 0.05 instance mIoU of the coordinates it did write. The model reaches 82.58% instance mIoU on ShapeNetPart with 40.3% fewer parameters than a lightweight PointNet++ control and 86.59% on ModelNet40 from the state alone, while merge, removal, and attenuation match direct reconstruction to within 10⁻⁶. These results show that useful recognition accuracy can be retained in a fixed-size state that supports exact merging, removal, and attenuation after the points are released.