OpenNUMA: A Custom Shared System-Memory Allocator for Heterogeneous Neural-Network Computing Across CPU and Integrated GPU

Abstract

Neural network and machine-learning workloads commonly rely on dedicated GPUs with tensor cores, While existing systems with integrated GPUs (iGPUs) may have unused computing power. This study investigates whether a custom memory allocator can enable existing or older systems to utilize integrated GPUs for such workloads without requiring hardware-level modification. The proposed
method does not require unified virtual memory mappings.

OpenNUMA (Open Source Neural Unified Memory Allocator) is a custom allocator developed for these tasks. Although OpenNUMA proposed to platform-independent, in this study it relies on PyTorch XPU backend, Intel oneAPI SYCL and Shared Unified Shared Memory (USM), and Microsoft Windows.
OpenNUMA provides a memory-management layer that allows CPU and iGPU workloads to access shared system-memory allocations through the parent application interface. This prototype implements memory pooling, allocation tracking, deallocation tracking, and thread safety using mutex locking.

Feasibility testing successfully shows OpenNUMA initialization, iGPU tensor allocation, tensor operations, and tensor aliasing through PyTorch. CPUto-iGPU transfer testing achieved a 34.076% reduction in transfer time, while iGPU-to-CPU transfer time increased by 5.258%. General workload testing showed minimal performance differences for iGPU and neural-network workloads compared to default allocator.

The performance of OpenNUMA is nearly identical and for some cases improved compared to default allocator. Although the results can vary due to different systems, OpenNUMA shows such allocator can exist and perform effectively. However, CPU-iGPU shared tensor access could not be verified through
the current PyTorch allocator interface.

Future work will investigate memory-release mechanisms, improved synchronization, and queue tracking.