This document provides a formal proof and supple- mentary information of the paper: Class-Incremental Learning for Wireless Device Identification in IoT. The original paper focuses on providing a novel and efficient incremental learning algorithm. In this document, we explicitly explain why the mem- ory representations (latent device fingerprints in our application) in Artificial Neural Networks approximate orthogonality with insights for the invention of our Channel Separation Incremental Learning algorithm.