Class integration testing is an essential issue in software integration testing, and different class integration test order significantly impact the cost of testing. The Class Integration Test Order (CITO) problem is to find an optimal order of class integration test order to reduce the cost of software testing. The existing approaches tend to fall into local optimality when applied to complex systems and fail to achieve a better test order. This paper proposes a CITO generation approach based on deep reinforcement learning Categorical Double DQN (CDDQN) to address this limitation. The process uses the continuous interaction of the agent with the environment generated by inter-class dependencies to learn valuable experience and eventually obtain the optimal class integration test order. Experiments are conducted in eight systems to compare with graph-based, search-based, and reinforcement learning-based approaches. The experimental results show that the approach proposed in this paper can find CITO with lower stubbing complexity for most systems.