Using computer simulations,  we  analyze the bit-error rate (BER) characteristics of a high-order QAM  optical transmission system repeated by two-mode phase-sensitive optical parametric amplifiers (PSAs).  In this  system, an idler, which is a phase-conjugated twin of a signal, propagate through the link together with the signal. One advantage of this system is that the  noise figure of two-mode PSAs used as repeaters can approach 0 dB. The other advantage is that  two-mode PSAs cancel out self-phase modulation of the signal at each repeater through the interaction between the transmitted  signal and idler. Owing to these advantages, the BER characteristics  of 16, 64, and 256QAM signals are significantly improved compared with the system using erbium-doped fiber amplifiers and phase-conjugation twin waves, provided that dispersion-shifted fibers are employed for transmission.