The long transmission distance in wireless power transmission(WPT) system will lead to a sharp decline in transmission efficiency, this paper presents a novel ortho-octagonal double helix metamaterial that can effectively enhance the transmission efficiency of WPT systems, theoretically derived and experimentally verified the enhancement of evanescent wave transmission by this metamaterial dielectric plate. A WPT simulation system operating at 10.78 MHz is constructed by HFSS software, and the effects of adding passive relay coils and different shapes of metamaterials on the transmission efficiency are comparatively investigated. Both simulation and experimental results show that the transmission efficiency of the WPT system can be improved to different degrees after inserting different shapes of metamaterials at the transceiver end of the system. When the transceiver coil spacing of the WPT system is greater than 22 cm, the effect of metamaterials on the system transmission efficiency enhancement shows an increasing and then decreasing trend as the distance increases. The positive octagonal metamaterial improves 12% over the circular metamaterial in the mid-range transmission distance due to its better magnetic coupling effect. When the transceiver coil spacing is 22 cm, the open-circuit voltage of the receiving coil after loading the metamaterial is stabilized to be enhanced by nearly 5 times, and it has the best effect at 30 cm, and the transmission efficiency is enhanced from 30% to 60%, which verifies the effective enhancement of the metamaterial on the transmission efficiency of the WPT system.