Abstract
We propose Conditional Value-at-Risk (CVaR) investment agents to solve
the problems of single asset trading and assets allocation under the
Direct Reinforcement Learning framework. We propose two convex
CVaR-based agents, the CVaR-constrained and the unconstrained
CVaR-sensitive. Convexity allows conveniently implementing incremental
learning, leading to an adaptive investing agent at an efficient
computational cost with the merit of guaranteed policy convergence. Our
experiments with frictional investment under various markets reveal the
CVaR-constrained potency in improving investment return per unit of
risk. The unconstrained CVaR-sensitive agent, on the other hand,
exhibits robustness in handling intense market pullbacks, with both
CVaR-based agents showing superior risk management to a risk-insensitive
one. Our approach also showed superiority over state-of-the-art methods,
demonstrating the potential of CVaR-based RL investment models. We
finally show how our agents are extendable to learn investing under the
most general investment problem of optimizing a multi-asset portfolio.