Abstract
Learning through tests is a broadly used methodology in human learning
and shows great effectiveness in improving learning outcome: a sequence
of tests are made with increasing levels of difficulty; the learner
takes these tests to identify his/her weak points in learning and
continuously addresses these weak points to successfully pass these
tests. We are interested in investigating whether this powerful learning
technique can be borrowed from humans to improve the learning abilities
of machines. We propose a novel learning approach called learning by
passing tests (LPT). In our approach, a tester model creates
increasingly more-difficult tests to evaluate a learner model. The
learner tries to continuously improve its learning ability so that it
can successfully pass however difficult tests created by the tester. We
propose a multi-level optimization framework to formulate LPT, where the
tester learns to create difficult and meaningful tests and the learner
learns to pass these tests. We develop an efficient algorithm to solve
the LPT problem. Our method is applied for neural architecture search
and achieves significant improvement over state-of-the-art baselines on
CIFAR-100, CIFAR-10, and ImageNet.