Dimensional Reduction Work Group

Please download the toolbox and pdf from here. While by no means “officially" the best toolbox out there, it is a single command prompt that can call on several different dimensional reduction techniques. This way we can spend less time on writing code, and more time on understanding what the logic of the different methods are. To that end, I will be following this pdf . I think it does a very good job of explaining the mathematics.

Introduction

Manifold learning is a significant problem across a wide variety of information processing fields including pattern recognition, data compression, machine learning, and database navigation. In many problems, the measured data vectors are high-dimensional but we may have reason to believe that the data lie near a lower-dimensional manifold. In other words, we may believe that high-dimensional data are multiple, indirect measurements of an underlying source, which typically cannot be directly measured. Learning a suitable low-dimensional manifold from high-dimensional data is essentially the same as learning this underlying source. Dimensionality reduction1 can also be seen as the process of deriving a set of degrees of freedom which can be used to reproduce most of the variability of a data set. Consider a set of images produced by the rotation of a face through different angles. Clearly only one degree of freedom is being altered, and thus the images lie along a continuous one- dimensional curve through image space. Figure 1 shows an example of image data that exhibits one intrinsic dimension.

A canonical dimensionality reduction problem from visual perception. The input consists of a sequence of 4096-dimensional vectors, representing the brightness values of 64 pixel by 64 pixel images of a face. Applied to N = 698 raw images. The first coordinate axis of the embedding correlates highly with one of the degrees of freedom underlying the original data: left-right pose.