This paper presents a new framework for improving the performance of K-Nearest-Neighbor (KNN) classification models. Localized models, such as KNN, have advantages over globalized models such as Support Vector Machines (SVM’s) when the distribution of the data is not known, but are rarely used in practice because of their drawbacks. This framework overcomes some of the drawbacks of KNN classifiers including sensitivity to irrelevant features and disgregard for feature importance. It presents a new method for increasing the revelance of the feature space, removing noise, and introducing feature weights for the KNN classifier, using a class of models known as unsupervised feature learning as well as a gradient-based feature selection and weighting method. Results on various benchmark datasets are presented and it is shown how this approach outperforms global and local models by themselves as well as previous approaches to combine the two strategies.