K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation


What we have here is an article entitled K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. Its authors are Michal Aharon, Michael Elad, and Alfred Bruckstein and it was taken from IEEE TRANSACTIONS ON SIGNAL PROCESSING,it has been published on 11, NOVEMBER 2006. First of all we are going speak of the context of the work and This part will explain the scientific context of this article then in another section called Positioning we will detail what are the existing works on the subject of designing Dictionaries. For the tow others sections Contributions and Experiments we are going to present the contribution,the experiments and the validations that are shown in the article. The last section concludes this works.

Context of the work

In brief We can consider that sparse representation is the solution of one of these two equations (1) and (2):

  • D: overcomplete dictionary matrix

  • y: the signal

  • x: the vector of the representation coefficients of the signal

Sparse representation has attracted much attention from researchers in fields of signal processing, image processing, computer vision, and pattern recognition. It also has a good reputation in both theoretical research and practical applications (Zhang 2015).

Similarly, this article gives clear information about many applications of the K-SVD Algorithm including ”compression, regularization in inverse problems, feature extraction(Aharon 2006), and more. To put it in a nutshell, the paper come up with a totally new algorithm based on adjusting dictionaries used for the sparse signal representations. Actually, the K-SVD is a combined method that presents in one hand sparse coding and on the other hand a technique of customizing the dictionary to get the best and the most adequate representation of the data. That last feature, as a matter of fact, allows the dictionary to adapt to different types of data.


To deal with this section we have to divide it into tow parts SPARSE CODING and DESIGN OF DECTIONARIES. In the first place, the sparse coding is a fundamental stage in the K-SVD method and as a consequence many pursuit algorithms have been proposed, such as ”orthogonal matching poursuit (OMP) algorithm, the basis poursuit (BP) and The focal underdetermined system solver (FOCUSS)”. Then, the DESIGN OF DICTIONARIES represent the main topic of the paper. In fact, ”the K-SVD algorithm is a generalization of the K-means algorithm”(Aharon 2006), which is the most effective used technique. As a result this algorithm is used when we have to cope with the problem of dictionary updating. We also have the ”Maximum Likelihood Methods, The MOD Method, the Maximum A-Posteriori Probability Approach and the Unions of Orthonormal Bases”. Finally we can assume that all methods presented in this article is a generalizations of the K-means algorithm, however, there are many differences between them. And as cited in the article we need to respect some characteristics to get a great dictionary training algorithm:

  • Flexibility

  • Simplicity

  • Efficiency

  • Well-Defined Objective

(Aharon 2006)


We already defined The K-SVD as a combination of sparse coding and an update process for the dictionary. And We use this definition to deduce that this method is the best way to get the best sparse representation. Furthermore We can easily notice that this algorithm is fast due to using a method that accelerate the convergence. Moreover the K-SVD algorithm is flexible and capable to work with any pursuit method.


To illustrate the contribution of this new algorithm, the article shows the results of many experiments. It’s obvious that they use tow types of experiments. The first one is SYNTHETIC EXPERIMENTS, where they experiment different algorithms (K-SVD, MOD and MAP-based) on synthetic signals testing the efficiency of these methods. The second type of experiments is the application to real data so they did two APPLICATIONS TO IMAGE PROCESSING, Filling In Missing Pixels and Compression. In light of all the experiments we have a better understanding of the K-SVD algorithm and we are convinced that this method has better results than the other algorithms.


Above all, it seems pertinent to remember that in this article, the presented K-SVD algorithm is based on generating over-complete dictionaries in such a way that train and reconstruct these matrices. And all of the work is illustrated with experiments such as filling in missing pixels and compression. All things considered, it seems reasonable to assume that the K-SVD algorithm beat similar methods and alternatives ways, that’s why it may successfully take over the popular representation methods both in ”image enhancement and in compression”(Aharon 2006). But still a future work required to improve this algorithm.