Overcomplete Basis for Sparse Coding?

An overcomplete basis has the number of basis vectors greater than the dimensionality of the input, and the representation of an input is not a unique combination of basis vectors. Overcomplete representations have been advocated because they have greater robustness in the presence of noise, can be sparser, and can have greater flexibility in matching structure in the data. Overcomplete codes have also been proposed as a model of some of the response properties of neurons in primary visual cortex. Overcomplete bases is expected to yield better approximation of the underlying statistical distribution of the data and can thus lead to greater coding efficiency. This can be viewed as a generalization of the technique of independent component analysis and provides a method for Bayesian reconstruction of signals in the presence of noise and for blind source separation when there are more sources than mixtures. Initially the technique might seem to contradict with the concept of sparseness due to the increased number of analytical coefficients. But the increased dimensions produces extra degrees of freedom which can be exploited to increase the sparsity of the set of coefficients. Thus only those coefficients that deemed significant are evident in the representation. This type of coding forms the basis for what is called the 'Holy Grail' of Audio coding called the Object coding which can be seen as a footstep towards musically intelligent acoustic coding.