Best rank-k approximations for tensors: generalizing Eckart–Young

Abstract

Given a tensor f in a Euclidean tensor space, we are interested in the critical points of the distance function from f to the set of tensors of rank at most k, which we call the critical rank-at-most-k tensors for f. When f is a matrix, the critical rank-one matrices for f correspond to the singular pairs of f. The critical rank-one tensors for f lie in a linear subspace , the critical space of f. Our main result is that, for any k, the critical rank-at-most-k tensors for a sufficiently general f also lie in the critical space . This is the part of Eckart–Young Theorem that generalizes from matrices to tensors. Moreover, we show that when the tensor format satisfies the triangle inequalities, the critical space is spanned by the complex critical rank-one tensors. Since f itself belongs to , we deduce that also f itself is a linear combination of its critical rank-one tensors.