TY - JOUR

T1 - Speeding up the convergence of the alternating least squares algorithm using vector ε acceleration and restarting for nonlinear principal component analysis

AU - Kuroda, Masahiro

AU - Mori, Yuichi

AU - Iizuka, Masaya

N1 - Publisher Copyright:
© 2022, The Author(s), under exclusive licence to Springer-Verlag GmbH Germany, part of Springer Nature.

PY - 2022

Y1 - 2022

N2 - Principal component analysis (PCA) is a widely used descriptive multivariate technique in the analysis of quantitative data. When applying PCA to mixed quantitative and qualitative data, we utilize an optimal scaling technique for quantifying qualitative data. PCA with optimal scaling is called nonlinear PCA. The alternating least squares (ALS) algorithm is used for computing nonlinear PCA. The ALS algorithm is stable in convergence and simple in implementation; however, the algorithm tends to converge slowly for large data matrices owing to its linear convergence. Then the vε-ALS algorithm, which incorporates the vector ε accelerator into the ALS algorithm, is used to accelerate the convergence of the ALS algorithm for nonlinear PCA. In this paper, we improve the vε-ALS algorithm via a restarting procedure and further reduce its number of iterations and computation time. The restarting procedure is performed under simple restarting conditions, and it speeds up the convergence of the vε-ALS algorithm. The vε-ALS algorithm with a restarting procedure is referred to as the vεR-ALS algorithm. Numerical experiments examine the performance of the vεR-ALS algorithm by comparing its number of iterations and computation time with those of the ALS and vε-ALS algorithms.

AB - Principal component analysis (PCA) is a widely used descriptive multivariate technique in the analysis of quantitative data. When applying PCA to mixed quantitative and qualitative data, we utilize an optimal scaling technique for quantifying qualitative data. PCA with optimal scaling is called nonlinear PCA. The alternating least squares (ALS) algorithm is used for computing nonlinear PCA. The ALS algorithm is stable in convergence and simple in implementation; however, the algorithm tends to converge slowly for large data matrices owing to its linear convergence. Then the vε-ALS algorithm, which incorporates the vector ε accelerator into the ALS algorithm, is used to accelerate the convergence of the ALS algorithm for nonlinear PCA. In this paper, we improve the vε-ALS algorithm via a restarting procedure and further reduce its number of iterations and computation time. The restarting procedure is performed under simple restarting conditions, and it speeds up the convergence of the vε-ALS algorithm. The vε-ALS algorithm with a restarting procedure is referred to as the vεR-ALS algorithm. Numerical experiments examine the performance of the vεR-ALS algorithm by comparing its number of iterations and computation time with those of the ALS and vε-ALS algorithms.

KW - Acceleration of convergence

KW - Alternating least squares algorithm

KW - Nonlinear principal component analysis

KW - Restarting procedure

KW - Vector ε algorithm

UR - http://www.scopus.com/inward/record.url?scp=85129230882&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85129230882&partnerID=8YFLogxK

U2 - 10.1007/s00180-022-01225-4

DO - 10.1007/s00180-022-01225-4

M3 - Article

AN - SCOPUS:85129230882

SN - 0943-4062

JO - Computational Statistics

JF - Computational Statistics

ER -