Advances in Stochastic Gradient and Coordinate Descent Methods
主 题: Advances in Stochastic Gradient and Coordinate Descent Methods
报告人: Dr. Lin Xiao (Microsoft Research )
时 间: 2015-06-25 14:00 - 15:00
地 点: 北京国际数学研究中心全斋-29教室
Stochastic gradient and coordinate descent methods are two classical methods in continuous optimization, each with a long and rich history. They are among the most efficient optimization algorithms for solving modern large-scale machine learning problems, mainly due to their very low computational cost per iteration (typically only process one or few examples or coordinates at a time). Their slow convergence rates usually are not a big concern for machine learning problems because of the training-generalization tradeoff. I will present some recent progresses on new variants of stochastic gradient and coordinate descent methods for minimizing the average of a large number of simple convex functions. By exploiting the finite average structure, these algorithms achieve order of magnitude faster convergence rates than the classical methods, while maintaining the same low complexity per iteration. These advances significantly speed up machine learning practice with big data applications.
