Novel first order optimization classification framework

Numerous scientific and engineering fields extensively utilize optimization techniques for finding appropriate parameter values of models. Various opt imization methods are available for practical use. The optimization algorit hms are classified primarily due to the rates of convergence. Unfortunately , it is often the case in practice that the particular optimization method with specified convergence rates performs substantially differently on dive rse optimization tasks. Theoretical classification of convergence rates the n lacks its relevance in the context of the practical optimization. It is t herefore desirable to formulate a novel classification framework relevant t o the theoretical concept of convergence rates as well as to the practical optimization. This article introduces such classification framework. The pr oposed classification framework enables specification of optimization techn iques and optimization tasks. It also underlies its inherent relationship t o the convergence rates. Novel classification framework is applied to categ orizing the tasks of optimizing polynomials and the problem of training mul tilayer perceptron neural networks.