In the lecture I will introduce a new approach to the main problems of Statistical Learning Theory: pattern recognition, regression estimation, and density estimation. I will introduce the so-called direct approach which requires solving operator equations that define the desired functions. The solutions of these equations are based on solving stochastic ill-posed problems. To solve them effectively I will combine ideas that were originated within three different branches of mathematics: the theory of ill-posed problems, classical non-parametric statistics, and statistical learning theory. In the lecture I will introduce the necessary results from all three branches and combine corresponding techniques to obtain a new type of algorithms that are well founded theoretically, computationally effective, and have no free parameters.