This image shows Ingo Steinwart

Ingo Steinwart

Univ.-Prof. Dr. rer. nat.

Professor, Head of Institute
Institute for Stochastics and Applications
Chair for Stochastics

Contact

+49 711 685 65388
Website

Pfaffenwaldring 57
70569 Stuttgart
Germany
Room: 8.544

Office Hours

Please contact me by E-Mail

Subject

  • Statistical Learning Theory
  • Kernel-based learning algorithms
  • Cluster Analysis
  • Neural Networks
  • Efficient learning algorithms for large data sets
  • Loss functions and their risks
  • Learning from non .i.i.d. data
  • Applications of machine learning
  • Reproducing kernel Hilbert spaces

Look here for short descriptions of these subjects and our corresponding publications.

  1. 2021

    1. Steinwart, I., Fischer, S.: A Closer Look at Covering Number Bounds for Gaussian Kernels. J. Complexity. 62, 101513 (2021). https://doi.org/10.1016/j.jco.2020.101513.
    2. Hamm, T., Steinwart, I.: Adaptive Learning Rates for Support Vector Machines Working on Data with Low Intrinsic Dimension. Ann. Statist. 49, 3153--3180 (2021). https://doi.org/10.1214/21-AOS2078.
    3. Zaverkin, V., Kästner, J., Holzmüller, D., Steinwart, I.: Fast and Sample-Efficient Interatomic Neural Network Potentials for Molecules and Materials Based on Gaussian Moments. J. Chem. Theory Comput. (2021). https://doi.org/10.1021/acs.jctc.1c00527.
    4. Hamm, T., Steinwart, I.: Intrinsic Dimension Adaptive Partitioning for Kernel Methods. Fakultät für Mathematik und Physik, Universität Stuttgart (2021).
    5. Hang, H., Steinwart, I.: Optimal Learning with Anisotropic Gaussian SVMs. Appl. Comput. Harmon. Anal. 337–367 (2021). https://doi.org/10.1016/j.acha.2021.06.004.
    6. Steinwart, I., Ziegel, J.F.: Strictly proper kernel scores and characteristic kernels on compact spaces. Appl. Comput. Harmon. Anal. 51, 510--542 (2021). https://doi.org/10.1016/j.acha.2019.11.005.
    7. Nonnenmacher, M., Reeb, D., Steinwart, I.: Which Minimizer Does My Neural Network Converge To? In: Oliver, N., Pérez-Cruz, F., Kramer, S., Read, J., and Lozano, J.A. (eds.) Joint European Conference on Machine Learning and Knowledge Discovery in Databases. pp. 87--102. Springer International Publishing, Cham (2021). https://doi.org/10.1007/978-3-030-86523-8_6.

  2. 2020

    1. Steinwart, I.: Reproducing Kernel Hilbert Spaces Cannot Contain all Continuous Functions on a Compact Metric Space. Fakultät für Mathematik und Physik, Universität Stuttgart (2020).
    2. Fischer, S., Steinwart, I.: Sobolev Norm Learning Rates for Regularized Least-Squares Algorithm. J. Mach. Learn. Res. 1--38 (2020).
    3. Holzmüller, D., Steinwart, I.: Training Two-Layer ReLU Networks with Gradient Descent is Inconsistent. Fakultät für Mathematik und Physik, Universität Stuttgart (2020).

  3. 2019

    1. Steinwart, I.: A Sober Look at Neural Network Initializations. Fakultät für Mathematik und Physik, Universität Stuttgart (2019).
    2. Steinwart, I.: Convergence Types and Rates  in Generic Karhunen-Loève Expansions with Applications to Sample Path Properties. Potential Anal. 51, 361--395 (2019). https://doi.org/10.1007/s11118-018-9715-5.
    3. Mücke, N., Steinwart, I.: Empirical Risk Minimization in the Interpolating Regime with Application to Neural Network Learning. Fakultät für Mathematik und Physik, Universität Stuttgart (2019).
    4. Farooq, M., Steinwart, I.: Learning Rates for Kernel-Based Expectile Regression. Mach. Learn. 108, 203--227 (2019). https://doi.org/10.1007/s10994-018-5762-9.
    5. Defant, A., Mastyo, M., Sánchez-Pérez, E.A., Steinwart, I.: Translation invariant maps on function spaces over locally compact groups. J. Math. Anal. Appl. 470, 795--820 (2019). https://doi.org/10.1016/j.jmaa.2018.10.033.

  4. 2018

    1. Blaschzyk, I., Steinwart, I.: Improved Classification Rates under Refined Margin Conditions. Electron. J. Stat. 12, 793--823 (2018). https://doi.org/10.1214/18-EJS1406.
    2. Hang, H., Steinwart, I., Feng, Y., Suykens, J.A.K.: Kernel Density Estimation for Dynamical Systems. J. Mach. Learn. Res. 19, 1--49 (2018).

  5. 2017

    1. Hang, H., Steinwart, I.: A Bernstein-type Inequality for Some Mixing Processes and Dynamical Systems with an Application to Learning. Ann. Statist. 45, 708--743 (2017). https://doi.org/10.1214/16-AOS1465.
    2. Steinwart, I.: A Short Note on the Comparison of Interpolation Widths, Entropy Numbers, and Kolmogorov Widths. J. Approx. Theory. 215, 13--27 (2017). https://doi.org/10.1016/j.jat.2016.11.006.
    3. Steinwart, I., Sriperumbudur, B.K., Thomann, P.: Adaptive Clustering Using Kernel Density Estimators. Fakultät für Mathematik und Physik, Universität Stuttgart (2017).
    4. Farooq, M., Steinwart, I.: An SVM-like Approach for Expectile Regression. Comput. Statist. Data Anal. 109, 159--181 (2017). https://doi.org/10.1016/j.csda.2016.11.010.
    5. Steinwart, I., Thomann, P.: liquidSVM: A Fast and Versatile SVM Package. Fakultät für Mathematik und Physik, Universität Stuttgart (2017).
    6. Steinwart, I.: Representation of Quasi-Monotone Functionals by Families of Separating Hyperplanes. Math. Nachr. 290, 1859--1883 (2017). https://doi.org/10.1002/mana.201500350.
    7. Thomann, P., Steinwart, I., Blaschzyk, I., Meister, M.: Spatial Decompositions for Large Scale SVMs. In: Singh, A. and Zhu, J. (eds.) Proceedings of Machine Learning Research Volume 54: Proceedings of the 20th International Conference on Artificial Intelligence and Statistics 2017. pp. 1329--1337 (2017).

  6. 2016

    1. Hang, H., Feng, Y., Steinwart, I., Suykens, J.A.K.: Learning theory estimates with observations from general stationary stochastic processes. Neural Computation. 28, 2853--2889 (2016). https://doi.org/10.1162/NECO_a_00870.
    2. Steinwart, I., Thomann, P., Schmid, N.: Learning with Hierarchical Gaussian Kernels. Fakultät für Mathematik und Physik, Universität Stuttgart (2016).
    3. Meister, M., Steinwart, I.: Optimal Learning Rates for Localized SVMs. J. Mach. Learn. Res. 17, 1–44 (2016).

  7. 2015

    1. Steinwart, I.: Fully Adaptive Density-Based Clustering. Ann. Statist. 43, 2132--2167 (2015). https://doi.org/10.1214/15-AOS1331.
    2. Steinwart, I.: Measuring the capacity of sets of functions in the analysis of ERM. In: Gammerman, A. and Vovk, V. (eds.) Festschrift in Honor of Alexey Chervonenkis. pp. 223--239. Springer, Berlin (2015). https://doi.org/10.1007/978-3-642-41136-6.
    3. Steinwart, I.: Supplement A to ``Fully Adaptive Density-Based Clustering’’. Fakultät für Mathematik und Physik, Universität Stuttgart (2015). https://doi.org/10.1214/15-AOS1331SUPP.
    4. Steinwart, I.: Supplement B to ``Fully Adaptive Density-Based Clustering’’. Fakultät für Mathematik und Physik, Universität Stuttgart (2015). https://doi.org/10.1214/15-AOS1331SUPP.
    5. Thomann, P., Steinwart, I., Schmid, N.: Towards an Axiomatic Approach to Hierarchical Clustering of Measures. J. Mach. Learn. Res. 16, 1949--2002 (2015).

  8. 2014

    1. Steinwart, I., Pasin, C., Williamson, R., Zhang, S.: Elicitation and Identification of Properties. In: Balcan, M.F. and Szepesvari, C. (eds.) JMLR Workshop and Conference Proceedings Volume 35: Proceedings of the 27th Conference on Learning Theory 2014. pp. 482--526 (2014).
    2. Hang, H., Steinwart, I.: Fast Learning from $\alpha$-mixing Observations. J. Multivariate Anal. 127, 184--199 (2014). https://doi.org/10.1016/j.jmva.2014.02.012.

  9. 2013

    1. Eberts, M., Steinwart, I.: Optimal regression rates for SVMs using Gaussian kernels. Electron. J. Stat. 7, 1--42 (2013). https://doi.org/10.1214/12-EJS760.
    2. Steinwart, I.: Some Remarks on the Statistical Analysis of SVMs and Related Methods. In: Schölkopf, B., Luo, Z., and Vovk, V. (eds.) Empirical Inference -- Festschrift in Honor of Vladimir N. Vapnik. pp. 25–36. Springer, Berlin (2013). https://doi.org/10.1007/978-3-642-41136-6.

  10. 2012

    1. Sriperumbudur, B.K., Steinwart, I.: Consistency and Rates for Clustering with DBSCAN. In: Lawrence, N. and Girolami, M. (eds.) JMLR Workshop and Conference Proceedings Volume 22: Proceedings of the 15th International Conference on Artificial Intelligence and Statistics 2012. pp. 1090–1098 (2012).
    2. Bornn, L., Anghel, M., Steinwart, I.: Forecasting with Historical Data or Process Knowledge under Misspecification: A Comparison. Fakultät für Mathematik und Physik, Universität Stuttgart (2012).
    3. Steinwart, I., Scovel, C.: Mercer’s Theorem on General Domains: on the Interaction between Measures, Kernels, and RKHSs. Constr. Approx. 35, 363--417 (2012). https://doi.org/10.1007/s00365-012-9153-3.

  11. 2011

    1. Steinwart, I.: Adaptive Density Level Set Clustering. In: Kakade, S. and von Luxburg, U. (eds.) JMLR Workshop and Conference Proceedings Volume 19: Proceedings of the 24th Conference on Learning Theory 2011. pp. 703--738 (2011).
    2. Steinwart, I., Christmann, A.: Estimating Conditional Quantiles with the Help of the Pinball Loss. Bernoulli. 17, 211--225 (2011). https://doi.org/10.3150/10-BEJ267.
    3. Eberts, M., Steinwart, I.: Optimal learning rates for least squares SVMs using  Gaussian kernels. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P., Pereira, F.C.N., and Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 24. pp. 1539--1547 (2011).
    4. Steinwart, I., Hush, D., Scovel, C.: Training SVMs without offset. J. Mach. Learn. Res. 12, 141--202 (2011).

  12. 2010

    1. Scovel, C., Hush, D., Steinwart, I., Theiler, J.: Radial kernels and their reproducing kernel Hilbert spaces. J. Complexity. 26, 641–660 (2010). https://doi.org/10.1016/j.jco.2010.03.002.
    2. Christmann, A., Steinwart, I.: Universal Kernels on Non-Standard Input Spaces. In: Lafferty, J., Williams, C.K.I., Shawe-Taylor, J., Zemel, R.S., and Culotta, A. (eds.) Advances in Neural Information Processing Systems 23. pp. 406--414 (2010).
    3. Steinwart, I., Theiler, J., Llamocca, D.: Using support vector machines for anomalous change detection. In: IEEE Geoscience and Remote Sensing Society and the IGARSS 2010. pp. 3732--3735 (2010).

  13. 2009

    1. Steinwart, I., Anghel, M.: Consistency of support vector machines for forecasting the evolution of an unknown ergodic dynamical system from observations with unknown noise. Ann. Statist. 37, 841--875 (2009). https://doi.org/10.1214/07-AOS562.
    2. Steinwart, I., Christmann, A.: Fast Learning from Non-i.i.d. Observations. In: Bengio, Y., Schuurmans, D., Lafferty, J., Williams, C.K.I., and Culotta, A. (eds.) Advances in Neural Information Processing Systems 22. pp. 1768--1776 (2009).
    3. Steinwart, I., Hush, D., Scovel, C.: Learning from dependent observations. J. Multivariate Anal. 100, 175--194 (2009). https://doi.org/10.1016/j.jmva.2008.04.001.
    4. Christmann, A., van Messem, A., Steinwart, I.: On consistency and robustness properties of support vector machines for heavy-tailed distributions. Stat. Interface. 2, 311--327 (2009). https://doi.org/10.4310/SII.2009.v2.n3.a5.
    5. Steinwart, I., Hush, D., Scovel, C.: Optimal Rates for Regularized Least Squares Regression. In: Dasgupta, S. and Klivans, A. (eds.) Proceedings of the 22nd Annual Conference on Learning Theory. pp. 79--93 (2009).
    6. Steinwart, I.: Oracle inequalities for SVMs that are Based on Random Entropy Numbers. J. Complexity. 25, 437--454 (2009). https://doi.org/10.1016/j.jco.2009.06.002.
    7. Steinwart, I., Christmann, A.: Sparsity of SVMs that use the $\epsilon$-insensitive loss. In: Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L. (eds.) Advances in Neural Information Processing Systems 21. pp. 1569--1576 (2009).
    8. Steinwart, I.: Two oracle inequalities for regularized boosting classifiers. Stat. Interface. 2, 271--284 (2009). https://doi.org/10.4310/SII.2009.v2.n3.a2.

  14. 2008

    1. Christmann, A., Steinwart, I.: Consistency of kernel based quantile regression. Appl. Stoch. Models Bus. Ind. 24, 171--183 (2008). https://doi.org/10.1002/asmb.700.
    2. Steinwart, I., Christmann, A.: How SVMs can estimate quantiles and the median. In: Platt, J.C., Koller, D., Singer, Y., and Roweis, S. (eds.) Advances in Neural Information Processing Systems 20. pp. 305--312. MIT Press, Cambridge, MA (2008).
    3. Steinwart, I., Christmann, A.: Support Vector Machines. Springer, New York (2008). https://doi.org/10.1007/978-0-387-77242-4.

  15. 2007

    1. Steinwart, I., Hush, D., Scovel, C.: An Oracle Inequality for Clipped Regularized Risk Minimizers. In: Schölkopf, B., Platt, J., and Hoffman, T. (eds.) Advances in Neural Information Processing Systems 19. pp. 1321--1328. MIT Press, Cambridge, MA (2007).
    2. Scovel, C., Hush, D., Steinwart, I.: Approximate duality. J. Optim. Theory Appl. 135, 429--443 (2007). https://doi.org/10.1007/s10957-007-9281-2.
    3. Christmann, A., Steinwart, I.: Consistency and robustness of kernel-based regression in convex risk minimization. Bernoulli. 13, 799--819 (2007). https://doi.org/10.3150/07-BEJ5102.
    4. Steinwart, I., Scovel, C.: Fast rates for support vector machines using Gaussian kernels. Ann. Statist. 35, 575--607 (2007). https://doi.org/10.1214/009053606000001226.
    5. List, N., Hush, D., Scovel, C., Steinwart, I.: Gaps in support vector optimization. In: Bshouty, N. and Gentile, C. (eds.) Proceedings of the 20th Conference on Learning Theory. pp. 336--348. Springer, New York (2007). https://doi.org/10.1007/978-3-540-72927-3_25.
    6. Steinwart, I.: How to compare different loss functions. Constr. Approx. 26, 225--287 (2007). https://doi.org/10.1007/s00365-006-0662-3.
    7. Christmann, A., Steinwart, I., Hubert, M.: Robust learning from bites for data mining. Comput. Statist. Data Anal. 52, 347--361 (2007). https://doi.org/10.1016/j.csda.2006.12.009.
    8. Hush, D., Scovel, C., Steinwart, I.: Stability of unstable learning algorithms. Mach. Learn. 67, 197--206 (2007). https://doi.org/10.1007/s10994-007-5004-z.

  16. 2006

    1. Steinwart, I., Hush, D., Scovel, C.: A new concentration result for regularized risk minimizers. In: Giné, E., Koltchinskii, V., Li, W., and Zinn, J. (eds.) High Dimensional Probability IV. pp. 260--275. Institute of Mathematical Statistics, Beachwood, OH (2006). https://doi.org/10.1214/074921706000000897.
    2. Steinwart, I., Hush, D., Scovel, C.: An explicit Description of the reproducing kernel Hilbert spaces of Gaussian RBF kernels. IEEE Trans. Inform. Theory. 52, 4635--4643 (2006). https://doi.org/10.1109/TIT.2006.881713.
    3. Steinwart, I., Hush, D., Scovel, C.: Function classes that approximate the Bayes risk. In: Lugosi, G. and Simon, H.U. (eds.) Proceedings of the 19th Annual Conference on Learning Theory. pp. 79--93. Springer, New York (2006). https://doi.org/10.1007/11776420_9.
    4. Hush, D., Kelly, P., Scovel, C., Steinwart, I.: QP Algorithms with Guaranteed Accuracy and Run Time for Support Vector Machines. J. Mach. Learn. Res. 7, 733--769 (2006).

  17. 2005

    1. Steinwart, I., Hush, D., Scovel, C.: A classification framework for anomaly detection. J. Mach. Learn. Res. 6, 211--232 (2005).
    2. Steinwart, I.: Consistency of support vector machines and other regularized kernel machines. IEEE Trans. Inform. Theory. 51, 128--142 (2005). https://doi.org/10.1109/TIT.2004.839514.
    3. Steinwart, I., Hush, D., Scovel, C.: Density Level Detection is Classification. In: Saul, L.K., Weiss, Y., and Bottou, L. (eds.) Advances in Neural Information Processing Systems 17. pp. 1337--1344. MIT Press, Cambridge, MA (2005).
    4. Steinwart, I., Scovel, C.: Fast Rates for Support Vector Machines. In: Auer, P. and Meir, R. (eds.) Proceedings of the 18th Annual Conference on Learning Theory. pp. 279--294. Springer, New York (2005). https://doi.org/10.1007/11503415_19.
    5. Scovel, C., Hush, D., Steinwart, I.: Learning Rates for Density Level Detection. Anal. Appl. 3, 356--371 (2005). https://doi.org/10.1142/S0219530505000625.
    6. Hush, D., Kelly, P., Scovel, C., Steinwart, I.: Provably fast algorithms for anomaly detection. In: International Workshop on Data Mining Methods for Anomaly Detection at KDD 2005. pp. 27--31 (2005).

  18. 2004

    1. Steinwart, I.: Entropy of convex hulls---some Lorentz norm results. J. Approx. Theory. 128, 42--52 (2004). https://doi.org/10.1016/j.jat.2004.04.001.
    2. Christmann, A., Steinwart, I.: On robustness properties of convex risk minimization methods for pattern recognition. J. Mach. Learn. Res. 5, 1007--1034 (2004).
    3. Steinwart, I.: Sparseness of support vector machines---some asymptotically sharp bounds. In: Thrun, S., Saul, L., and Schölkopf, B. (eds.) Advances in Neural Information Processing Systems 16. pp. 1069--1076. MIT Press, Cambridge, MA (2004).

  19. 2003

    1. Steinwart, I.: Entropy numbers of convex hulls and an application to learning algorithms. Arch. Math. 80, 310--318 (2003). https://doi.org/10.1007/s00013-003-0476-y.
    2. Mittmann, K., Steinwart, I.: On the existence of continuous modifications of vector-valued random fields. Georgian Math. J. 10, 311--317 (2003). https://doi.org/10.1515/GMJ.2003.311.
    3. Steinwart, I.: On the optimal parameter choice for $\nu$-support vector machines. IEEE Transactions on Pattern Analysis and Machine Intelligence. 25, 1274--1284 (2003). https://doi.org/10.1109/TPAMI.2003.1233901.
    4. Steinwart, I.: Sparseness of support vector machines. J. Mach. Learn. Res. 4, 1071--1105 (2003).

  20. 2002

    1. Creutzig, J., Steinwart, I.: Metric entropy of convex hulls in type $p$ spaces---the critical case. Proc. Amer. Math. Soc. 130, 733--743 (2002). https://doi.org/10.1090/S0002-9939-01-06256-6.
    2. Steinwart, I.: Support vector machines are universally consistent. J. Complexity. 18, 768--791 (2002). https://doi.org/10.1006/jcom.2002.0642.
    3. Steinwart, I.: Which data--dependent bounds are suitable for SVM’s?, (2002).

  21. 2001

    1. Steinwart, I.: On the influence of the kernel on the consistency of support vector machines. J. Mach. Learn. Res. 2, 67--93 (2001).

  22. 2000

    1. Steinwart, I.: Entropy of $C(K)$-valued operators. J. Approx. Theory. 103, 302--328 (2000). https://doi.org/10.1006/jath.1999.3428.
    2. Steinwart, I.: Entropy of $C(K)$-valued operators and some applications, (2000).

  23. 1997

    1. Bartels, I.: Gewichtete Normungleichungen für Operatoren zwischen Räumen Bochner-integrierbarer Funktionen, (1997).

Education

02/2000 Doctorate (Dr. rer. nat.) in Mathematics, Friedrich-Schiller-University, Jena
03/1997 Diploma in Mathematics, Carl-von-Ossietzky University, Oldenburg

Appointments

07/2017 — Faculty Member, International Max Planck Research School for Intelligent Systems, Stuttgart/Tübingen
04/2010 — Full Professor, Institute for Stochastics and Applications, Department of Mathematics, University of Stuttgart
01/2010 — 06/2011 Associate Adjunct Professor, Jack Baskin School of Engineering, Department of Computer Science, University of California, Santa Cruz
07/2008 — 04/2010 Scientist Level 4, CCS-3, Los Alamos National Laboratory
03/2003 — 04/2010 Technical Staff Member, CCS-3, Los Alamos National Laboratory
03/2002 — 09/2002 Visiting Scientist, Johannes-Gutenberg University, Mainz
03/2000 — 03/2003 Scientific Staff Member, Friedrich-Schiller-University, Jena
04/1997 — 02/2000 Stipendiary, DFG graduate college “Analytic and Stochastic Structures and Systems”, Friedrich-Schiller-University, Jena

Administrative Services

10/2010 — Member of the Senate Committee for Organisation, University Stuttgart
04/2011 — 10/2012 Vice Dean for Mathematics, Faculty of Mathematics and Physics, University of Stuttgart

Editorial Services

01/2013 — Associate Editor, Journal of Complexity
12/2008 — Action Editor (Associate Editor), Journal of Machine Learning Research
01/2010 — 12/2012 Associate Editor, Annals of Statistics

Program Responsibilities at Conferences

Chair COLT 2013
Program Committee NIPS 2008, 2011
Program Committee ICML 2020
Program Committee ICLR 2020
Program Committee COLT 2006, 2008, 2009, 2011, 2012, 2015

LiquidCluster estimates the cluster tree with the help of some density estimators. Key features are:

  • automated hyper-parameter selection procedure
  • speed

The currently available Linux command line version has an interface that is very similar to the one of liquidSVM.

The package together with some additional information can be found here.

Support vector machines (SVMs) and related kernel-based learning algorithms are a well-known class of machine learning algorithms, for non-parametric classification and regression. liquidSVM is an implementation of SVMs whose key features are:

  • fully integrated hyper-parameter selection,
  • extreme speed on both small and large data sets,
  • Bindings for R, Python, MATLAB/Octav, Java, and Spark
  • inclusion of a variety of different learning scenarios:
    • least-squares, quantile, and expectile regression
    • binary and multi-class classification, ROC, and Neyman-Pearson learning
  • full flexibility for experts.

The package together with additional information can be found here.

Prof. Steinwart, what does machine learning have to do with mathematics?

click here

Interview with members of the Department of Mathematics

To the top of the page