Methods for Data Driven modelling - Département de Physique de l'Ecole Normale supérieure

Why choose this program? Program, courses and calendar Master ICFP 1st year (2023-2024) 2023-2024 Calendar and timetable Courses Internships Program, courses and calendar Master ICFP 1st year (2022-2023) 2022-2023 Calendar and timetable Courses Internships Program, courses and calendar Master ICFP 2nd year 2023-2024 First Semester Second Semester "Physics Now" Seminars 2019/2020 "Physics Now" Seminars 2020/2021 "Physics Now" Seminars 2021/2022 "Physics Now" Seminars 2023/2024 "Physics Now" Seminars 2022/2023 Internships 2023 2024 calendar Program, courses and calendar Master ICFP 2nd year 2022-2023 First Semester Second Semester "Physics Now" Seminars 2019/2020 "Physics Now" Seminars 2020/2021 "Physics Now" Seminars 2021/2022 "Physics Now" Seminars 2022/2023 Internships 2022 2023 calendar Admission and Language Requirements How to apply Information for students Information for International Students Fees and Scholarships Careers Master Intranet Archives Erreur ( A supprimer) Program, courses and calendar Master ICFP 1st year (2021-2022) Program, courses and calendar Master ICFP 1st year (2021-2022) Program, courses and calendar Master ICFP 2nd year 2021-2022 Program, courses and calendar Master ICFP 2nd year 2020-2021 Program, courses and calendar Master ICFP 2nd year 2019-2020 Archives 2015-2016 Archives 2016-2017 Archives 2017-2018 Archives 2018-2019 Archives 2019-2020 Archives 2020-2021

[Methods for Data Driven modelling->]

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr

Responsible : Rémi Monasson

Teachers : Rémi Monasson, Jorge de Cossío Díaz

number ECTS : 6

Language of instruction : English

Evaluation : written exam + TD projects

Week 1 What is Bayesian inference?
Bayes’ rule, notions of prior, likelihood and posterior, two historical illustrations: the German Tank and the Boy/Girl Birth Rate problems

Week 2 Asymptotic inference
Rate of convergence, Kullback-Leibler divergence, Fisher information, variational inference, illustration: mean field in stat. mech.

Week 3 Entropy and information - Application to dimensional reduction
Shannon’s entropy, principle of maximum entropy, mutual information, principal and independent component analysis

Week 4 Phase transitions in high-dimensional principal component analysis
Spiked covariance model, large dimensional setting & spectrum of random correlation matrices, the phase transition, when is learning retarded?

Week 4 Priors (1): regression and regularization
Linear regression, L2 prior, cross-validation, harmful and benign overfitting in high-dimensional inference

Week 5 Priors (2): sparsity and beyond
L1 prior, conjugated priors and pseudo-counts, shrinkage, universal priors

Week 6 Graphical models: sampling and learning
Boltzmann Machines (BM), Monte Carlo sampling, Convexity of log-likelihood, BM Learning, Mean-field inference, Pseudo-likelihood method

Week 7 Unsupervised learning: representations and generation
Notion of representation, Autoencoders, restricted Boltzmann machines, Auto-regressive models

Week 8 Unsupervised learning: manifold learning and clustering
Multi-dimensional scaling, Local Linear Embedding, K-means

Week 9 Supervised learning: support vector machines
Linear classifiers, enumeration of dichotomies, perceptron learning algorithm, Kernel methods

Week 10 Supervised learning: multilayer nets
Deep classifiers, stochastic gradient descent, statistical mechanics of two- layer neural nets

Week 11 Learning from streaming data
On-line classification, on-line PCA (Oja’s rule) and sparse PCA

Week 12 Time series analysis (1): hidden Markov models
Markov and hidden Markov processes, Transfer matrix calculations, Viterbi algorithm, Expectation-Maximization procedure

Week 13 Time series analysis (2): recurrent neural nets
Approximation theorem, low-dimensional rank nets: justification and analysis, Some applications

Quick links

Next student seminar :
Access to the program

Here you can find information about your internships:
Experimental Internship - Undergraduate program
Master ICFP first year Internship

News : ICFP Research seminars
November 14 - 18, 2022 :
All information about the program

Contact us - Student support and Graduate School office :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr