Prochain Séminaire de la FIP :
Accéder au programme
Retrouvez toutes les informations pour vos stages :
Stages L3
Stages M1 ICFP
Actualités : Séminaire de Recherche ICFP
du 14 au 18 novembre 2022 :
Retrouvez le programme complet
Contact - Secrétariat de l’enseignement :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr
r>
Prochain Séminaire de la FIP :
Accéder au programme
Retrouvez toutes les informations pour vos stages :
Stages L3
Stages M1 ICFP
Actualités : Séminaire de Recherche ICFP
du 14 au 18 novembre 2022 :
Retrouvez le programme complet
Contact - Secrétariat de l’enseignement :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr
r>
Responsible : Rémi Monasson
Teachers : Rémi Monasson, Jorge de Cossío Díaz
number ECTS : 6
Language of instruction : English
Evaluation : written exam + TD projects
Week 1 What is Bayesian inference ?
Bayes’ rule, notions of prior, likelihood and posterior, two historical illustrations : the German Tank and the Boy/Girl Birth Rate problems
Week 2 Asymptotic inference
Rate of convergence, Kullback-Leibler divergence, Fisher information, variational inference, illustration : mean field in stat. mech.
Week 3 Entropy and information - Application to dimensional reduction
Shannon’s entropy, principle of maximum entropy, mutual information, principal and independent component analysis
Week 4 Phase transitions in high-dimensional principal component analysis
Spiked covariance model, large dimensional setting & spectrum of random correlation matrices, the phase transition, when is learning retarded ?
Week 4 Priors (1) : regression and regularization
Linear regression, L2 prior, cross-validation, harmful and benign overfitting in high-dimensional inference
Week 5 Priors (2) : sparsity and beyond
L1 prior, conjugated priors and pseudo-counts, shrinkage, universal priors
Week 6 Graphical models : sampling and learning
Boltzmann Machines (BM), Monte Carlo sampling, Convexity of log-likelihood, BM Learning, Mean-field inference, Pseudo-likelihood method
Week 7 Unsupervised learning : representations and generation
Notion of representation, Autoencoders, restricted Boltzmann machines, Auto-regressive models
Week 8 Unsupervised learning : manifold learning and clustering
Multi-dimensional scaling, Local Linear Embedding, K-means
Week 9 Supervised learning : support vector machines
Linear classifiers, enumeration of dichotomies, perceptron learning algorithm, Kernel methods
Week 10 Supervised learning : multilayer nets
Deep classifiers, stochastic gradient descent, statistical mechanics of two- layer neural nets
Week 11 Learning from streaming data
On-line classification, on-line PCA (Oja’s rule) and sparse PCA
Week 12 Time series analysis (1) : hidden Markov models
Markov and hidden Markov processes, Transfer matrix calculations, Viterbi algorithm, Expectation-Maximization procedure
Week 13 Time series analysis (2) : recurrent neural nets
Approximation theorem, low-dimensional rank nets : justification and analysis, Some applications
Prochain Séminaire de la FIP :
Accéder au programme
Retrouvez toutes les informations pour vos stages :
Stages L3
Stages M1 ICFP
Actualités : Séminaire de Recherche ICFP
du 14 au 18 novembre 2022 :
Retrouvez le programme complet
Contact - Secrétariat de l’enseignement :
Tél : 01 44 32 35 60
enseignement@phys.ens.fr
r>