Advanced topics in artificial intelligence I (CK0146)

The course overviews selected topics in machine learning and pattern recognition. We study some of the central principles of statistical machine learning, including probabilistic modelling, density estimation and generalised linear models for regression and classification:
  1. Introductory refresher: Probability theory, decision theory and information theory;
  2. Probability distributions: Binary and multinomial variables, the Gaussian, the exponential family, non-parametric distributions;
  3. Linear models for regression: Linear basis function models, Bayesian linear regression, evidence approximation;
  4. Linear models for classification: Discriminant functions, probabilistic generative models, probabilistic discriminative models, Bayesian logistic regression.
Stuff from the past: Previous version of the course are available (internal links may need adjustment) for 2016.2 and 2015.2



Instructor ♞: Francesco Corona (FC), francesco döt corona ät ufc döt br

Physical location ♜: Tuesdays and Thursdays 14:00-16:00, Bloco 950, Sala 05.
Internet location ♖: Here! Or, here (CK0146) for mambojumbo related to administration.

Evaluation ✍: Approx. half a dozen theoretical and practical problem sets will be assigned as homework. Homework assignments are equivalent to partial evaluations (APs). If needed a final evaluation will be arranged.

Go to:   Lectures and schedule | Problem sets | Supplementary material | As it pops out |

News

>>>>>> Avaliação Institucional 2017.1 <<<<<<

>>>>>> PIBIC 2017/2018 - 1 Position @ CC/DC <<<<<<
>>>>>> PIBIC 2017/2018 - 2 Positions @ CT/DETI <<<<<<

Lectures and schedule

  1. About this course

    A About this course (FC)
    • Slides (✎ MAR 14 and ✎ MAR 16)
    • About the type of machine learning, pattern recognition, and advanced topics in artificial intelligence that we study

    Reality check (✎ MAR 14) Hand-in by MAR 22 at 23:59:59 Fortaleza time
    Exercises (✎ MAR 16) Hand-in by APR 14 (was APR 09) at 23:59:59 Fortaleza time

  2. Introductory refresher

    A Probability theory (FC)
    • Slides (✎ MAR 21, ✎ MAR 23, ✎ MAR 28, ✎ MAR 30 (cancelled, Encontros Universitários 2017), ✎ APR 04, and ✎ APR 06)
    • Definitions and rules, densities, expectations and covariances
    • Bayesian probabilities, the univariate Gaussian distribution
    • Bayesian polynomial regression
    • Graphical models
    B Decision theory (FC)
    • Slides (✎ APR 25 and ✎ APR 27)
    • Minimisation of the classification rate
    • Minimisation of the expected loss
    • The reject option
    • Inference and decision
    • Loss functions for regression
    C Information theory (FC)
    • Slides (✎ APR 18, ✎ APR 20 and ✎ APR 25)
    • Shannon information content and entropy
    • Relative entropy
    • Mutual information

    Exercises (Last updated, APR 24 | ✎ APR 18) Hand-in by MAY 03 (was APR 30) at 23:59:59 Fortaleza time

  3. Probability distributions

    A Binary and multinomial variables (FC)
    • Slides (✎ MAY 02 and ✎ May 04)
    • Bernoulli, binomial and beta distributions
    • Multinomial and Dirichlet distributions
    B The Gaussian distribution (FC)
    • Slides (✎ MAY 09, ✎ MAY 11,✎ MAY 16, ✎ MAY 18 (cancelled, no power),✎ MAY 23, ✎ MAY 25)
    • Conditional and marginal Gaussians, Bayes' rule for Gaussians, maximum likelihood, sequential estimation, Bayesian inference, Student's t-distribution
    • Mixtures of Gaussians (naive MLE, with latent variables and EM, k-means)
    C The exponential family (FC)
    • Maximum likelihood and sufficient statistics
    • Conjugate and non-informative priors
    D Non-parametric distributions (FC)
    • Slides (✎ MAY 30, ✎ JUN 01)
    • Histograms, kernel and nearest-neighbour density estimators

    Exercises (Part 1 | ✎ APR 20) Hand-in by MAY 21 at 23:59:59 Fortaleza time
    Exercises (Part 2 | ✎ MAY 30) Hand-in by JUN 20 (was JUN 13) at 23:59:59 Fortaleza time

  4. Generalised linear models for regression

    A Linear basis function models (FC)
    • Slides (✎ JUN 06 and ✎ JUN 08)
    • Maximum likelihood and least-squares, geometry of the least-squares
    • Sequential learning
    • Regularised least-squares
    • Multiple outputs
    B Bayesian linear regression (FC)
    • Slides (✎ JUN 13 and ✎ JUN 20)
    • Parameter distribution and predictive distribution
    • The equivalent kernel and Gaussian processes for regression
    C Bias-variance decomposition (FC)
    • Bias-variance decomposition
    D Bayesian model comparison (FC)
    • Bayesian model comparison

    Exercises (✎ JUN 20) Hand-in by JULY 08 (was JULY 02) at 23:59:59 Fortaleza time

  5. Generalised linear models for classification

    A Discriminative functions (FC)
    • Two- and multi-class classification
    • Least-squares for classification
    • Fisher's linear discriminant
    B Probabilistic generative models (FC)
    • Continuous inputs, maximum likelihood solution
    C Probabilistic discriminative models (FC)
    • Logistic regression and iterative re-weighted least-squares
    • Probit regression
    • Canonical link functions
    D Laplace approximation (FC)
    • Model comparison and BIC
    E Bayesian logistic regression (FC)
    • Laplace approximation, predictive distribution

[Top]

Problem sets

As we use problem set questions covered by books, papers and webpages, we expect you not to copy, refer to, or look at the solutions in preparing your answers. We expect you to want to learn and not google for answers: If you do happen to use other material, it must be acknowledged clearly with a citation on the submitted solution.

☯ The purpose of problem sets is to help you think about the material, not just give us the right answers.

Homeworks must be done individually: Each of you must hand in his/her own answers. In addition, each of you must write his/her own code when requested. It is acceptable, however, for you to collaborate in figuring out answers. We are assuming that you take the responsibility to make sure you personally understand the solution to any work arising from collaboration (though, you must indicate on each homework with whom you collaborated).

⚛ To typeset assignments, students and teaching assistants are encouraged to use this LaTeX template: Source (PDF).

⚠ Assignments must be returned before deadline via SIGAA (if you're in it, you'll get notified of the opening of a new task) or, if you're not in SIGAA, return via email to one of the responsible teaching assistants - Delays will be penalised.

[Top]

Material

Course slides will suffice. Slides are mostly based on the following textbook: The material can be complemented using material from the following textbooks (list not exhaustive):
  1. Machine Learning: A Probabilistic Perspective, by Kevin Murphy;
  2. The Elements of Statistical Learning (Book website), by Trevor Hastie, Robert Tibshirani and Jerome Friedman;
  3. Bayesian Reasoning and Machine Learning (Book website), by David Barber.
☠ Copies of these books are floating around.

>>>>>> Course material is prone to a typo or two - Please inbox FC to report <<<<<<

Top

Read me or watch me

Top