Reinforcement Learning (DAT605)

This course would introduce AI and Optimization in a fun, easy, interesting, immersive, and hands-on way. Optimization problems are becoming essential across multiple disciplines. The skills gained through this course would allow the use of efficient optimization strategies in work processes. These could include optimization of complex machine learning models making them more efficient, creating exploratory models that without training can evaluate a situation and gradually make positive decisions, explore financial data to discover patterns that lead to beneficial outcomes, and others.


Course description for study year 2024-2025. Please note that changes may occur.

Facts

Course code

DAT605

Version

1

Credits (ECTS)

5

Semester tution start

Autumn

Number of semesters

1

Exam semester

Autumn

Language of instruction

English

Content

NB! This is an elective course and may be cancelled if fewer than 10 students are enrolled by August 20th.

Artificial Intelligence in this era has become synonymous with Supervised and Unsupervised Learning. Supervised learning is best suited for cases that have a vast set of examples of inputs and desired outputs and the objective is to learn based on such examples in order to generate output from some future, currently unseen input. Text classification, Image Classification, Object location, Regression problems, and Sentiment analysis are areas where supervised learning is extensively used. Whereas, Unsupervised learning aims to discover some hidden structure of the data without the need to have a specific distinction in the input and output values. Such learning techniques are commonly used for clustering of data that tries to combine data items into a set of clusters revealing relationships in data.

Reinforcement learning lies somewhere in between supervised and unsupervised forms of learning techniques. On one hand, it builds on established methods of supervised learning for function approximation, stochastic gradient descent, and backpropagation to learn data representation, however, on the other hand, it does not require supervision in order to discover hidden patterns and relationships in data. Reinforcement learning primarily focuses on the problem of automatic learning of optimal decisions over time in a complex environment by building on advances in computer science, behavioral psychology, and neuroscience. Due to its flexibility and generality, the field of RL is developing very quickly and attracting lots of attention, both from researchers who are trying to improve existing methods or create new methods and from practitioners interested in solving their problems in the most efficient way.

The target group for this course would be professionals and students working or interested in areas of artificial intelligence, machine learning, game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, and statistics.

Learning outcome

Concepts covered in this course would provide relevant theoretical and hands-on programming knowledge. Every topic is demonstrated using easy-to-understand real-world examples. The following topics would be covered during the course duration:

Knowledge:

  • Topic 1: Reinforcement Learning - an introduction
  • Topic 2: Course Materials, Supplementary Resources, and Development Environment
  • Topic 3: Tabular Methods
    • Multi-Armed Bandit
    • Markov Decision Processes
    • Cross-Entropy Method
  • Topic 4: Dynamic Programming
  • Topic 5: Monte-Carlo & Temporal Difference and Q-Learning
  • Topic 6: Policy Gradients
  • Topic 7: The Actor-Critic Method
  • Topic 8: Deep Q-Network - an Overview
  • Topic 9: Further Exploration

Skills:

  • Good programming knowledge
  • Knowledge of basic algebra, probability, and statistics
  • Python Programming Knowledge
  • Understanding of Numpy, Matplotlib

General competence:

  • Artificial Neural Networks
  • Deep Learning
  • Convolutional Neural Networks

Required prerequisite knowledge

None

Recommended prerequisites

DAT120 Introduction to Programming, DAT540 Introduction to Data Science, STA500 Probability and Statistics 2

Exam

Project report and oral presentastion

Form of assessment Weight Duration Marks Aid
Project report (group) 1/2 Letter grades
Oral exam 1/2 Letter grades None permitted

Project Work in GroupsThe project is completed in groups. Project work is to be performed in the groups that are assigned and published. Absence due to illness or for other reasons must be communicated as soon as possible to the lecturer.A project report including source code, and an oral exam contribute to the grade. If a student fails the project work, he/she has to take it again the next time the subject is lectured.

Coursework requirements

Assignments

Throughout the semester students work individually on 2 assignments. These assignments are mandatory and must be passed within the announced deadline so that the student has the right to take the exam.

Absence due to illness or for other reasons must be communicated to the laboratory personnel as soon as possible. One cannot expect that provisions for completion of lab assignments at other times are made unless prior arrangements with the laboratory personnel have been agreed upon.

Course teacher(s)

Course coordinator:

Antorweep Chakravorty

Head of Department:

Tom Ryen

Method of work

The work will consist of 4 hours of lecture and 2 hours of scheduled laboratory. Students are expected to spend an additional 4-8 hours a week on self-study, group discussions, and development work.

Open for

Admission to Single Courses at the Faculty of Science and Technology
Data Science - Master of Science Degree Programme

Course assessment

There must be an early dialogue between the course supervisor, the student union representative and the students. The purpose is feedback from the students for changes and adjustments in the course for the current semester.In addition, a digital subject evaluation must be carried out at least every three years. Its purpose is to gather the students experiences with the course.

Literature

The syllabus can be found in Leganto