Pilot case

Joint Pilot for Smart Content and Courseware

This is a follow-up pilot for the 2016 A!OLE Join Pilot for Programming in which developed a process to design, create, and maintain interactive learning materials for programming courses.
Joint Pilot for Smart Content and Courseware

Pilot leader

Ari Korhonen


School of Science


about 3000 students yearly


Jan 2017 – Dec 2017


In this pilot, we aim at

  • streamlining and standardizing the above processes,
  • creating novel tools for developing new content,
  • improving usability (students, instructors, maintenance),
  • eliminating overlapping work from the processes,
  • implementing and producing scalable automatic grading back ends,
  • developing the automatic feedback provided by the back ends,
  • developing novel smart content,
  • testing the above and taking them into use in real courses, and
  • taking part into the road map work, e.g., realizing pitfalls in rapid digitalization such as platform and vendor lock ups.

Platforms and tools

Most of the courses can found from A-plus. A-plus is an open source project that can be downloaded from GitHubCHEM-A2600 and CS-A1111 use MyCourses. Although most of the courses utilize A+ learning platform, we have also sub projects aiming at utilizing other platforms such as MyCourses.

H5P is a technology to create interactive and reusable HTML5 content for e-learning materials. During this pilot, we have integrated the H5P tool with the standard MyCourses “Book” activity to create interactive learning materials for the course CHEM-A2600 (Programming for Chemical Engineers). The results can be viewed at the MyCourses page of the course (should work even for guests). H5P is an open-source technology, which can be used as a cloud-type service at H5P.org or installed locally in Moodle, WordPress or Drupal.

When using H5P.org as a cloud-type service, the H5P activities can be integrated in MyCourses without any additional plugins. Only three simple steps are required: 1) Create a new activity at H5P.org, 2) Copy the URL given by H5P for embedding the activity, 3) Paste the URL to any MyCourses content by using the built-in HTML editor (label, e-book, or any place where the built-in HTML editor is available.

On the CHEM-A2600 course, the interactive H5P building blocks are being used to create various “mini-exercises” within the course e-book (there is a mini-exercise at the end of each subchapter). The aim of these rather simple mini-exercises is to illustrate some basic concepts of Python programming to the students. Based on student feedback, the easy-to-use interactive content in the e-book supports their learning, especially for those who are learning programming for the very first time. The mini-exercises are not graded at the moment (this would be possible if a full H5P plugin would be installed in MyCourses). The grading of the course is based on CodeRunner programming exercises (100+ exercises at different difficulty levels).

Involved courses

CS-A1110 – Ohjelmointi 1 (5 ECTS)
CS-A1111 – Ohjelmoinnin peruskurssi Y1 (5 ECTS)
CS-A1120 – Ohjelmointi 2 (5 ECTS)
CS-A1130 – Tietotekniikka sovelluksissa (5 ECTS)
CS-A1140 – Tietorakenteet ja algoritmit (5 ECTS)
CS-A1141 – Tietorakenteet ja algoritmit Y (5 ECTS)
CS-A1150 – Tietokannat (5 ECTS)
CS-C2110 – Ohjelmointistudio 1: mediaohjelmointi (5 ECTS)
CS-C3170 – Web Software Development (5 ECTS)
CS-E3220 – Declarative Programming (5 ECTS)
CS-E4800 – Artificial Intelligence (5 ECTS)
CHEM-A2600 – Programming for Chemical Engineers (5 ECTS)


Course improvements

Timi Seppälä made improvements to the A+ platform, including upgrades to the way the platform handles course lifespans from drafts to running courses to partially active courses to archived courses, and the access privileges of various groups during those stages. He also made various minor improvements to the A+ user interface. Working primarily for CS-A1110 Programming 1 and the corresponding MOOC course, he made numerous course-specific improvements such as partially reimplementing an assignment that relies on an external maps API (necessary due to a change in licencing terms), overseeing the transition to using the Slack discussion forum and producing associated instructional materials, and implementing a new visual appearance for the course’s ebook.

CS-A1130 Applications of Computing (Tietotekniikka sovelluksissa) and CS-A1141 Data Structures and Algorithms Y (Tietorakenteet ja algoritmit Y) courses were developed further by Aliisa Pietilä, Juho Pekkinen, Riku Ala-Laurinaho, Miki Tolonen, and Ari Korhonen. CS-A1130 has gradually turned from pure manually graded exercises to allow more and more automatic assessment. This is an ongoing process that continues in the future. One of the challenges here is that the course utilizes many programming languages such Python, MATLAB, LabView, and Javascript. All of which require implementing and producing their own scalable automatic grading back ends. In addition, developing understandable automatic feedback is time consuming process that need to be improved in the forthcoming years.  CS-A1141, on the other hand, already had full pack of automatically assessed exercises. Thus, here the focus was on streamlining and eliminating overlapping work from the processes, improving the automatic feedback provided by the back ends, utilizing novel tools for developing new smart content, and improving usability.

In CS-A1140 Data Structures and Algorithms, Markus Arlander (under supervision of Tommi Junttila) developed and implemented new programming assignments (including the student code packages and the graders) for the course, e.g. a debugging assignment for the recap round that helps the students to refresh and improve their debugging skills (which will be essential in the following rounds when they most likely have to find out why their own algorithms do not function properly) and an assignment for the hashing round asking the students to implement a rolling hash function and the Rabin-Karp string search algorithm in a rather efficient way so that it is competitive against the string search algorithms in the Java and Scala libraries. Some of the developed assignments have been in “production use” in the Autumn 2017 course instance (to give an estimate of the student volume, 206 students have reached the grade 1 or higher level by Dec 5).

In course CS-A1150, Jonatan Lehtinen implemented (under supervision of Kerttu Pollari-Malmi) automatic assessment of relational algebra queries. Automatic assessment reduces the workload of the teacher, but also enhances learning by giving the student immediate feedback. When the system was introduced in a seminar in August, the teacher of the corresponding databases course in Tampere University of Technology was also interested in using the new system in his course. Jonatan Lehtinen also transferred the Python programming problems of course CS-A1111 from old Greengoblin system to A+ system and tested having the course text material and exercise problems interleaved on the same platform. However, the final deployment of the new system was decided to be postponed to Summer2018 course to make sure that everything works without problems in the course of 700 enrolled students.

Teemu Pudas has implemented declarative visualizion interface for the Artificial Intelligence (CS-E4800) and Declarative Programing (CS-E3220) courses. The idea is that the content of an image can be specified in a declarative way, i.e., the required graphical elements are given a logical specification which may involve complex conditions on which elements should appear and where they should be placed. Technically, such specifications yield a collection of graphical elements (such as text, lines, circles, rectangular areas etc) along with style parameters (such as color, line style, etc). Any collection of graphical images can be translated into some well-known graphical format (such as tikz under LaTeX or svg) and converted into a concrete graphical image. The current formats of interest are pdf and svg because the production of web-based materials is of main interest. For future extensions, it will be easy to extend the approach by new graphical elements and style parameters supported by the tikz format, for instance. The interface has been defined by the needs of existing use cases which go back to various kinds of images drawn for the purposes of the courses involved. In addition to the declarative visualization approach, Teemu Pudas has implemented mechanisms for the automated evaluation of exercises in the AI and DP courses.

The Artificial Intelligence course (CS-E4800) is the first course utilizing active elements that have been implemented by Roosa Piitulainen in the context of A+ system. The idea to embed external computations as part of web-based course material (like chapters in A+). Active elements enable the incorporation of various kinds of tools at students disposal. Each computation is based on a number of input fields and an output field. For the moment, the supported input fields are text fields and menu-based fields based on fixed inputs. Once all inputs have been defined by the student, the associated external computation is launched. Technically, this amounts to the evaluation of an A+ exercise, but the computation of the respective output (rather than points and feedback) is of interest. Currently, the forms of output supported by active elements are plaintext, html, and svg images. Input fields can be shared and output fields can be freely placed in the material (in contrast with traditional A+ exercises which conform to a more fixed format). To give a concrete use case, the AI course embeds a Sudoku solver in the material: given the hints of a Sudoku puzzle as input, the respective output fields presents the solution as an svg image. The image is updated whenever the input is changed.


Ari Korhonen
School of Science, Department of Computer Science
Pilot leader

Pilot Group and Courses related to the pilot

  • CS-A1110 Ohjelmointi 1, Juha Sorva
  • CS-A1111 Ohjelmoinnin peruskurssi Y1, Kerttu Pollari-Malmi
  • CS-A1120 Ohjelmointi 2, Petteri Kaski
  • CS-A1130 Tietotekniikka sovelluksissa, Aliisa Pietilä, Ari Korhonen
  • CS-A1140 Tietorakenteet ja algoritmit, Tommi Junttila
  • CS-A1141 Tietorakenteet ja algoritmit Y, Ari Korhonen
  • CS-A1150 Tietokannat, Kerttu Pollari-Malmi
  • CS-C2110 – Ohjelmointistudio 1: mediaohjelmointi, Otto Seppälä
  • CS-C3170 – Web Software Development, Otto Seppälä
  • CS-E3220 Declarative Programming, Tomi Janhunen and Tommi Junttila
  • CS-E4800 Artificial Intelligence, Tomi Janhunen and Alex Jung
  • CHEM-A2600 Programming for Chemical Engineers, Antti Karttunen

The following is the list of personnel in each course: Riku Ala-Laurinaho, Juho Pekkinen, Aliisa Pietilä, and Miki Tolonen (CS-A1130 and CS-A1141), Kyösti Alkio (CS-C3170), Markus Arlander (CS-A1140), Matias Hyyppä (CS-A1120), Jonatan Lehtinen (CS-A1150), Teemu Pudas (CS-E3220 and CS-E4800), Aatos Saarinen (CS-C2110), Timi Seppälä (CS-A1110), Juhani Teeriniemi (CHEM-A2600), and Jarno Linnera (CHEM-A2600).

In addition, the following persons have significantly contributed this pilot: Jaakko Kantojärvi, Teemu Lehtinen, Matias Lindgren, Le Hoang Kha Nguyen, Roosa Piitulainen, and Markku Riekkinen.

Contact us

Got a great idea on how to support learning via online means?
We help make it reality!

Find out more