Teaching Showcase: Exploring Interaction and Evaluation through Game Design

When I was working at Bauhaus-Universität Weimar in 2015/16, I planned and taught this full-semester project course. Its goal was to familiarize the participants with the principles of evaluation methods for human-computer interaction through a process of designing, implementing and evaluating a video game.

Context

BUW has several master’s degree programmes that are thematically adjacent to human-computer interaction, and they share many courses. One course that every student in one of these master’s programmes needs to complete is a student project. In a student project, participants come together as a small group of typically 5 or 6 students to put the theoretical expertise they have acquired in prior courses into practice. Different projects have varying focuses on analytical vs. constructive components, but they all require the students to define their own realistic goals (within a thematic framework provided by the project advisor) and work towards them.

In the winter semester of 2015/16, I offered this project as a one-time opportunity to explore game design as a context where HCI evaluation methods can be used.

In the German higher education system post-Bologna, degree programmes consist of modules which can then contain one or several courses. My project course allowed participants to fulfill the “student project” module. The following is the full entry for my course in the official course catalogue, which the students could use to make their choice of project course. At the time of this writing, the original entry can also still be found on the BUW server.

Basic Information
Type of CourseProject
Number4447148
TermWiSe 2015/16
Expected no. of participants
FrequencyOne-time
Hyperlink
Additional Linkshttp://www.uni-weimar.de/medien/hci
LanguageEnglish
Hours per week in term10
Max. participants6
Assigned Module
Responsible Instructor
Responsible InstructorResponsibilities
Hornecker, Eva, Professor, Dr.-Ing.
Curriculae
GraduationCurriculaTermECTS-Points
MasterComputer Science and Media (M.Sc.), ER 29-15
MasterComputer Science and Media (M.Sc.), ER 11-15
MasterHuman-Computer Interaction (M.Sc.), ER 14-15
Assign to Departments
Human Computer Interaction (HCI)
Faculty of Media
Contents
engl. Description / Comments

Exploring Interaction and Evaluation through Game Design

In the project, you will learn about and make use of game design and game evaluation methods while creating and prototyping a game of your own and incorporating innovative ideas for user interaction.

Analytical and constructive aspects of the project will be present in roughly equal amounts. At the beginning of the project, the focus will be on acquiring knowledge about methods and theories via a weekly literature review and discussion. In parallel, we will start exploring various novel approaches to user interaction in- and outside of gaming contexts, incorporating previous knowledge and experiences of the project participants. Wherever possible, we will combine existing games that use innovative interaction concepts with various evaluative methods, allowing us to learn and experience both.

The goal for the second half of the project is to design and develop a game of our own that incorporates ways of interaction that have not been explored in gaming yet, and to conduct a thorough evaluation of it using the methods acquired earlier. The game prototype should be functional and robust enough for evaluation. It will make use of an existing modern game engine (such as Unity, Unreal or Pyglet), depending on participants’ previous experience and preferences.

Literature
  • Tracy Fullerton, 2008. Game Design Workshop: A Playcentric Approach to Creating Innovative Games. Taylor & Francis Ltd.
  • Ben Lewis-Evans, 2012. Finding Out What They Think: A Rough Primer To User Research (Part 1 & 2). Gamasutra.
Remarks

Time and place will be announced at the project fair.

participants: max. 6 (2-4 HCI master’s students, 2-4 CSM master’s students)

Prerequisites

Basic background in HCI (for example, prior attendance of the bachelor level course ‘HCI’); basic knowledge of and interest in modern computer games (can be PC, console, mobile, …).

Programming experience; willingness to engage with new programming languages, frameworks, and technologies through self-directed learning. Experience with team-based software development (version control and alike) is helpful, but not required.

Interest in empirical HCI research, in particular methods for evaluation; Willingness to engage with the literature on a conceptual and practical level; Willingness and ability to work in a team, good time- and self-management skills.

CertificatesActive participation, weekly readings, presentations of literature, and managing group discussions. Small-scale evaluations of existing systems, using a selection of existing methods. Designing and developing a game based on existing technologies (such as modern game engines), and evaluating it using appropriate methods. Final project report at the end of the project.
Target GroupM.Sc. Computer Science and Media / Human-Computer Interaction / Medieninformatik

The basic idea for the project course was to apply the HCI evaluation methods that the students already knew from prior courses (interviews, questionnaires, surveys, etc.) to a context where they had no prior experience: the creation of a video game. The project group would be tasked with learning the basics of game design, and would then be prompted to devise and prototype a game of their own, with the added challenge that it had to implement some sort of novel or interesting idea for user input instead of just a standard game controller or mouse and keyboard. When this game was in a playable state, the project group’s task would be to conduct a scientific evaluation using established methods of their choice. I planned the details of the course with this concept in mind.

The different project courses were advertised to students at the yearly project fair. I presented the course and its thematic focus with the help of a few slides.

Exploring Interaction and Evaluationthrough Game DesignJulian FietkauBauhaus-Universität WeimarOctober 13th, 2015
Exploring Interaction and Evaluation through Game DesignJulian FietkauEvaluation methodsFocus groups, interviews,observational studies, ...+Game designRule systems, player engagement,goal conveyance, ...
Exploring Interaction and Evaluation through Game DesignJulian FietkauJourney (thatgamecompany, 2012)
Exploring Interaction and Evaluation through Game DesignJulian FietkauSuper Mario Bros (Nintendo, 1985)
Exploring Interaction and Evaluation through Game DesignJulian Fietkau
Exploring Interaction and Evaluation through Game DesignJulian FietkauRequirements?suitable for M.Sc. students(HCI, CS&M)programming capabilityinterest in evaluation methodspassion for gamesJulian FietkauEva Hornecker
Exploring Interaction and Evaluation through Game DesignJulian FietkauThanks for your attention!Contact: julian.fietkau@uni-weimar.de / eva.hornecker@uni-weimar.deFeel free to approach me after the presentations!

Because students have many different project courses to choose from, I was able to assume at least some intrinsic motivation for my specific topic. Nonetheless, the course had to be designed in such a way that students with little to no experience with video games would be able to complete it. In addition, I considered it important to revisit the evaluation methods that the participants already knew from the perspective of a game designer, and to refer to relevant literature from the field of game studies and from the game development industry to help the team apply their knowledge.

Semester Plan

Before the start of the project course, I developed a week-by-week plan to set topical goals throughout the semester. This would serve as a guidepost for myself as well as for the students.

AnalyticalConstructive
W1 (Oct 12)n/a (Because the project fair happened later this week, the course would start in week 2.)
W2 (Oct 19)Initial meeting + thematic appetizer, discuss goals and expectations and preexisting knowledge
W3 (Oct 26)Methods overviewLet’s talk about game input and technology
W4 (Nov 2)Focus groupsBrainstorming and collecting ideas
W5 (Nov 9)Heuristic evaluationPrototyping
W6 (Nov 16)Questionnaires and surveysGame loops and core systems
W7 (Nov 23)InterviewsProgramming challenges in games + working on your game
W8 (Nov 30)Observational studiesWorking on your game
W9 (Dec 7)Gameplay metricsWorking on your game
W10 (Dec 14)Methods summaryWorking on your game
Christmas break
W11 (Jan 4)Working on your game + planning the evaluation
W12 (Jan 11)Working on your game + preparing the evaluation
W13 (Jan 18)Conducting the evaluation
W14 (Jan 25)Discussing evaluation results, lessons learned

On the analytical track, each week until christmas there would be a literature review and a discussion on a particular empirical method and how it could be applied to games research. On the constructive track, the project would start with some general input on game creation and then smoothly move into the phase where the project team would create their own game by taking it from ideation through physical prototyping into a software implementation.

For the discussions on evaluation methods, one person from the team would be designated as the “expert” and would be given two or three relevant academic articles by me, whereas everyone else would read just one article for that week. The expert would then be tasked with guiding the discussion and making sure that all important aspects would get covered. There are a number of teaching methods that call for something like designated experts, but I decided to come up with my own set of guidelines and called the concept semi-structured discussion.

Semi-structured discussion is a method for getting the most out of group discussions. It could be described as somewhere in the middle between regular plenary discussion and interactive presentation. It works like this:

For each discussion, one group member is designated as the expert. They prepare the discussion topic more in-depth than the other members. During the discussion, they take care that important points get covered by steering the direction of the discourse. However, it is important to note that the expert is not a presenter, it is not their job to provide all of the content. Their role is closer to that of a moderator or facilitator.

Here are some loose guidelines for semi-structured discussions.

If you are the expert:

  • DO: Prepare the discussion topic in-depth using additional literature.
  • DO: Create a list of points you’d like to see discussed and questions you want to get covered.
  • DO: Guide the group to the next point if the discussion gets stuck on one aspect for too long.
  • DO: Try to get a feeling for the level of knowledge of the group members, allow time for areas where they have a lot to say.
  • DO: Point out similarities and differences between group members’ opinions that you notice during the discussion.
  • DON’T: Prepare any slides – you are not a presenter this time.
  • DON’T: Be talking the majority of the time – you can add your perspective, but again, you are not a presenter.
  • DON’T: Sit back and relax – you will need to concentrate not just on the content, but also on the timeframe.
  • DON’T: Get hung up on micro-moderation – it’s nice if you make sure everyone gets equal speaking time, but your primary goal is to get the material covered.
  • DON’T: Write a protocol of the discussion – unless you are the supreme emperor of multitasking, your full attention will be needed for the discussion as it happens.

If you are a regular participant:

  • DO: Prepare the discussion topic – you don’t have to research as deeply as the expert, but you should have a general idea of the topic.
  • DO: Be respectful towards the expert’s guidance – if they say you have to go to the next topic, there is probably a good reason for it.
  • DO: Be constructive and work with the other group members towards a better understanding of the subject.
  • DON’T: Sit back and relax – your discussion input is needed, you are not a consumer today.

On the constructive track, the sessions would start with getting the participants into a creative mindset. To let them experience that ideas for games can be plentiful with the right prompting once you get into it, I created the Game Idea Game, which is now available for download on a separate area of my website.

Game Idea Game image consisting of an objective card and four white cards

In the typical game design process, lots and lots of ideas are created, iterated, improved, sometimes fleshed out and developed, sometimes discarded. When searching for that next brilliant game concept, it can be very useful to get a few people together for a brainstorming session and just start pumping out ideas and talking about them. Coming up with a wide variety of game ideas is a skill that does not come naturally to everyone, but it is something that can be practiced and trained.

This set of cards was developed to give aspiring game designers those initial sparks. Staring at a blank page can be daunting, so the cards give keywords from various categories that can foster and steer creativity, to be used in any way that gets the ideas flowing.

Print the card collection on A4 paper (or larger if you want bigger cards) and cut along the lines. Glue to colored cardboard for extra sturdiness.

Suggested gameplay: Shuffle objective cards seperately, shuffle all white cards in a big pile. Each player draws six white cards and keeps them on hand. Starting with the most experienced player and then going clockwise, each turn the current player flips over an objective card and adds three cards from their hand to it. They then have 30 seconds to explain a game concept that incorporates all those elements. They may add as many cards from their hand as they can incorporate into the idea. Afterwards they draw two new white cards from the pile. If they cannot come up with an idea at all, they have to take the three cards plus one from the pile back on their hand. The first player to completely clear out all cards on hand during their turn wins the game.

Reminder: Above all, this is a creativity tool. If something about this doesn’t work or isn’t fun, change it! Add or remove any cards you want, change the rules to be less competitive (or more competitive), just do whatever works for you and have fun!

For the physical prototyping stage, I prepared a collection of classic board games and a few mixed sets of Lego bricks. This would allow the team to visualize and “tangible-ize” their ideas. Getting computer scientists to warm up to the idea of physical prototyping can be a challenge, but I also knew from previous experience how worthwhile it could be.

I drew the necessary literature support for the constructive part of the project mostly from Tracy Fullerton’s Game Design Workshop: A Playcentric Approach to Creating Innovative Games, an excellent book that covered all required bases.

Challenges Along the Way and Final Results

This section is still under construction, please check back later!

The project result, consisting of the game “Poseidon’s Trident” and its custom input device, is documented here: Student Project: Poseidon’s Trident