Accepted Papers and Posters 2021

Accepted Papers and Posters 2021

Full papers and short papers

Sebastian Mateos Nicolajsen, Magda Pischetola, Paweł Grabarczyk and Claus Brabrand. Three +1 Perspectives on Computational Thinking

Abstract: Computational Thinking (CT) is a highly contentious subject with many diverging meanings and definitions. This study presents a systematic literature review of 68 peer-reviewed articles on CT published between 2006 and 2021. The papers indicate the existence of five main aspects that have historically been used in association with its definition: Algorithm, Abstraction, Modelling, Simulation, and Implementation. Based on this preliminary literature study, semi-structured interviews with eight CT scholars are conducted, in order to identify qualitatively different perspectives on CT, which integrate the mentioned aspects in different ways. From the interviews, three perspectives emerged, focusing on: Reasoning, Simplification, and Automation. Furthermore, the goal of having computationally educated citizens is extrapolated from the interviews, indicating as an additional perspective (+1) titled Empowerment, which appears as embedded within all the previous three perspectives. This paper proposes to put these three (+1) perspectives in dialogue, in an effort to support researchers and practitioners working with CT across different fields.
Aleksi Lukkarinen, Arto Hellas and Lassi Haaranen. A Kingdom for a Button: Students’ Thoughts about Buttons

Abstract: In learning programming and learning to construct applications with graphical user interfaces, there exists a large body of concepts from everyday life that are used to guide students. In this study, we explore whether such concepts from everyday life that we might believe require no explanation are actually understood in different ways by students in an introductory browser-based applications course. Our analysis focuses on an elementary interactive element: a button. Analyzing survey data from 185 students, we observe that even the simple concept of a button may be understood in a myriad of ways and that the context in which the concept is represented significantly influences beliefs of how the concept behaves. Our results indicate that students can rarely disentangle a concept from the context, and that some even believe that the text shown on a button in a graphical user interface is used to define the functionality of the button.
Jordan Allison. The Importance of Context: Assessing the Challenges of K-12 Computing Education Through the Lens of Biggs 3P Model

Abstract: The evolution and innovation of technology within society has caused many changes and challenges for computing education. However, the root cause of these challenges are often misunderstood, or neglected. Therefore, this paper deals with the question of identifying the contextual factors that create the challenges which exist in K-12 computing education. To do this, the perspectives of 32 English college employees were considered, with the challenges and contextual factors identified being mapped to Biggs 3P Model of classroom learning. Through this process, in addition to a short comparison between English and Finnish education systems and policy, the 3P Model was re-framed to include the fourth P of Policy consisting of ‘government priorities’, and ‘regulatory bodies’. Findings indicate the extent to which contextual factors influence the challenges and effectiveness of computing education, and how many of these factors are interlinked, despite ranging from macro-environment trends and policy to an individual student’s knowledge of programming. Together, these findings raise important philosophical questions of what computing education is, and what the goals of computing education should be. The answers to these questions have significant implications for practice, which practitioners and policy makers must consider for the design of any computing program of study, while the amended 3P Model can be used to identify the contextual factors which may influence the effectiveness of such programs. Furthermore, this paper contributes to the limited body of research concerning English college education, and more specifically perspectives regarding computing education in this context.
Anja Hawlitschek, Sarah Berndt and Sandra Schulz. Towards a Framework of Planning Collaborative Learning Scenarios in Computer Science

Abstract: The planning and implementation of collaborative learning is perceived by teachers as demanding and time-consuming. In addition to individual learners, groups must be taken into account – with their group dynamics, demands on group coordination, and group experiences. The aim of this paper is to develop a framework for the instructional design of collaborative learning (particularly pair programming) in computer science to support a systematic evidence based implementation. We describe a work in progress framework, encompassing didactic analyses and decision fields, which are relevant for the planning phase, and provide recommendations for actions based on the results of empirical studies.
Luca Chiodini and Matthias Hauswirth. Wrong Answers for Wrong Reasons: The Risks of Ad Hoc Instruments

Abstract: To evaluate novel pedagogies, approaches, and tools, Computer Science Education researchers often conduct experiments to look for differences among groups treated with different interventions. The methodological rigor of such experiments affects the soundness of the conclusions the researchers can draw. In this paper we focus on a central aspect of such experimental research: the instruments used to assess participants’ knowledge. Specifically, we study the use of ad hoc instruments and the risks due to their insufficient validation.
We present a literature survey that highlights how, even though standardized instruments exist, the majority of published experiments in the last five years at major Computer Science Education conferences carries out pre/post-tests using ad hoc instruments, often with multiple-choice as question type.
We demonstrate the risks of such commonly used but insufficiently validated multiple-choice instruments. We propose a richer way to analyze and assess the correctness of answers to multiple-choice questions, requiring participants to add brief explanation texts as a justification of each answer.
We run an experiment and analyze the collected answers using the two approaches, with and without explanations, to show that the risk of drawing opposite conclusions from the statistical analysis is concrete.
Jordan Allison. Flipped Classroom Teaching in a Maths for Technology Course: Recommendations for Success

Abstract: Mathematics is an important topic for computing and engineering students, but students often find the subject challenging, which emphasises the importance of using effective pedagogical strategies. The constructivist approach of the flipped classroom, where students learn new content out of class, and practice problems and engage in discussion in class, is raising in popularity. However, it is seldom reported how such an approach is used for teaching mathematics in the context of computing courses. This paper reports the experience and findings of implementing a flipped classroom approach for a foundation university module titled ‘Mathematics for Technology’. A flipped classroom approach is shown to be successful in this context, but there are key considerations for its implementation. This includes factors such as constructive alignment, the desired learning and assessment outcomes, and the disposition of the student cohort. Recommendations are provided for practitioners who may be interested in implementing the flipped classroom approach in a similar context.
Zak Risha, Jordan Barria-Pineda, Kamil Akhuseyinoglu and Peter Brusilovsky. Stepwise Help and Scaffolding for Java Code Tracing Problems With an Interactive Trace Table

Abstract: In this paper, we describe the integration of a step-by-step interactive trace table into an existing practice system for introductory Java programming. These autogenerated trace problems provide help and scaffolding for students who have trouble in solving traditional one-step code tracing problems, accommodating a wider variety of learners. Findings from a classroom deployment suggest the scaffolding provided by the trace table is a plausible form of help, most notably increases in performance and persistence and lower task difficulty. Based on usage data, we propose future implications for an adaptive version of the interactive trace table based on learner modeling.
Sebastian Dziallas, Sally Fincher, Matthew Barr and Quintin Cutts. Learning in Context: A First Look at a Graduate Apprenticeship

Abstract: In this paper, we examine a new Graduate Apprenticeship program. Graduate Apprenticeships (and degree-level apprenticeships more broadly) were established in the United Kingdom with the expressed goal of addressing the persistent skills gap, with employers facing difficulties in recruiting adequately skilled employees, and at the same time promoting opportunities for students who would not have otherwise pursued a computing degree.
We take a 360-degree view of this novel program and explore why the university chose to pursue it, why the employers decided to participate in it, and how the students are experiencing it. To do so, we draw on three kinds of data: life story interviews with students in the first cohort of the Graduate Apprenticeship that focus on the individual learning journeys that brought them to the program; a consultation and series of individual interviews with employers to examine their aims for participating in the program; a diary study with all apprentices over a period of four weeks to explore the kinds of work they were doing on a day-to-day basis in their companies.
In our “first look” at this program, we highlight aspects that make it particularly attractive to this student body and explore the actual work apprentices are doing in their companies after a year in the program. As well as providing local insight, we more broadly present this program in its context as an example of an evolutionary response to forces on an educational system.
Caroline Hardin. Risk and Persistence at Hackathons

Abstract: Hackathons are time bound collaborative computer science competitions increasingly popular at undergraduate institutions. It is often claimed that their failure-friendly environment helps students become better computer scientists.
The same features which encourage risk taking, however, may increase imposter syndrome and stereotype threat. Furthermore, if hackathons really do radically celebrate emergent projects, why do so many participants leave without showcasing their work?
This work addresses two research questions: 1) Do students consider hackathons as a space which is safe for taking risks and learning through failure? And 2) How does participants’ perception of risk affect their decisions to persist or quit the main event competition?
Extensive mixed methods data was collected from student attendees, professors, and industry sponsors of 3 undergraduate hackathons.
Constructionism was used to understand participant learning through making in a failure-friendly environment, and social cognitive career theory for how risk affects decisions on continued participation.
Results showed participants largely felt that risk was good for learning, although they did worry about classwork, health, meeting their own standards, and public embarrassment. They felt much more free to take risks then in classes, and wanted to use 60\% new skills. Of the half who didn’t submit, reasons were external conflicts or an unfinished project. These findings have implications for how risks can be productively used for learning both at hackathons and CS classes.
Emma Choi, Lisa Meng and John Hott. Open Source Software Practices in CS2

Abstract: By contributing to open source software (OSS), students can gain professional software development experience and learn about applications of computer science (CS) concepts in pragmatic contexts. However, integrating such projects in classrooms requires substantial logistical planning by instructors as well as adequate programming skills from students. To mitigate these challenges, we propose four model curricula to serve as accessible strategies of integrating practicable learning opportunities in lower-level CS classes. Depending on classroom circumstances, instructors can assign projects that involve student contributions to OSS, custom plug-ins, simulated open source communities, or practical code excerpts. As a result, students will be able to explore the utility of CS and discover an exciting future in computing.
Luisa Greifenstein, Isabella Graßl and Gordon Fraser. Challenging but Full of Opportunities: Teachers’ Perspectives on Programming in Primary Schools

Abstract: The widespread establishment of computational thinking into school curricula requires teachers to introduce children to programming already at primary school level. As this is a recent development, primary school teachers may neither be adequately prepared for how to best teach programming, nor may they be fully aware why they have do so.
In order to gain a better understanding of these questions, we contrast insights taken from practical experiences with the anticipations of teachers in training. By surveying 200 teachers who have taught programming at primary schools and 97 teachers in training, we identify relevant challenges when teaching programming, opportunities that arise when children learn programming, and strategies how to address both of these in practice.
While many challenges and opportunities are correctly anticipated, we find several disagreements that can inform revisions of the curricula in teaching studies to better prepare primary school teachers for teaching programming at primary schools.
Anna Ly, Jack Parkinson, Quintin Cutts, Michael Liut and Andrew Petersen. Spatial Skills and Demographic Factors in CS1

Abstract: Motivation: Prior studies have established that training spatial skills may improve outcomes in computing courses. Very few of these studies have, however, explored the impact of spatial skills training on women or examined its relationship with other factors commonly explored in the context of academic performance, such as socioeconomic background and self-efficacy.
Objectives: In this study, we report on a spatial skills intervention deployed in a computer programming course (CS1) in the first year of a post-secondary program. We explore the relationship between various demographic factors, course performance, and spatial skills ability at both the beginning and end of the term.
Methods: Data was collected using a combination of demographic surveys, existing self-efficacy and CS1 content instruments, and the Revised PVST:R spatial skills assessment. Spatial skills were evaluated both at the beginning of the term and at the end, after spatial skills training was provided.
Results: While little evidence was found to link spatial skills to socioeconomic status or self-efficacy, both gender identity and previous experience in computing were found to be correlated to spatial skills ability at the start of the course. Women initially recorded lower spatial skills ability, but after training, the distribution of spatial skills scores for women approached that of men.
Discussion: These findings suggest that, if offered early enough, spatial skills training may be able to remedy some differences in background that impact performance in computing courses.
Ethel Tshukudu, Quintin Cutts and Mary Ellen Foster. Evaluating a Pedagogy for Improving Conceptual Transfer and Understanding in a Second Programming Language Learning Context

Abstract: Motivation: Near novice programmers face challenges when learning a second or subsequent programming language (PL). A recently developed model for programming language transfer suggests that, at the early stages of relative novice programmers learning a new language, when their new language knowledge is limited, the prior language knowledge is generally the main source for perceiving syntax similarities and subsequent semantic transfer. Implicit learning by language association may result in both positive and negative transfer as well as lack of conceptual transfer from prior language to a new language. Although these transfer challenges are known, no attempt has been made to develop a pedagogy model that can guide educators in improving transfer in the classroom.
Objectives: We, therefore, propose a new transfer pedagogy that uses implicit, explicit, and bridging techniques which are aligned with the transfer model’s predictions.
Method: To evaluate this pedagogy, we conducted a between-subject study with a total of 62 second-year students who were transitioning from Python to Java. The study was for the duration of the first two and a half weeks of the Java course.
Results: We provide the quantitative and qualitative results on the effects of this pedagogy on learning programming concepts in the new Java language as well as the lecturer’s views on using the pedagogy. The results show that students who used the transfer pedagogy performed significantly better in the Java test than the control group in most concepts. The qualitative results showed that 88% of the students appreciated the explicit interventions with some noting they helped with avoiding transfer mistakes, also they believed it made them understand concepts better. The lecturer also appreciated the value of the pedagogy and expressed that they took it as an opportunity to help students learn deeper concepts. However, they expressed some challenges too.
Discussion: These findings suggest that the transfer pedagogy can be of benefit to second language learning and can be of value in teaching second programming languages.
Aleksi Lukkarinen, Teemu Lehtinen, Lassi Haaranen and Lauri Malmi. An Event Listener or an Event Handler? Students Explain Event-drivenness in JavaScript

Abstract: When students in programming courses are taught event-driven programming (EDP) for the first time, they face new terminology and concepts that they should internalize. Moreover, they learn a fully new approach for reasoning about program logic and execution order. However, there is a lack of research in students’ understanding of these concepts. In this paper, we describe a study, in which we asked web development students to explain their conception of EDP: what are the main concepts involved and how they interact. Moreover, we asked them to explain the execution of a short piece of JavaScript code that focuses on basic usage of events and event listeners. The answers, which we requested as concept maps and text, were analyzed using inductive content analysis. Our results clearly demonstrate serious shortcomings in the students’ learning and illustrate various misunderstandings that they may have regarding EDP. Based on the findings, we give suggestions for improving the teaching of EDP.
Augie Doebling and Ayaan M. Kazerouni. Patterns of Academic Help-Seeking in Undergraduate Computing Students<

Abstract: Knowing when and how to seek academic help is crucial to the success of undergraduate computing students. While individual help-seeking resources have been studied, little is understood about the factors influencing students to use or avoid certain resources.
Understanding students’ patterns of help-seeking can help improve the quality of help-seeking resources by identifying factors contributing to utilization for different groups of students. We present a mixed-methods study investigating the help-seeking behavior of undergraduate computing students. We collected survey data (n=138) about students’ frequency of using several resources followed by one-on-one student interviews (n=15) to better understand why they use those resources. Several notable patterns were found, including differences in help-seeking behavior between student demographic groups. These include women using office hours more frequently then men and computing majors seeking help from their peers more often than non-computing majors. Additionally, interview data revealed a common progression in which students started from easily accessible but low utility resources (online sources and peers) before moving on to less easily accessible, high utility resources (like instructor office hours).
Jin Kang, Adrian Chan, Chantal Trudel, Boris Vukovic and Audrey Girouard. Diversifying Accessibility Education for Graduate Students: Presenting and Evaluating an Interdisciplinary Accessibility Training Program

Abstract: There is a growing emphasis to educate STEM students about accessibility, so that they can become accessibility advocates. While many pedagogical approaches to teach accessibility are course-based, we introduce a community-based accessibility training program that brings together graduate students in STEM and related fields, called the Anonymous Program (AP). Going beyond academic degree training, this program includes five training components: (1) a graduate course on accessibility and inclusive design, (2) an Action Team Project (ATP), (3) a Retreat, (4) Workshops, and (5) a Symposium. As our initial program assessment, we analyzed 22 students’ written program reflection and found three themes that highlight what students learned about accessibility and professional skills (Theme 1: Learning Outcomes), what students planned on doing after the training (Theme 2: Future Endeavors), and how students want the program to improve (Theme 3: Program Improvement). We advance accessibility education by introducing an innovative training that embraces collaboration among local community, faculty, and multidisciplinary cohorts of graduate students.
Matti Tedre, Peter Denning and Tapani Toivonen. CT 2.0

Abstract: CT has been the central rallying point for K-12 computing education at least since the early 2010s. A quickly growing number of teachers, school administrators, and policymakers have joined the movement since the 2010s. A consensus is emerging over the conceptual landscape of CT.
But at the same time, machine learning (ML) has triggered some major changes in many sectors of computing. Children’s lives today are full of ML-driven services—take TikTok’s spot-on recommendations, Facebook’s ability to tag their friends in photos, and targeted advertisement, just to mention a few. All those use technology that cannot be explained by teaching children classical programming.
This paper demonstrates why and how a number of classical CT concepts, such as debugging, problem-solving workflow, correctness, and notional machines, need to be extended as an increasing number of applications and services combine classical programming solutions and ML solutions. The paper presents some practices of teaching ML in K–12 and discusses how these practices challenge the traditional views related to CT in K–12. Due to these challenges, there is a need to air our views on some elements of K–12 computing education that fit well traditional programming but fit poorly machine learning and other new computing paradigms.

Monica McGill, Rebecca Zarch, Stacey Sexton, Julie Smith, Christine Ong, Melissa Rasberry and Shelly Hollis. Evaluating Computer Science Professional Development for Teachers in the United States

Abstract: Teachers influence student academic achievement and play a role in impacting students’ self-efficacy, confidence, identity, and a sense of belonging. Teacher professional development (PD) is a key factor in enabling teachers to develop mindsets and tools that positively impact students. Teacher PD is also a key step in building capacity for computer science (CS) education in primary and secondary schools. Successful CS PD will meet their primary learning goals and enable teachers to grow their self-efficacy, asset and equity mindset, and overall interest in teaching CS.
As part of a larger study, we conducted a secondary analysis of CS PD evaluation instruments (\begin{math}n=14\end{math}). We found that instrumentation across providers were highly dissimilar with limited data collected for measures related to teacher learning. Likewise, they were limited in being connected to student learning CS and academic growth. As a way to enable PD providers to construct measures that align with known impacting factors, we offer a set of recommendations for collecting data for demographics and to measure program satisfaction, content knowledge, pedagogical content knowledge, growth and equity mindset, and self-efficacy. We also outline critical questions for PD providers to consider when constructing their evaluation instruments, including reflecting provider and participant community values, the goals of the PD, and how the data collected will be used to continually improve the CS programs.

Philipp Kather and Jan Vahrenhold. Exploring Algorithm Comprehension:Linking Proof and Program Code

Abstract: An algorithm consists of the description of a process solving a well-defined problem and a chain of reasoning regarding one or more properties of this process, such as efficiency or correctness.Often, this chain of reasoning is presented as a formal proof.Understanding the links between this reasoning and the process is a critical task for understanding algorithm comprehension.In this article, we present results from a qualitative study in which we explored how and where links between parts of the process and parts of the chain of reasoning are established by a reader attempting to comprehend an algorithm.We relate our findings to research in program and proof comprehension and observe that the connection between process and reasoning introduces another dimension of comprehension not covered by previous work.We identify potential mechanisms in this comprehension process and derive suggestions for how to support comprehension when teaching algorithms.
Ebrahim Rahimi, Bas Van Zadelhoff and Erik Barendsen. Principles to facilitate design-based learning environments for programming in secondary education while making learning visible in an authentic way

Abstract: Design-based learning (DBL) environments seem a promising instructional
approach to facilitating and motivating students’ learning
of programming concepts via applying them to develop various
digital artifacts such as games, websites, robots, and software applications.
However, there is a lack of theory-grounded principles
to facilitate DBL environments for programming in K-12. In the
absence of such principles, some essential elements of DBL such
as the reciprocal link between “design” and “learning” processes
might be easily neglected which can sacrifice “learning” in favor of
“making” and degrade constructionist-based learning-by-making
initiatives in programming to a form of “shallow constructionism”
or doing for the sake of making with no significant impact on the
students’ programming knowledge.
In this study we formulated a set of principles to facilitate DBL
environments for learning programming concepts in K-12 while
making learning visible in an authentic way (i.e., without breaking
the realistic design setting using separate tests), built upon
related educational theories and concepts, including Constructionism,
scaffolding, Context-based learning, Collaborative learning,
and design-based learning. Then, we instantiated these principles
to develop a course for learning about algorithms for secondary
school students. The developed course was implemented in five
secondary schools (with 5 teachers and 87 students) and evaluated
based on the experience of the participating teachers and students.
The results of the evaluation helped us to reflect on the applicability
of the principles and refine them.
Lovisa Sundin, Nourhan Sakr, Juho Leinonen, Sherif Aly and Quintin Cutts. Visual recipes for slicing and dicing data: teaching data wrangling using subgoal graphics

Abstract: The rising demand for data wrangling skills in today’s global market poses new challenges for the programming education community. Non-majors often need to learn it quickly alongside their other subjects. Previous research suggests that subgoal labels offer a powerful scaffolding strategy to help novices decompose problems. Because data wrangling is inherently easy to represent graphically, we wonder whether such labels could be augmented with subgoal graphics. To test this idea, we developed an online tutorial that features subgoal graphics in both programmatic and non-programmatic data wrangling exercises. Following an RCT paradigm, a control group is only given subgoal labels, without any graphics. The platform collects learner activity in order to evaluate the pedagogical benefits. Participants were recruited from multiple institutions (N=197, 134). Our results did not show a significant difference in various learner performance metrics, however subjective feedback from our participants suggest that learners perceive the graphics to be very helpful. We discuss possible reasons for the apparent disparity between objective and subjective data.
Samiha Marwan, Preya Shabrina, Alex Milliken, Ian Menezes, Veronica Catete, Thomas W. Price and Tiffany Barnes. Promoting Students’ Progress-Monitoring Behavior during Block-Based Programming

Abstract: Providing students with adaptive feedback on their progress on programming problems has been shown to motivate students and improve their performance, but little is known about how such feedback might impact student self-regulated learning during programming. Self-regulated learning (SRL) involves student planning a task, monitoring their progress, and reflecting on the outcome. We explored students’ SRL behaviors, particularly progress monitoring, when programming using each of three different scaffolds. The first scaffold is a subgoal checklist for the given programming task, the second adds automated, binary completion feedback on each subgoal, and the third adaptively reflects an automated percent progress estimate of student progress on each. Through interviews and programming logs from 17 students solving a problem in a block-based programming environment, we investigated the extent to which each scaffold supported student SRL. Our qualitative study results suggest that all three scaffolds could be useful for student SRL, but students felt that a combination of the checklist and progress feedback provided them with a balance of autonomy and motivation to persevere in programming. Furthermore, our results suggest that explaining how the automated feedback system works may have encouraged students to reason about the feedback they receive, which was a key intended outcome to improve SRL during programming.
Giulia Paparo, Marco Hartmann and Mareen Grillenberger. A Scratch Challenge: Middle School Students Working with Variables, Lists and Procedures

Abstract: With the “Lehrplan 21” school curriculum, a new subject “Media and Computer Science” was introduced in the German-speaking cantons of Switzerland. The curriculum defines competence areas that should be achieved by students within this subject. One such competence area is that middle school students should be able to develop executable and correct computer programs that use variables and subprograms. To evaluate and analyze how middle school students work with these more abstract CS concepts, we organized a Scratch Challenge – an online programming competition using the block-based programming language Scratch. We then compared the 203 submitted projects with an analysis of projects from the general Scratch repository. We found a similar use of types and quantities of blocks per projects (when considering projects developed in a short timeframe), but also an increased use of the targeted abstract concepts (especially of procedures), potentially influenced by the educational materials we provided for the challenge. Secondly, we examined the way middle school students work with these abstract concepts in Scratch, the errors they typically make, which challenges they face and the preconceptions they might have. These results contribute to the little studied area of middle school students working with more abstract concepts, such as variables, lists and procedures in Scratch. With this contribution, we hope to support schools to implement the aims of the “Lehrplan 21” successfully, and at the same time gain insight into the use of Scratch for teaching and learning more advanced CS concepts.
Teresa Busjahn, Simon and James H. Paterson. Looking at the main Method – An Educator’s Perspective

Abstract: There have been a number of studies on gaze tracking in programming, examining how people read program code when tasked with understanding it, but the implications for programming education are not always entirely clear. We tracked the gaze of both novices and experts while they were reading small Java programs, and subsequently interviewed some of the participants about that task. In analysing the interviews we noted that while experts typically say that they start by looking for the program’s entry point, novices appear not to follow such a purposeful approach. Subsequent analysis of the gaze data confirms this effect. Experts attend to the main method early on when reading a program without looking at much else beforehand. Furthermore, they read main more comprehensively than the rest of the program. Novices, on the other hand, arrive at main much later and only after having already read
a greater part of the code above main, which was located at the end of the code. We conclude that there is potential benefit in explicitly guiding novices in the art of reading code – teaching them how to read while trying to teach them how to write.
J. Ángel Velázquez Iturbide. An Analysis of the Formal Properties of Bloom’s Taxonomy and Its Implications for Computing Education

Abstract: Educational taxonomies provide conceptual frameworks to deal with different educational issues. In particular, Bloom’s taxonomy is popular in CSE, especially in programming courses for assessment purposes. However, different authors have reported difficulties of use, and have even suggested as hypothetical cause an inherent deficiency of the taxonomy. In this paper, we present an analysis of the formal properties of the two versions of Bloom’s taxonomy. According to their authors’ intents, the original taxonomy should be considered a comparative system, while the revised taxonomy would be a classification system derived from the product of two simpler classifications. However, our analysis shows that, among other deficiencies, the taxonomies are ill-defined as classification systems. Two major implications of the analysis are: (a) that instructors’ difficulties are primarily due to the taxonomies themselves, not to instructors, and (b) an identification of those features of the taxonomies that are deficient, and that therefore allow explaining the difficulties reported. Finally, we discuss some implications of these findings, both in general and for CSE, as well as possible future actions.
Taj Muhammad Khan and Syed Waqar Nabi. English versus Native Language for Higher Education in Computer Science: A Pilot Study

Abstract: While the official language of instruction for many higher education programs in Pakistan, an ex-British colony, is English, in practice the language of instruction can vary from English-only, to some combination of English and a native language, to native language only. There is a lack of consistency not just across institutions, but also across classrooms in the same institution. A more consistent, coherent and evidence-based approach is required. We have conducted a pilot study based on a small cohort of computer science students, who happened to have some of the lectures of the same course delivered in Urdu, and the others in English. Based on a questionnaire response, we investigated students’ self reported oral and written communication skills, as well as their preference of language for lectures. We found, albeit based on a very limited, piloted study, that using Urdu, the first language of most of our students, as the medium of oral communication (lectures, general classroom communication) should be preferred, whereas English should remain the language of choice for written communication. We expect to expand this study to multiple classrooms across institutions in Pakistan so that more definitive conclusions can be derived, hopefully leading to more evidence-based policy in the future.
Børge K. Gjelsten, Gunnar R. Bergersen, Dag I. K. Sjøberg and Quintin Cutts. No Gender Difference in CS1 Grade for Students with Programming from High School: An Exploratory Study

Abstract: Programming is an increasingly important skill in the 21st century. Therefore, many education systems internationally offer non-compulsory programming (NCP) courses during high school years. Aim. Our goal is to study the effect of NCP along with other high-school courses and self-reported affective factors and prior programming experience on first-semester student performance in CS1. Method. A total of 232 students from a Norwegian university were involved in the study. Grades from high school from the public student registry, questionnaire responses, and CS1 grades were analysed. Results. The students with NCP performed significantly better in CS1 than those without (average grade 4.4 vs. 3.6, where A, B,…, F is coded as 5, 4,…, 0). For women the difference in performance with and without NCP was 4.4 vs. 3.2, for men it was 4.4 vs. 3.8. Conclusion. This study shows the striking result that for students with NCP, the notorious gender difference in CS1 performance was absent. Other results merit further considerations regarding gender differences and the effect of NCP.
Jack Wrenn and Shriram Krishnamurthi. Reading Between the Lines: Student Help-Seeking for (Un)Specified Behaviors

Abstract: A thorough understanding of specified behavior is essential for the completion of most programming tasks. Researchers have created automated tools to help students with this task. Yet, even with automated feedback, students may still face self-insurmountable challenges for which they must seek aid from the course staff.
What self-insurmountable challenges do students face? And (how) does access to automatic, on-demand feedback shape student help-seeking? To find out, we manually reviewed the 1,247 assignment-related student posts in the online help-forum of a post-secondary accelerated introductory computer science course. We report on the high-level relationships between student help-seeking and (under)specification in assignments, and identify a number of behaviors relevant to both researchers and educators.


Daphne Miedema, Efthimia Aivaloglou and George H. L. Fletcher. Exploring the Prevalence of SQL Misconceptions: a Study Design

Abstract: The Structured Query Language (SQL) is an established language for data manipulation in relational databases. It is widely used in industry, and therefore part of the typical Computer Science curriculum. From the large amounts of mistakes higher education students make while learning and using SQL, we know that this language is not easy to learn. Various researchers have examined the types of mistakes SQL novices make, and recently, the first step towards understanding the origins of these mistakes has been made. In this poster abstract, we propose a study to examine the prevalence of these origins, also called misconceptions. We hope the Computer Science Education community will help us reflect on and strengthen our methodology, and ultimately, our findings.
Friday Joseph Agbo, Solomon Sunday Oyelere, Jarkko Suhonen and Markku Tukiainen. iThinkSmart: Immersive Virtual Reality Mini Games to Facilitate Students’ Computational Thinking Skills.

Abstract: This paper presents iThinkSmart, an immersive virtual reality-based application to facilitate the learning of computational thinking concepts. Computational thinking skills such as problem decomposition, abstraction, algorithmic thinking, pattern recognition, and recursive thinking are necessary skills for 21st-century learners. As a way to supplement the traditional teaching and learning of computational thinking, iThinkSmart was developed with three virtual mini games, namely, River Crossing, Tower of Hanoi, and Mount Patti treasure hunt, to foster immersion, interaction, engagement, and personalization for an enhanced learning experience. The system was developed to be played on a smartphone with a Goggle Cardboard and hand controller. This first prototype of the game accesses players’ competency of computational thinking by applying the Objective Distance model and renders feedback based on the outcome during the gameplay. We hope to extend the prototype after the initial evaluation is conducted, which will provide an opportunity for further evaluation that can concretize the efficacy or otherwise of the application.
Denis Zhidkikh, Janne Fagerlund, Marika Peltonen and Mikko Vesisenaho. “CodeInnova”: A Unified Framework for Teaching Programming and Computational Thinking In Primary Schools

Abstract: Teaching programming and computational thinking (CT) in primary schools have become more common in the last decade. However, there has been little international consensus on what teaching the topics entices. We present CodeInnova, a framework developed jointly between four partnering countries for teaching programming and CT in K–9. In this poster, we present the curriculum for teaching CT and the accompanying teaching materials developed in CodeInnova. We also discuss preliminary results of testing the developed resources in classrooms.
Bernhard Standl and Nadine Schlomske-Bodenstein. Investigating pre-service teachers’ self-efficacy in solving programming tasks within one semester

Abstract: The study presented in this paper examines the development of self-efficacy with regard to solving tasks on algorithmic thinking and programming. N = 8 novices in programming are assessed at three measurement points within one semester each time before and after they are put into a situation to solve algorithmic thinking and programming tasks. Their assumptions on being confident in solving the task and possible indicators of why the programming task was perceived as easy or not are qualitatively assessed. In order to get a deeper insight into the instructional setting, the instructional quality, task quality, and student-centered learning climate was additionally assessed. The results of this study provide implications for designing learning and instructional design in pre-service computer science teachers’ seminars.
Susanne Podworny, Yannik Fleischer, Sven Hüsing, Rolf Biehler, Daniel Frischemeier, Lukas Höper and Carsten Schulte. Using data cards for teaching data based decision trees in middle school

Abstract: Using the example of which foods are rather recommendable or rather not recommendable, students can explore how computers are trained based on data to “make decisions”. Students can get a first idea about how artificial intelligence and machine learning can work. To do this, students create decision trees and understand that a decision tree is a rule system, which is derived from data. With our innovative approach of using data cards to teach about machine learning the students can explore the concepts by enactivly preparing the data and creating their own visualizations for reasoning. In doing so, they learn about a method of AI, the role of data and humans in this process, what mistakes can happen and how to deal with them.
Joseph Maguire and Rosanne English. Opportunities to Fail: Using Peer-review to support Assessment Literacy in Cyber Security

Abstract: The importance of cyber security to the global economy has only grown in recent years. Effective cyber security technical policy is an important defence against numerous threats. Consequently, it is important that future computing science and software engineering graduates are able to produce effective cyber security policy.

However, written assessments, such as cyber security policy, for some senior computing science students may be challenging due to poor assessment literacy for such assessments. The lack of familiarity with assessments not only has the potential to produce poor results, but could led to disappointment and frustration. In this poster, the practice of integrating peer-review as part of a cyber security policy assessment task to support assessment literacy is presented. The aim is elicit feedback from conference participants not only on the practice itself, but on addressing the challenge of assessment literacy in the context of cyber security policy.

Zihan Wu, Barbara Ericson and Christopher Brooks. Regex Parsons: Using Horizontal Parsons Problems to Scaffold Learning Regex

Abstract: Regular expressions (regex) are a popular text processing method used in programming, and are widely used in data analysis, web scraping, and input validation, and are supported by all mainstream programming languages. However, both students and even professional programmers perceive writing regex as difficult. Meanwhile, Parsons problems are a type of code completion problem used for introductory level programming education and there is evidence that they can be a more efficient way to practice programming. Since regex is also a formal language, we created a horizontal version of Parsons problems to support teaching regex. To evaluate its effect on students, we compared this new type of Parsons regex problems with traditional write regex problems in a MOOC setting and present first results in this poster.
Olli Kiljunen. Teaching Students to Fix Programming Errors with Tutorials Embedded in an IDE

Abstract: While modern software development tools provide programmers with features that help locating and fixing errors in the code, using these tools is not necessarily easy for novice programmers. We call for methods of instruction that teach students how to fix errors in practice using a modern, industry-grade IDE. We suggest an approach—utilizing tutorials embedded in an IDE—that we think could be suitable for this purpose. Our ongoing and future work include implementing and evaluating that approach.
Natalie Kiesler and Benedikt Pfülb. The Boolean Dilemma: Representing Gender as Data Type

Abstract: Unfortunately, the Boolean data type is still used in teaching and learning scenarios as default for the distinction of male or female gender.
This demo paper focuses on the identification of the challenges associated with assigning binary gender identities as part of programming exercises.
Due to an increasingly diverse society and computer science community, the authors advocate for new approaches, such as including aspects of diversity into the curriculum, exercises, educators’ mindsets and students’ socialisation in higher education.
We also encourage the use of other data structures than the Boolean data type when referring or assigning gender identity, as both students and educators can benefit from an adequate and gender-sensitive CS education.
Jonathan Calver, Paul Muir and Thomas Fairgrieve. Improving Student Takeaway in an Introductory Numerical Analysis/Scientific Computing Course: A Threshold Concepts Approach

Here we describe a project whose goal is to address issues associated with student takeaway, i.e., enduring learning, in an introductory course in Numerical Analysis/Scientific Computing (NA/SC) commonly taught in undergraduate programs in Computer Science, Mathematics, and Engineering. We have employed the well-known framework of Threshold Concepts (TCs) in order to identify essential ”takeaway” concepts in an introductory NA/SC course. We report on the four TCs we have proposed for NA/SC and discuss how the TC framework can be used to improve student takeaway.
Muyu Sandy Wang, Naaz Sibia, Ilir Dema, Michael Liut and Carlos Aníbal Suárez. Building a Better SQL Automarker for Database Courses

Abstract: This work introduces and demonstrates the viability of a novel SQL automarking tool (“SQAM”) that: (1) provides a fair grade to the student, one which matches the student’s effort and understanding of the course material, and (2) to provide personalized feedback, allowing the student to remain engaged in the material and learn from their mistakes while still being in that headspace. Additionally, we strive to ensure that our tool maintains the same standards (grade and feedback) that a highly qualified member of teaching staff would produce, so we compare and contrast our automarker’s results to that of teaching assistants over several historic offerings of the same database course at a large research intensive public institution, while reducing the grading time, thus enabling the teaching staff to channel more time into instruction.
Furthermore, we describe SQAM’s design and our model which applies the aggregate result of four different string similarity metrics to compute solution similarity in conjunction with our discretization process to fairly evaluate a student’s submission. Our results show that SQAM produces very similar grades to those which were historically given by teaching assistants.

Doctoral Consortium Posters

Lukas Höper. Developing and evaluating the concept data awareness for K12 computing education

Abstract: Students often have a lack of understanding and awareness of where, how, and why personal data about them is collected and processed. Especially, when interacting with data-driven digital artifacts, an appropriate perception of the data collection and processing is necessary for self-determination. This dissertation deals with the development and evaluation of a concept called data awareness which aims to foster students’ self-determination interacting with data-driven digital artifacts.
Sven Hüsing. Epistemic Programming – An insight-driven programming concept for Data Science

Abstract: The thesis referred here describes a didactic concept that focuses on using, adapting, and programming data-driven digital artifacts to gain new insights about the own world. In terms of orientation, this approach differs from those that focus on creating software or learning how to use data as an end in itself. Instead, in the context of authentic data projects, students learn to use computer science methods as a magnifying glass to explore and analyze the world. To achieve this, they need to learn, in particular, about the use of these data-driven digital artifacts and about the data itself. By conducting such authentic programming projects, students are supposed to gain a differentiated perspective of programming in terms of getting to know another view that differs from the more product-orientated one, often imparted in school today. The concept, therefore, highlights the role of data and digital artifacts regarding the active acquisition of knowledge and therefore illustrates the general educational character of programming. The goal of the respective dissertation is therefore to evaluate, how far this programming concept changes students’ and teachers’ attitudes towards programming and enables students to actively gain insights about their own world.
Michael Lenke. Preconceptions of Artificial Intelligence and their change through co-constructed explorations

Abstract: While most XAI approaches focus on mathematically correct and complete explanations, users have individual explanation needs. This is why everyday explanations are social practices that are influenced by many factors, e.g. the social and institutional context of the explanation. To overcome this shortcoming in AI education, explanations need to be co-constructed by the explainer and the explainee to be adapted to the requirements and needs of the learning person based on concrete and simplified examples – especially in educational contexts where there is necessity for general AI education.
This Ph.D. project investigates the effects of co-constructed explanations created while exploring notional machines on the mental models of AI of everyday people. The goal is to identify aspects of AI technology that are important for everyone, to empower people to become a responsible and active member of society in a digitalized world where AI technology is ubiqitious.
Salseng Mrong. Tool for enhancing Artificial Neural Network education

Abstract: Artificial Neural Networks (ANN) has applications in several branches of Computer Science, Machine Learning (ML), Machine Vision and Robotics for instance. Despite the popularity, it is often quite challenging to understand the inner workings of the Neural Networks because of the complex mathematics involved in the algorithm. In this article a tool for enhancing the ANN education. The tool utilizes an open-source hardware Arduino and a computer based visualization tool. The Arduino based robot is trained using a simple neural network and the training process is visualized on a computer.
Ioanna Bouri and Sanna Reponen. Elements of AI: Busting AI Myths on a Global Scale

Abstract: Considering the quick technological development in our society, as well as the continuous demand for workforce equipped with sufficient technology skills, self-paced online learning materials on hot topics like artificial intelligence (AI) play an important role in filling knowledge gaps and supporting life-long learning. In addition to technology professionals, citizens with all kinds of backgrounds have the right to gain the basic understanding in ground-breaking technologies, in order to have an equal and balanced say in what kind of technological solutions we should have in the future.
Furthermore, the need for quality distance learning has been accelerated by the COVID-19 pandemic, and massive, open online courses (MOOCs) have the capability to cater for both academic students and the general public searching for reskilling and upskilling opportunities.
Ismaila Temitayo Sanusi. Intercontinental evidence on learners’ differentials in sense-making of machine learning in schools

Abstract: Given the importance of machine learning for K-12 levels, finding out ways to communicate the concept to students such that it will be less intimidating is necessary. Drawing on qualitative methodology approach, this study aims to explore how to teach machine learning in K-12 context using middle school students’ samples in Nigeria, Finland, and United States. Considering the cross-contextual approach and the study aim, this research will be a valuable addition to the limited evidence that support differentials in students learning of machine learning technology across culture and background. The study outcome will be an indispensable resource for addressing how the uniqueness of each contexts can be leveraged on to best introduce machine learning to schools.