Accepted papers and posters 2020
On Programming Competence and its Classification
Abstract: When the concept of competence entered the discourse on learning and teaching in computing education, modeling and classifying programming competences became a new research area.However, universal catalogs of competencies as a basis for curriculum design in Computer Science (CS) do not yet exist. In this paper several educational taxonomies are discussed with regard to their application in CS.Additionally, an empirical study is considered, that gathered expected competencies of introductory programming courses and by means of interviews with university professors.Both of these data sets were analyzed using Mayring’s qualitative content analysis and the categories of the Anderson Krathwohl Taxonomy (AKT) for learning, teaching and assessing. A summarized competency structure model is presented as a continuation of this empirical research. In terms of the knowledge and cognitive process dimensions of the AKT, the empirical analysis proved its suitability for the classification of programming competences. Therefore, this paper outlines the advantages and superiority of the AKT in contrast to other models proposed in the related works. Moreover, an adapted version of the AKT with sub-types and examples for programming is presented in order to help support CS educators in the design of future programming classes and assessments.
Towards an Assessment Rubric for EiPE Tasks in Secondary Education: Identifying Quality Indicators and Descriptors
Abstract: The ability to explain program code has shown to correlate with code writing skills. Explain in Plain English (EiPE) tasks have the potential to serve as learning activities, in order to develop and enhance skills that are critical for programming (e.g., program comprehension and abstraction). However, assessing students’ work and providing clear feedback on these tasks can be challenging, as quality indicators are not well established.This work aims to fill this gap by investigating indicators and descriptors which characterize the quality of students’ code explanations in secondary education. Firstly, we carried out a literature review to identify quality characteristics and descriptors that differentiate the quality of students’ responses in tertiary education. Then, we analyzed the quality of upper-secondary students’ responses to a simple EiPE task. The results of that analysis were used to refine our quality characteristics and descriptors, which were then translated into a prototypical rubric. In the final phase, the rubric was iteratively tested by two teachers and improved based on their feedback. The final rubric prototype can be used to provide feedback during formative assessment.
Disguising Code to Help Students Understand Code Similarity
Abstract: To act with academic integrity in programming, students need to understand the concept of code similarity and the aspects that contribute to it, because undue similarity is often used as a first step in detecting plagiarism or collusion. However, if students are told at all about code similarity, it is typically in very general terms, which might not be clear to all students. This paper proposes CSD, a tool to independently educate students about code similarity through the application of semantic-preserving disguises. Given Java or Python code, CSD will disguise the code without changing its semantics. CSD can apply 60 types of disguise covering five code components: comments, white spaces, identifier names, data types, and constants. The tool will be integrated in a submission system, so that each time a student submits their code, they will be shown a version of the code with some disguises applied, along with a related explanation. This is expected to motivate further use of the tool while educating students about the futility of trying to copy and disguise code.
Fostering Knowledge of Computer Viruses among Children: the Effects of a Lesson with a Cartoon Series
Abstract: Children increasingly use computing devices. However, it is unclear whether they have basic knowledge of security-related issues such as computer viruses and in case they do not, what they can learn about them. It was found previously that Czech 8-year-olds have only limited knowledge of computer viruses, but neither naïve understanding of older children nor what they can learn has been researched. Here, we first examined preconceptions of computer viruses among Czech 5-6-graders (N = 14) and German 3-4-graders (N = 28) by means of a written test. Second, the German sample (experimental group), but not the Czech one (control group), received an intervention to learn about computer viruses, antiviruses, and software updates by means of a 45-min lesson combining a cartoon series on viruses, frontal instruction, and discussion. Both groups again completed the written test. A joint analysis of both samples indicated that Czech and German children already knew key points concerning computer viruses. These included, for instance, that viruses harm our computers (88% of the total sample). However, overall, their knowledge was patchy, and children also had misconceptions such as that viruses can only infect devices connected to the internet (57%), and antiviruses can delete viruses from the internet (40%). Due to the intervention the experimental group improved from pre to posttest (d = 1.06), while this was not the case for the control group. A more in-depth analysis indicated that knowledge gains were mostly related to information repeatedly mentioned during the lesson, but it was less clear whether the lesson helped to correct previously held misconceptions. Taken together, the results indicated that knowledge of computer viruses should and can be taught to primary school children, but attention must be paid to existing preconceptions.
Semi-Automatically Mining Students’ Common Scratch Programming Behaviors
Abstract: In this paper, we introduce a semi-automatic approach to enable scalable data collection and analysis of students’ programming processes in Scratch. We describe a logging tool that records students’ interactions with the Scratch environment as they code. We analyze these programming interaction logs using a combination of automated and manual techniques to uncover patterns that provide insights into common programming behaviors among users during the coding process. We demonstrate this semi-automatic logging and mining approach with a case study, in which students performed the same open-ended coding task in two college courses introducing programming (37 Education and Human Development majors taking an Ed Tech course and 44 non-CS majors taking a general introductory CS course). Our findings demonstrate that we can uncover patterns that help identify which constructs students commonly used for a given task, various types of tinkering behavior within a given task, and some indications of students’ levels of understanding of the blocks. This kind of information is a valuable starting point for where and how to direct teaching efforts.
Weekly Open-Ended Exercises and Student Motivation in CS1
Abstract: Improving student engagement and learning within introductory Computer Science (CS1) courses has been heavily studied within CS Education (CSE) research due to their fundamental role in a student’s early experiences with learning foundational CS concepts. One issue with many CS1 courses is that the practice of standardized grading leads to limited opportunities for students to find personal relevance within school projects, which could be demotivating for beginner students. This paper examines the impact of incorporating opportunities for creative thinking into a CS1 course through the use of weekly open-ended exercises. Data was collected from 284 students in a CS1 course, where roughly half of the students completed weekly exercises which included an open-ended aspect that allowed them to make their own decisions about their projects. The other half completed similar exercises but with a specifically defined, closed-ended checklist of requirements. Although no effect on academic performance was found, an analysis of survey data collected throughout the course revealed a significant connection between open-ended exercises and increased motivation, especially in terms of confidence and satisfaction. A deeper look at each particular exercise showed variances in how motivating an exercise was based on time in the semester and structure of the exercise, revealing important lessons about the design of effective open-ended CS1 exercises.
To Be or Not to Be a Teacher? Exploring CS Students’Perceptions of a Teaching Career
Abstract: There has been an increasing tendency and an interest to make computer science (CS) a core subject in the K-12 curriculum, however in many countries there is a CS teacher shortage in high schools. In order to define the possible reasons for CS teacher shortage, we investigated the factors that influence CS students’ career choices and their preferences about teaching. We also studied the requirements students view as necessary. Semi-structured interviews were conducted with 18 CS bachelor/ master students. The results show. that the most important factors influencing the career choice of CS students are organization related expectations such as the characteristics of the organization (scale, reputation etc.), the field ofexpertise and salary expectations. Regarding CS students career plans about becoming a teacher, a large part (almost 75%) of the students stated that they definitely do not think of teaching or teaching is not a priority career goal. The most cited demotivating factors for following a teaching career are the nature of the teaching job and low salaries. Perceived required skills for good CS teachers are frequently defined as pedagogical skills and social skills.
Student’s Rating of Contexts for Teaching Data Literacy at School regarding the Context Characteristics relation to everyday life and uniqueness
Abstract: Preparing young people for their future with digital competenciesis an important goal for all educational systems. Competency frameworksalready exist in many countries and function as guidelinesfor teaching at school. A central area is data and informationliteracy, which also plays a key role in computing (education). As aschool relevant topic it should therefore be examined more closely.For any learning activity, the interest in what to learn has a majorimpact on learning motivation and success. But the developmentof interests in school subject contents is also affected by its teachingcontexts. In this respect, it is an open question whether thesecontexts should be related close to the everyday life of the studentsor whether they should be unique to have a positive effect on thesituational interest of the students. Results can help selecting motivationalteaching contexts and to facilitate the learning process,making it easier for students to acquire the required competencies.This paper presents results of a pilot study (N = 28), which is partof a bigger research project consisting of two main studies. Thispilot study belongs to the first main study. We present results onhow 7th and 8th graders rated 12 contexts for teaching data literacyregarding the relation to their everyday life or uniqueness. This pilotstudy primarily serves to test a self developed questionnaire andits comprehensibility to use it in the first main study. It also givesfirst insights into expected results of the upcoming main study.
From the Mathematical Impossibility Results of the High School Curriculum to Theoretical Computer Science
Abstract: The academic results that students of computers science degrees obtain in their learning of subjects of theoretical computer science are traditionally low, as a consequence of the difficulties students experience due to their high mathematical level or their little theoretical background. In order to reverse this problematic situation in theoretical computing education, it is essential that students acquire an intuitive and progressive knowledge of the main theoretical computing concepts and their associated skills and abilities, even before they finish secondary school. To achieve this goal, in this paper we offer a novel methodology to systematically introduce questions of computability and algorithmic complexity in the curriculum of the final years of high school education. We propose to start from those results of impossibility that are already included in the curriculum of mathematics courses, and around these classic theorems of impossibility, conveniently adapted and presented to students, identify motivating and interesting theoretical computing questions. To provide evidence of the impact of the applicability of the proposed methodology, we analyze the experimental results we have obtained to confirm that the introduction of theoretical computing questions from impossibility results increases students’ academic grades. Moreover, the introduction of these questions in computing education increases motivation and interest in studying computer science subjects of upper high school levels and first level of university degrees beyond traditional programming courses.
Analysis of Programming Assessments – Building an Open Repository for Measuring Competencies
Abstract: Within different approaches and aims to teach programming, context-specific programming languages are used, which might support different paradigms. Therefore, we are developing a framework for modelling programming competencies regardless of the used language or paradigm. In this paper, we present the work we have done to support our theoretical model by building an open repository for measuring competencies. Our goal is to make use of already existing assessments for programming by evaluating their quality and fit to our competency framework. Therefore, we conducted a systematic literature review to find assessments present in the ACM digital library. We also developed a scheme for evaluating the quality of the assessments following three criteria (objectivity, reliability, and validity) and a scheme for evaluating their fit to the competency framework. Out of 23 papers describing 26 assessments, 13 were publicly accessible and considered for an in-depth analysis. The in-depth analysis showed that all fit to our competency framework with an average coverage of 39% of all concepts. Regarding the quality of the assessments, three reported the reliability by evaluating Cronbach’s alpha and five the validity by using different methods. To expand our open repository and to improve our framework for modelling competencies in the field of programming we plan a five step program: analyze more, develop a guide, fill gaps, specialize and replicate assessments. We hope that providing this framework will foster the development of competency models in the field of programming.
Exploring the Bug Investigation Techniques of Intermediate Student Programmers
Abstract: Bug investigation – testing and debugging – is a significant part of software development. Ineffective practices of bug investigation can greatly hinder project development. Therefore, we seek to understand bug investigation practices among intermediate student programmers. To this end, we used a mixed-methods approach to study the testing and debugging practices of students in a junior-level Data Structures course with 3–4 week long projects. First, we interviewed 12 students of varying project performances. From the interviews, we identified five techniques that students use for both testing and debugging: 1) writing diagnostic print statements, 2) unit testing, 3) using source-level debugger, 4) submission to online auto-grader, and 5) manual tracing. Using the Grounded Theory approach, we developed four hypotheses regarding students’ use of multiple techniques and their possible impact on performance. We used clickstream data to analyze the level of use of the first four of these techniques. We found that over 92%, 87%, and 73% of the students used JUnit testing, diagnostic print statements, and the source-level debugger, respectively. We found that the majority of the students (91%) used more than one technique to investigate bugs in their projects. Moreover, students who used multiple techniques had overall better performance in the projects. Finally, we identified some ineffective practices correlated with lower project scores. We believe that the findings of our research will help understand, identify, and teach better practices in bug investigation.
Preprocessing for Source Code Similarity Detection in Introductory Programming
Abstract: It is well documented that some students either work together on programming assessments when required to work individually (collusion) or make unauthorised use of existing code from external sources (plagiarism). One approach used in the detection of these violations of academic integrity is source code similarity detection, the automatic checking of student programs for unduly high levels of similarity. Preprocessing of source code files has the potential to increase the effectiveness, the efficiency, or both, of the source code comparison process. There are many possible steps in the preprocessing, and examination of the literature suggests that these steps are selected and implemented without any empirical evidence as to their value. This paper lists 19 preprocessing steps that have been used in code similarity detection, and assesses the effectiveness and the efficiency of most of these steps on data sets of student programs from introductory programming courses. The results should help researchers to decide what preprocessing steps to include when designing source code similarity detection techniques or software. According to the study, identifier removal increases both effectiveness and efficiency. Token renaming and syntax tree linearisation increase effectiveness at a cost of efficiency. Other preprocessing steps are dependent upon characteristics of the data set and should ideally be empirically tested before being applied. The paper should also help alert programming educators to the sorts of disguise that students can apply to copied programs.
Mindset and Study Performance: New Scales and Research Directions
Abstract: Beliefs that people associate to themselves, often called implicit theories, self-theories, or mindsets, are a fundamental source of bias in human thinking. Research shows that mindsets radically shape human thinking, behaviour, and progress. There is strong evidence about the impact of mindset to personal development and academic achievement. Mindset is also a well acknowledged research domain in computer science education. In most cases, previous mindset research has been conducted in the context of one or two courses, typically introductory programming. Also, mindsets are typically measured only in the intellectual domain, while mindset on eg. soft skills is not researched almost at all. In this research, we investigated the relationships between mindset and study performance in first year computer science studies in University of \emph{omitted for review}. Also, we investigated change in mindset during the first year of studies. In addition to scales on the intellectual domain, we introduced new scales on social skills and creativity. Our results show that mindset in intelligence and mathematics got more fixed during first year studies, while mindset on computing remained growth oriented. Mindset on creativity was the most fixed of all scales. Fixed mindsets on social ability, creativity, and computing were moderately associated with study performance. A number of future research directions are proposed.
Beauty and Joy of Computing: CS Principles Professional Development
Abstract: Since the Advanced Placement Computer Science Principles (AP CSP) course has been released, it has vastly increased the need for highly trained CSP teachers who are prepared to bring CS to a diverse group of students. We have designed professional development (PD) workshops for high school teachers learning to teach this new CSP course, basing our design and iterative refinements on effective practices from other STEM disciplines. In summers 2012-2019, we have prepared over 600 teachers to teach CSP. Our PD provides teachers with time to learn CS content and pedagogical content knowledge. A key component of our PD design focuses on professionally-relevant activities– specifically, teachers develop and lead CSP lessons and provide feedback to each other through practice-focused discussions with experienced teachers and their peers. Another key component of our PD is including opportunities for continued professional growth, where we provide opportunities for teachers to engage as leaders who mentor others, curate materials, or facilitate future PDs. Our data has shown an increase in teachers’ confidence to lead in the classroom and also shown equal accessibility and growth for participants regardless of prior programming experience. In this paper, we examine and articulate the foundational theories for our CSP PD and how we have adapted these methodologies to cultivate an inclusive and productive learning environment for teachers. We also perform a retrospective analysis to determine which PD and program activities our teachers found most meaningful and relevant to their daily teaching.
Student Refactoring Behaviour in a Programming Tutor
Abstract: Producing high-quality code is essential for professionals working on maintainable software. However, awareness of code quality is also important for novices. In addition to writing programs meeting functional requirements, teachers would like to see their students write understandable, concise and efficient code. Unfortunately, time to address these qualitative aspects is limited. We have developed a tutoring system for programming that teaches students to refactor functionally correct code, focussing on the method-level. The tutoring system provides automated feedback and layered hints. This paper describes the results of a study of 133 students working with the tutoring system. We analyse log data to see how they approach the exercises, and how they use the hints and feedback to refactor code. In addition, we analyse the results of a student survey. We found that students with some background in programming were generally able to identify issue in code and solve them (on average 92%), that they used hints at various levels, and we noticed occasional learning in recurring issues. They struggled most with simplifying complex control flow. Students generally valued the topic of code quality and working with the tutor. Finally, we derive improvements for the tutoring system to strengthen students’ comprehension of refactoring.
Investigating Students’ Preexisting Debugging Traits: A Real World Escape Room Study
Abstract: Being able to find and fix errors is an essential skill in computer programming. Nevertheless, debugging poses a major hurdle in the K12 classroom, as students are often rather helpless and rely on the teacher hurrying from one student-PC to the other. Overall, there is a lack of respective concepts and materials for the classroom as well as research on how to teach debugging. According to the constructivist learning theory, teaching and developing concepts and materials for the classroom must take learners’ preexisting experience into account to be effective. In their daily lives, students are confronted with errors long before they build programming experience: Whether there is a problem with “the internet” or with their bicycle, they are troubleshooting and locating and fixing errors. Debugging is a special case of general troubleshooting and shares common characteristics, such as the overall process or particular strategies. Thus, the aim of this study is to investigate students’ preexisting debugging traits. To this end, we developed a real-world escape room consisting of debugging-related troubleshooting exercises. This allows us to observe students’ troubleshooting process, strategies, and overall behavior in a natural environment. Building upon this, we employed the escape room approach with around 150 high school students and analyzed the resulting video data. Based on the data we identify preexisting debugging traits such as students struggling to generate hypotheses or to undo changes. Furthermore, they are not able to effectively test a system and struggle with cognitive load in topographic search. Therefore, our study firstly contributes to understanding and explaining the behavior of novice debuggers. The second contribution is an innovative methodology to analyze preexisting debugging traits.
Equalizing Data Science Curriculum for Computer Science Pupils
Abstract: Data science is a new interdisciplinary field of research that focuses on extracting value from data and integrates knowledge and methods from computer science, mathematics and statistics, and the domain knowledge of the data. As an inherently interdisciplinary field, there are many ways to assemble a data science faculty or school and, accordingly, many views on how to teach data science.Due to the recently acknowledged importance of data science, several initiatives for designing high school data science curricula have emerged recently. Data science education requires advanced knowledge and skills in mathematics, statistics, computer science, and the data domain. Since it is unrealistic to teach these advanced topics at the high school level, the high school data science curriculum focuses on a broad understanding of the data science workflow rather than on mathematical and algorithmic details that characterize undergraduate programs.High school computer science pupils, however, are expected to have a deep understanding of algorithms, as algorithms are the heart of computer science. In this spirit, we designed a data science curriculum that balances the need to teach both the breadth of data science workflow and the depth of data science algorithmic facets, specifically, machine learning algorithms. We validated the fitness of this curriculum for 10th grade computer science pupils using a three-year pilot program taught in a public school, and a summer workshop for computer science teachers.
Mathematics, Computer Science and Career Inclinations — A Multi-Institutional Exploration
Abstract: Mathematics is at the heart of computer science as a discipline, yet not all students appreciate the relevance and importance of mathematics in a computing degree. Recent work presented a hypothesis that a student’s career inclinations may impact their view towards the value of learning mathematics, and thus their course selections. However, empirical support for this theory has previously been limited to a single institution in which students are required to include mathematics as part of their degree. In this work, we conduct a replication study in a different institutional context where mathematics is not a strict requirement for computer science students. We find robust support for the previously hypothesized relationship between career inclinations and perceptions of mathematics, and a strongly stated desire from students that the mathematics they learn should be more clearly connected and relevant to computer science.
High School Teachers’ Understanding of Code Style
Abstract: The introduction of computer programming into schools has meant that many teachers are expected to become familiar with computing concepts and techniques. For senior students, one important concept is \textit{code style}. Research indicates that teachers of senior high school programming students often struggle when providing feedback on code style and this means students may not be suitably prepared to study computing at tertiary level. Our goal is to understand the challenges faced by teachers in order that we might help them increase their effectiveness. In this paper, we describe a study in which we interviewed 8 teachers from disparate backgrounds. We overview the main themes that emerged from the first level analysis of the data. This analysis revealed several phenomena that require a deeper analysis. We describe one of these, the tendency for teachers to focus on syllabus requirements as opposed to understanding aspects of code style at a deeper level. Although our study takes place in the New Zealand (NZ) environment, we believe the kinds of issues exposed are generally relevant to programming teachers world-wide.
Digging into Computer Science Students’ Learning Journals
Abstract: Motivation. Learning journals have the potential of providing important pointers to improving students’ learning in Computer Science (CS) higher education.Objectives. We aim to identify attributes of CS students’ notes and study their effect on course performance.Method. We use a mixed method approach to analyse notes from 183 student journals. The analyzed notes, containing over 100,000 words, represented content of one lecture of an undergraduate course on introduction to software development.Results. We find students’ notes are mostly verbatim with little self-reflection, albeit presenting a rich variety of content highlighting and outlining. Statistical results indicate little correlation among journal attributes and course performance components.Discussion. We give recommendations for instructors and students relating to note-taking, and provide details to support replication for the CS education research community.
Employability Through Imagination, Alignment, and Engagement – Students’ Prospects and Change During Their First Year in Computing Education
Abstract: Employability can be defined as a part of one’s identity formation, or pre-professional identity formation. In this paper, we have interviewed six computing students at the beginning of their first semester and in the middle of their second semester, exploring their perspectives on being a student and future professional. The results show that students have started their identity and employability development by aligning themselves with the practices in the computing discipline, engaging in a study program community, and imagine their future professional identity. The article gives recommendations on how Wenger-Trayner & Wenger-Trayner’s concept of modes of identification can be used to enhance employability in first-year computing education.
Should Explanations of Program Code Use Audio, Text, or Both? A Replication Study
Abstract: Studies in educational psychology suggest that people learn better when visual learning materials are accompanied by audio explanations rather than textual ones. Research on how this modality principle applies to computing education is scarce and inconclusive. We explore whether the modality of instruction affects learning from videos that use a sequence of example programs to explain how variables work in Python. Learners (n=186) were crowd-sourced from the internet and randomly divided into three groups, who received explanations as audio, text, or both, respectively. We did not find significant differences between the groups in near transfer to code-tracing tasks or perceived cognitive load. The result affirms the need to further investigate instructional modalities in programming education. There are a number of theoretical, methodological, and instructional-design factors that might explain these and earlier findings; we trace out future research that could explore those factors.
Adapting Student IDEs for Blind Programmers
Abstract: What does it take to adapt a programming environment so visually-impaired programmers can comfortably use it? Every aspect of the environment needs attention, from toolbars to editors to interactive components. We describe the steps we had to take to adapt WeScheme, the environment used by Bootstrap:Algebra. We also summarize the experience of a group of blind students using the result, and present some lessons for other curricula to consider. A particular challenge in Bootstrap:Algebra is its heavy reliance on images, which many other media-rich curricula also use. Visual computing is, almost by definition, inaccessible to students with visual impairments. This poses curricular, legal, and moral obstacles for computing educators who want to use these curricula.
Combining Ideas and Artifacts: an Interaction-Focused View on Computing Education Using a Cybersecurity Example
Abstract: In this paper, we present a novel approach to design teaching interventions for computing education, elaborated using an example of cybersecurity education. Cybersecurity education, similar to other computing education domains, often focuses on one aspect and separate themselves from the other approach. In other words, they often focus on one of the two different aspects, which are teaching a) either how to use and to behave, or b) how technology works. Here we suggest another point of focal awareness for teaching — interaction — that allows the recombination of both approaches in a novel way, leading to a reconstruction of the teaching and learning content in a way that — as we hope — supports an understanding on a higher level and thus gives the chance to better develop agency. For this didactic reconstruction of teaching content, we use an approach called the hybrid interaction system framework. In cybersecurity training, teaching interventions oftentimes are in a way successful but seem to not lead to long-lasting changes towards secure behavior. Using simply password security as an example, we show how this new approach recombines the two different priory mentioned teaching approaches in a novel way.Within this short paper, we present our current research progress, discuss potentials and values of the approach in general, and by example. Our intention of this submission and early disclosure is to spark discussion and generate further insights especially regarding the following question: What implications does the hybrid interaction system approach have on learning scenarios?
Complex Online Material Development in CS Courses
Abstract: Online materials have become a ubiquitous part of CS education, whether a given course is realized as fully online or uses online materials as a part of the syllabus. The course platforms are inherently tied to teaching — limiting pedagogical approaches, types of exercises, and available functionality. Using service oriented architectures and protocols enables interoperability between systems allowing the use of multiple platforms and systems for different needs of the course. However, this increases the complexity of the material creation greatly. Complex online learning materials with multiple systems have become intricate software projects — with their inherent problems. We present two small studies investigating this issue. (1) A case study of a fully online introductory programming course developed and ran in spring 2020 that utilized many different systems. We highlight multiple pressing issues stemming from the complexity of the material. (2) We conducted a focused interview study, which sought to collect CS instructor experiences from various Finnish universities regarding online platforms and material development. Based on these two studies we argue that online courses should be developed with a stronger software engineering approach considering the development process and tools, in addition to learners’ perspective on the material. We highlight how the current approaches are insufficient and hope to start a conversation with computing education instructors and researches alike.
Lessons from Teaching HCI for a Diverse Student Population
Abstract: HCI education has been an active area in the past two decades. Many researchers have reported on challenges unique to teaching HCI and investigated different pedagogical approaches to improving the students’ learning experience. We add to this body of research by describing our experience in teaching HCI as a mandatory undergraduate course, but with the added complexity of having non-majors and graduate students. We describe three pedagogical approaches to teaching this course and summarize the lessons learned. We hope that our discussion can help shape future HCI course developments.
Computational Thinking Interventions in Higher Education
Abstract: Computational Thinking is seen as a crucial skill in an increasingly digital society. Researchers and educators in higher education therefore aim to improve the Computational Thinking (CT) skills of students using appropriate interventions. However, there is currently no overview of interventions used to teach CT and how effective they are. With this scoping literature review, we provide such an overview by identifying articles that discuss interventions used to teach CT in higher education. We identify the teaching approaches used in these interventions, and discuss their effectiveness and how this is assessed. Furthermore, we look at the use of adaptive interventions. Our search of three academic databases (Scopus, ACM and ERIC) resulted in 1839 articles. After screening, 49 articles remained.A detailed examination of the interventions discussed in these articles showed that CT is still often taught through programming assignments. The interventions are evaluated in a myriad of ways, making it difficult to compare the effectiveness of interventions. We therefore suggest making use of more standardized instruments to evaluate the effectiveness. Finally, although scaffolding is applied, interventions are not often adapted to the actual proficiency level of a student.
Infusing Computing: A Scaffolding and Teacher Accessibility Analysis of Computing Lessons Designed by Novices
Abstract: Creators of computing curricula do not always have formal pedagogical training. We investigated if exposing novice lesson designers to pedagogical best practices would result in the creation of lessons where evidence of successful use of these practices could be identified. We trained 29 high school students who were in a full-time computer science summer internship on how to create \snap programming lessons for non-computing courses. Over the course of three weeks they developed computing-infused lessons on their choice of learning topic (science, business, language, etc.). We examined these lessons for their use of scaffolding, teacher accessibility, equity, and content. We found that students implemented many of the scaffolding techniques that they themselves experienced and created lessons that were detailed enough to be accessible for teacher use. We also identified significant relationships between both subject area and gender on equity scores, as well as an impact of collaboration on scaffolding type included. No difference in artifact quality was identified by prior student coding experience. This project represents an innovative way to engage students in learning more computer science while creating educational materials for computing in K-12 classrooms.
A Proposal to Use Gamification Systematically to Nudge Students Toward Productive Behaviors
Abstract: While gamification in education has been used to increase motivation and engagement, it can also be used to nudge students to change behavior. This paper describes work in progress on a holistic gamification design for an introductory computer programming course that encourages students to adopt five core self-regulation skills: time management, incremental development, self-checking, persistence, and planning. We encourage growth mindset beliefs by reinforcing that skills can be increased and improved through targeted effort and practice. The self-regulation skills chosen are all associated with successful scholastic performance and present a model of “working smarter, not harder”. They relate to the student’s process of working, rather than the assignment solution being produced. A combination of role-playing game character development, experience points, leveling, daily missions, energy management, and a booster-based reward system are used as game mechanics. The goal is to communicate to students that mastery of these skills is valued by the course’s community, to encourage students to aspire to master these skills, and to recognize and reward students as they progress and eventually achieve these goals. The design leverages existing work on measuring student effort and progress as they work on assignments to drive game mechanics.
Learning to program hands-on: a controlled study
Abstract: Knowledge of computer programming is increasingly important in society. Many countries have introduced programming into the school curriculum.Programming has however been reported to be difficult to learn and it remains to be understood in more detail how different factors affect novice student’s learning of computer programming.In a controlled study with upper secondary school students (n=53) learning basic Java programming for three hours, we studied how factors like learning outcome, engagement, motivation, stress, and long-term memory are affected by hands-on and hands-off learning respectively. Students worked in pairs to solve programming problems. In each pair, one student was randomly selected to write the code hands-on, while the other student contributed hands-off. The roles did not switch during the session.We used tests and questionnaires to assess the learning outcome and some other aspects of relevance for learning. Statistical analysis of the results showed that working hands-on reduced stress. There was no difference in knowledge gain immediately after the teaching, but the hands-on group did slightly better on a follow-up test one week later.The results are discussed in relation to research about, e.g., stress, long-term learning and novices learning to program.
FIRST Principles to Design for Online, Synchronous High School CS Teacher Training and Curriculum Co-Design
Abstract: The global Covid-19 pandemic has offered new challenges andopportunities for teaching and research. It has forced constraintson in-person gathering of researchers, teachers, and students, and conversely, has also opened doors to creative instructional design. This paper describes a novel approach to designing an online, synchronous teacher professional development (PD) and curriculum co-design experience. It shares our work in bringing together seven high school teachers from North Carolina (NC), Tennessee, and Massachusetts, along with researchers in NC, Tennessee, and California. The teachers participated in a three-week summer PD to train them on ideas of Distributed Computing and how to teach this advanced topic to high school students using NetsBlox, an extension of the Snap! block-based programming environment.The goal of the PD was to prepare teachers to engage in collaborative co-design of a ~9-week curricular module that they (and eventually others) can then use in their classrooms and schools.Between their own training (Week 1) and the co-design process(Week 3), teachers co-taught 3 groups of 10 high school students(each) enrolled in a remote summer internship at [XYZ] to pilot the learned units and leverage ideas from their teaching experience for subsequent curricular co-design. Formative and summative feedback from teachers suggest that this PD model was successful in meeting desired outcomes. Our generalizable FIRST principles—Flexibility, Innovativeness, Responsiveness (and Respect), Supports, and Teamwork (collaboration)—that helped make this unique PD successful, can help guide future CS teacher PD designs.
Will Students Write Tests Early Without Coercion?
Abstract: Students faced with a programming task often begin their implementation without a sufficient understanding of the problem. Numerous prior papers suggest that formulating input–output examples before beginning one’s implementation is the key to averting problem misunderstandings, but that students are loathe to actually do it. Is outright coercion instructors’ only hope to convince students to follow this methodology and hence help themselves?We conjecture that students’ reluctance may stem from the disaffordances of their programming environments. In this work, we augment the student’s programming environment to encourage examples-first development, and design a novel measure to assess students’ adherence to this methodology. We apply these measures to students using our modified environment in a semester-long accelerated introductory computer science course, and find high voluntary adherence, especially relative to the literature’s low expectations.
The Value of Aligning Your Course for Curricular Improvement
Abstract: CS research has shown that many CS educators do not use an evidence-based approach to make changes to their course. Fossati and Guzdial have found that educators use their intuition, student feedback and anecdotal evidence to make course improvements. Using these approaches makes it difficult to identify the effectiveness of the improvement. In instructional design, curriculum alignment is used to evaluate the degree which the learning objectives, course content, and assessment methods are in agreement with each other. This provides instructors with a detailed view of their course and areas that need improvement. Current alignment processes are impractical for a course instructor to apply, requiring a panel of experts and training on the process. Elsherbiny and Cennamo developed a new alignment process applicable by a single course instructor limiting the need for a panel of experts, however their process took 75 hours to apply. In this experience report, we describe a prototype alignment tool, AlignET, that automates the new alignment process providing instructors with results they can use to make course improvements. We conducted a case study, where four instructors at university X used AlignET to evaluate the alignment of their course and identified specific improvements to make to their course. AlignET reduced the time to complete the alignment process to less than 11 hours. The instructors identified gaps in their instructional methods and identified learning objectives they emphasized more than others. The case study provided us with critical improvements to make to AlignET.
Crowdsourcing in Computing Education Research: Case Amazon MTurk
Abstract: Crowdsourcing platforms such as Amazon MTurk provide access to a human workforce that can be given tasks to complete online for a fee. In this article, we review studies in computing education research~(CER) that rely on crowdsourcing; we also describe our own experiences of using Amazon MTurk for a CER study. We discuss challenges in recruiting workers with specific backgrounds—such as no programming experience—and considerations in filtering out unreliable research participants. Combining recommendations from the literature with the lessons that we learned whilst conducting our study, we synthesize advice for researchers in CER who are considering crowdsourcing. In our case study, we did not find widespread foul play by crowdsourced workers and, overall, our experiences and the literature suggest that crowdsourced CER is feasible. It is, however, uncertain to what extent crowdsourced data can produce answers that apply to specific educational contexts. More research is needed before definitive conclusions can be drawn about the validity and generalizability of crowdsourced CER.
Escape Room Game for CT Learning Activities in the Primary School
Abstract: This systems short paper presents the design and implementation of an escape room game for practicing computational thinking (CT) skills in the primary school. The game, which can be packed into two small suitcases and set up anywhere, combines physical and virtual elements in order to engage learners into exploring the room. Aural elements with spoken tracks and background soundscape are combined with smart lighting to intensify the immersion. The game’s CT puzzles are modified from Bebras Challenge, and escaping the room requires solving all of them. This paper describes the game disposition and design, the room setup, design and implementation of computational thinking minigames, and the system architecture.
Students’ Preferences Between Traditional and Video Lectures: Profiles and Study Success
Abstract: Videos play a key role in modern education. Although the impact and design of video material are relatively well understood, individual differences and patterns, especially in the context of computing education, in video utilization are not. In this work, we study how students attend live lectures and watch video lectures in a data structures and algorithms course. Students had the option to attend the live lectures, watch the lectures as videos, or both. We cluster data related to live lecture attendance and video watching collected over multiple years and find three distinct profiles: 1) active students who prefer lectures, 2) active students who prefer videos and 3) those who provide only limited attention toward online or video lectures. Whereas previous research suggests that watching video lectures is related to at least equal performance when compared to attending live lectures, we found that (in our context) students attending live lectures outperformed all the other groups. This may be related to a selection bias in our setup, however. Thus, the use of video lectures as a part of versatile study material in computing education needs more attention in the future.
Posters
Poster: Towards Understanding the HCI Education Landscape
Abstract: Recent efforts have been made to realize the concept of the “HCI living curriculum”. To better understand the current landscape of our pedagogies, we implemented an online tool to gather and analyze educational patterns in HCI courses. Our goal is to uncover the pedagogical choices made in our HCI courses by analyzing syllabi information. Specifically, we wish to gain a better understanding of topic coverage, theories taught, and techniques practiced in the classroom. Furthermore, our tool enables regional comparisons of the data in the system. As a first step, we sampled a collection of syllabi from North American universities and present analyses from them.
Poster: GitCanary: A Tool for Analyzing Student Contributions in Group Programming Assignments
Abstract: Courses in computer science curricula often involve group programming assignments. In this paper, we present the GitCanary tool for monitoring project progress and member contributions using development productivity metrics. The tool is developed to support teachers and students with an overview that they can use during their guidance sessions. Feedback on the tool and its metrics was collected from students and teachers using interviews and questionnaires after utilizing it in a software development course. The results that the tool provides were found to be valuable in project guidance and to promote transparency and balance in work distribution.
Poster: Inferring Students’ Tracing Behaviors from Interaction Logs of a Learning Environment for Software Design Comprehension
Abstract: VeriSIM is a learning environment which helps software engineering students comprehend a given design, by enabling them to trace different scenarios in the design. Learners trace different scenarios by constructing a state diagram. In VeriSIM, learners go through four challenges which help them progressively trace scenarios in the design. VeriSIM provides learners affordances to add, edit, delete states, as well as “run” a given state diagram. All such learner interactions are logged in the system. In this paper, we analyze the interaction logs of 12 students, in order to infer certain behavior patterns. The preliminary analysis of these patterns shows that the order of the challenges help learners effectively trace scenarios.
Poster: An online tool for analyzing written student feedback
Abstract: Collecting student feedback is commonplace in universities. Feedback surveys usually have both open-ended questions and Likert-type questions, but the answers to open questions tend not to be analysed further than simply reading them. This paper presents a tool for analyzing written student feedback using topic modeling and emotion analysis. We demonstrate the utility of this tool using course survey responses from a software engineering (SE) programme.
Poster: What Competencies Do Student Teachers Have? – Developing a Test Instrument to Assess Algorithm-Related Competencies
Abstract: During the last years there have been intense discussions about how to include the acquisition of digital competencies in school education. The implementation is highly dependent on digitally-skilled educators. Currently, learning offers for a basic computing education of all teachers are needed and developed in many places. However, not much is known about the computing competencies and conceptions of student and in-service teachers, who would take such courses. Also, there is a need of a proper test instrument measuring computing competencies that (student) teachers might possess not necessarily due to formal computing education but due to their everyday use of computing systems. Therefore, this project aims to create such a test instrument which measures competencies in the field of algorithm without testing explicit computing knowledge.
Poster: Presenting Basic CS Concepts: A Content Analysis of AP CSA Textbooks
Abstract: There is extensive support for the use of examples to promote learning. In the field of computer science education, code examples are common, and research has focused on the efficacy of worked examples. Less attention has been paid to other aspects of presenting code examples in instructional materials, such as their type, characteristics, and order. This poster seeks to address some of those issues through a content analysis of the presentation of basic computer science concepts in CS textbooks. The study found that the most common presentation style was example code without comments. Further, authors were not consistent in their choice of code snippet styles across lessons. Further research is needed to probe the efficacy of these instructional design choices.
Poster: Detecting Students’ Affective States in Industry-focused Projects
Abstract: In industry-focused projects, students feel overwhelmed by the fast-paced nature of the group projects and the involvement of actual industry clients. To be able to provide timely feedback and support in challenging learning situations, we have experimented with the use of an emotion detection tool, in our case the IBM Watson™ Tone Analyzer, to automatically recognize the students’ emotional states and the potential issues they might have during their projects. Preliminary results show that the objectively recognised emotional tones moderately to strongly correlate with students’ issues as observed by the academic coordinators and also with students’ final marks for the project work.