We invite you to submit a paper or poster for the 23rd Koli Calling International Conference on Computing Education Research (Koli Calling 2023) to be hosted both online, 13-14 November 2023 and in Koli, Finland, 16-19 November 2023.

Koli Calling is one of the leading international conferences dedicated to the scholarship of teaching and learning and to education research in the computing disciplines. It is a single-track conference for original and novel work with research, practice and systems presentations as well as a keynote and invited talks. The conference is known for its moderate size, intimate atmosphere, and lively discussions. 

As in previous years, there will be a doctoral consortium before the main conference. See below for more details.

New for this year: The conference will consist both of an online part and a separate face-to-face part. The online part will offer room for presenting and discussing all accepted research papers and connect virtually with the community. The face-to-face part will focus specifically on offering room for sharing and discussing ideas in the welcoming atmosphere that Koli Calling is renowned for.

All accepted full papers will be presented in the online part. All accepted discussion papers, demos and posters – and a selection of full papers that have authors attending Koli – will be presented in the face-to-face meeting. In this way, we hope to maximize both the opportunity for researchers to take part in Koli without having to travel and the open and communicative atmosphere that Koli has been known for. So in short:

• All accepted full papers must be presented in the online part (November 13 – 14)
• All accepted discussion papers, demos and posters must be presented in the face-to-face part (November 16 – 19)
• Selected full papers will be presented in the face-to-face part, if authors are attending Koli.

We welcome submissions in the area of computing education: the teaching and learning of computing topics. Within that broad scope, contributions may involve one or more of the following:

• Empirical studies of computing education;
• Teaching approaches and assessment in computing education;
• Development, use, and evaluation of tools to support computing education;
• Theory in computing education;
• The learning of computing in any context; e.g., higher education, K-12, informal learning, lifelong learning, teachers’ professional development;
• Distance, online, and blended learning of computing topics;
• Learning analytics and educational data mining in the area of computing education;
• Methodological development and instruments for computing education research;
• Reviews to support computing education or computing education research;
• Any other aspects of the teaching and/or learning computing topics. 

Please note that generic educational technology and applications of computing for educational purposes are not in scope, except where they are used for computing education.

Submission Categories

Koli Calling 2023 will accept submissions of three types, in various categories. Each category of paper will be reviewed using a reviewer template specific to and appropriate for the submission category by reviewers interested in papers of that type. 

Full papers with up to 16 pages single-column excluding references and, optionally, supplemental material in either of the two categories:

1. Research papers present high-quality research. These submissions will be evaluated on the quality of the contribution, its contextualization in the literature, and sound application of methods appropriate to the topic of investigation. Most research papers can be described as empirical, theoretical, or review papers.

• Empirical papers report on studies that collect and analyze empirical data and employ methods rigorously, whether qualitative, quantitative, or mixed.
• Theoretical papers formulate, extend, apply, critically evaluate, or otherwise discuss theory for the purpose of improving computing education or computing education research. The discussion is grounded in the research literature and advances the field.
• Review papers survey the literature on a particular topic.

2. System and tool papers present a software system or other tool of value in a computing education setting. They will be evaluated based on the potential contribution of the system to the community, the effectiveness of the comparison to and differentiation from existing similar systems, and the quality of the evaluation and/or discussion of the design of the tool. We expect contributions in this category to present tools that have a certain degree of maturity and matching empirical evaluation! Papers featuring a system or tool can also be submitted to the research paper category, if the evaluation is the emphasis. This category focuses on the contribution of the system/tool, as supported by evidence of its unique contribution (differentiation from other tools), efficacy, and/or quality of design.

Discussion papers with up to 10 pages single-column excluding references and, optionally, supplemental material disseminate and discuss new ideas and promising initiatives in computing education practice or research. They will be reviewed based on the quality of the discussion, their use of appropriate evidence or argumentation, and their potential to be of interest to the community. Examples of discussion papers include:

• Short empirical papers, which are limited in scope or produce tentative results but are nevertheless of interest to the computing education research community and suggest paths for future research. 
• Proposition papers, which propose an innovation, debate a theoretical issue, or advance a cause. The argumentation must be solidly grounded in the research literature. A discussion paper must provide fresh perspectives or insights and present a compelling case to indicate how it promotes discussion and raises questions for future research to answer.

Poster/demo papers with up to 2 pages single-column excluding references are presentations of emerging ideas for research, teaching practice, or tools. These presentations are typically interactive with the participants.

New this year: We (strongly) suggest that all empirical submissions adhere to one of the reporting standards of SIGSOFT (https://acmsigsoft.github.io/EmpiricalStandards/docs). Authors can indicate upon submission the category that they have chosen as most appropriate for their work and follow the respective checklist to ensure that their submission fulfils the required standard. The standards will also be suggested to reviewers as a basis for their decision. We will evaluate the appropriateness and usefulness of this approach in the context of our community afterwards.  

For all accepted works, the method of final presentation will be discussed closer to the time of the conference. We expect at least one author from each accepted paper will participate in the final conference to present their work and be available for discussion. As indicated above, a purely remote attendance is only possible for certain types of submissions. 

All papers will be evaluated prior to review to ensure that they are clearly in the right track, have been correctly anonymised, and are ready to be reviewed by the program committee. Each paper that goes to review receives at least three reviews, with senior PC moderation of the review process.

New this year: If during the reviewing and discussion period there arises a need for clarification of certain aspects of a submission, the chairs can send a request of clarification to the authors who will then have 72 hours to respond to the questions. The answers of the authors will be taken into account during the discussion and decision making.

Papers may be accepted, conditionally accepted, or rejected. Papers in the second category will receive an opportunity to make requested revisions or to explain why revisions are not needed prior to undergoing final review by a shepherd, and will be moved to the accepted or rejected categories based on the shepherd’s review.

Doctoral Consortium

The Koli Calling Doctoral Consortium will be organized before the conference, November 15-16 as an in-person event at Koli. See a separate Call for Applications for more information.

Important Dates

• All deadlines are Anywhere on Earth (AoE), which is UTC-12.
• Submission deadline (full papers and discussion papers): Midnight Friday, June 23
• Extended submission deadline (see below): Midnight Friday, June 30
• Notification of acceptance (full papers and discussion paper): September 11
• Re-submission deadline for conditionally accepted papers: September 24
• Submission deadline (posters and demo papers): September 24
• Notification of acceptance (conditionally accepted papers, posters, and demo papers): October 6
• Final submission of revised manuscripts (all categories): October 13
• Registration opens: September 11
• Early Bird and Author registration deadline (at least one author from every accepted paper): October 1
• Conference registration deadline: October 27 for in-person participation
• Conference: Thursday, November 16 to Sunday, November 19

Extended submission deadline: For full papers and discussion papers we offer a re-submission slack of exactly one week. If a paper is submitted by the first deadline, it will be possible to submit updated versions of the paper until Midnight June 30. Papers that are not first submitted by June 23, or that are not reasonably complete at that time, will not be considered.

Special Author Notes

1. The official publication date is the date the proceedings are made available. This date is expected to be one week prior to the first day of the conference. The official publication date affects the deadline for any patent filings related to published work.
2. The Finnish Ministry of Education and Culture has released a policy recommending a freeze on cooperation with Russian institutions. The sponsoring university has forbidden creation of new research collaborations with Russian and Belarusian scholars. As a result, if any author only has status at a Russian or Belarusian institution, we are unable to welcome you to Koli this year.

Paper Formatting and Length Requirements

For full information on formatting and Koli policies, see the submission guide.

We look forward to welcoming you to Koli!

Organized by

Leibniz Institute for Science and Mathematics Education / Kiel University

University of Eastern Finland