Abstract

The purpose of this paper is to examine how teams in the RoboCup Standard Platform League (SPL) approach software quality assurance and performance measurement. RoboCup serves as an experimental platform for students to delve into robotics research. However, with the ambitious goal set by RoboCup, teams must continuously enhance their skills and competitiveness. Our study adopts a qualitative descriptive approach, involving interviews with team leaders and designated team members from seven different RoboCup Standard Platform League teams. During these interviews, we recorded field notes and subsequently conducted qualitative content analysis on the gathered data. The findings indicate that nearly all teams rely on expert judgment during test games conducted in their laboratory settings for quality assurance and performance assessment, including the most successful teams in the league. However, only a minority of teams have implemented more structured and automated mechanisms akin to those used in software engineering. Furthermore, almost none of the teams engage in performance measurement beyond manual assessment through observation. We explore potential explanations for these findings and conclude that there is ample room for improvement towards implementing more structured and automated approaches to quality assurance and performance measurement, both at the team level and within the league itself. These results hold practical significance for shaping the roadmap of RoboCup and offer valuable insights for establishing team-based quality assurance and performance measurement practices.

In case of any questions please contact me using the contact information provided in the paper.