Assessing Computational Thinking: Development and Validation of the
Algorithmic Thinking Test for Adults
Abstract
This study describes the development and validation process of a
computational thinking (CT) test for adults. The team designed a set of
items and explored a subset of those through a couple of qualitative
pilots. Then, in order to provide validity evidence based on the test
content, a team of 11 subject-matter experts coded the initial pool of
items using two different systems of categories based on CT components
and contents. Then the items were piloted on a sample of 289
participants, 137 experts in CT and 152 novices. After a series of
confirmatory factor analyses, a unidimensional model that represents
algorithmic thinking was adopted. After analyzing the psychometric
quality of the 27 items, 20 of them with excellent reliability indices
were finally selected for the test. Thus, this study provides a tool to
evaluate adults’ CT: the Algorithmic Thinking Test for Adults (ATTA),
which was developed according to psychometric standards. This article
also reflects on the nature of CT as a construct.