Study of the use of property probes in an educational setting
Risberg Alaküla, Anton LU
; Fors, Niklas
LU
and Söderberg, Emma
LU
(2025)
In The Art, Science, and Engineering of Programming
10(1).
- Abstract
- Context Developing compilers and static analysis tools (“language tools”) is a difficult and time-consuming task. We have previously presented property probes, a technique to help the language tool developer build understanding of their tool. A probe presents a live view into the internals of the compiler, enabling the developer to see all the intermediate steps of a compilation or analysis rather than just the final output. This technique has been realized in a tool called CodeProber. Inquiry CodeProber has been in active use in both research and education for over two years, but its practical use has not been well studied. CodeProber combines liveness, AST exploration and presenting program analysis results on top of source code. While... (More)
- Context Developing compilers and static analysis tools (“language tools”) is a difficult and time-consuming task. We have previously presented property probes, a technique to help the language tool developer build understanding of their tool. A probe presents a live view into the internals of the compiler, enabling the developer to see all the intermediate steps of a compilation or analysis rather than just the final output. This technique has been realized in a tool called CodeProber. Inquiry CodeProber has been in active use in both research and education for over two years, but its practical use has not been well studied. CodeProber combines liveness, AST exploration and presenting program analysis results on top of source code. While there are other tools that specifically target language tool developers, we are not aware of any that has the same design as CodeProber, much less any such tool with an extensive user study. We therefore claim there is a lack of knowledge how property probes (and by extension CodeProber) are used in practice. Approach We present the results from a mixed-method study of use of CodeProber in an educational setting, with the goal to discover if and how property probes help, and how they compare to more traditional techniques such as test cases and print debugging. In the study, we analyzed data from 11 in-person interviews with students using CodeProber as part of a course on program analysis. We also analyzed CodeProber event logs from 24 students in the same course, and 51 anonymized survey responses across two courses where CodeProber was used. Knowledge Our findings show that the students find CodeProber to be useful, and they make continuous use of it during the course labs. We further find that the students in our study seem to partially or fully use CodeProber instead of other development tools and techniques, e.g. breakpoint/step-debugging, test cases and print debugging. Grounding Our claims are supported by three different data sources: 11 in-person interviews, log analysis from 24 students, and surveys with 51 responses. Importance We hope our findings inspire others to consider live exploration to help language tool developers build understanding of their tool. (Less)
Please use this url to cite or link to this publication:
https://lup.lub.lu.se/record/4a42678e-7567-42cc-a82e-517672d20faa
- author
- Risberg Alaküla, Anton
LU
; Fors, Niklas
LU
and Söderberg, Emma
LU
- organization
- publishing date
- 2025-02-15
- type
- Contribution to journal
- publication status
- published
- subject
- keywords
- User studies, Debugging, Property probes, CodeProber
- in
- The Art, Science, and Engineering of Programming
- volume
- 10
- issue
- 1
- article number
- 10
- pages
- 46 pages
- external identifiers
-
- scopus:86000514409
- ISSN
- 2473-7321
- DOI
- 10.22152/programming-journal.org/2025/10/10
- project
- Cloud Based Language Tooling
- language
- English
- LU publication?
- yes
- id
- 4a42678e-7567-42cc-a82e-517672d20faa
- date added to LUP
- 2025-06-18 10:54:37
- date last changed
- 2025-07-31 17:33:53
@article{4a42678e-7567-42cc-a82e-517672d20faa,
abstract = {{Context Developing compilers and static analysis tools (“language tools”) is a difficult and time-consuming task. We have previously presented property probes, a technique to help the language tool developer build understanding of their tool. A probe presents a live view into the internals of the compiler, enabling the developer to see all the intermediate steps of a compilation or analysis rather than just the final output. This technique has been realized in a tool called CodeProber. Inquiry CodeProber has been in active use in both research and education for over two years, but its practical use has not been well studied. CodeProber combines liveness, AST exploration and presenting program analysis results on top of source code. While there are other tools that specifically target language tool developers, we are not aware of any that has the same design as CodeProber, much less any such tool with an extensive user study. We therefore claim there is a lack of knowledge how property probes (and by extension CodeProber) are used in practice. Approach We present the results from a mixed-method study of use of CodeProber in an educational setting, with the goal to discover if and how property probes help, and how they compare to more traditional techniques such as test cases and print debugging. In the study, we analyzed data from 11 in-person interviews with students using CodeProber as part of a course on program analysis. We also analyzed CodeProber event logs from 24 students in the same course, and 51 anonymized survey responses across two courses where CodeProber was used. Knowledge Our findings show that the students find CodeProber to be useful, and they make continuous use of it during the course labs. We further find that the students in our study seem to partially or fully use CodeProber instead of other development tools and techniques, e.g. breakpoint/step-debugging, test cases and print debugging. Grounding Our claims are supported by three different data sources: 11 in-person interviews, log analysis from 24 students, and surveys with 51 responses. Importance We hope our findings inspire others to consider live exploration to help language tool developers build understanding of their tool.}},
author = {{Risberg Alaküla, Anton and Fors, Niklas and Söderberg, Emma}},
issn = {{2473-7321}},
keywords = {{User studies; Debugging; Property probes; CodeProber}},
language = {{eng}},
month = {{02}},
number = {{1}},
series = {{The Art, Science, and Engineering of Programming}},
title = {{Study of the use of property probes in an educational setting}},
url = {{http://dx.doi.org/10.22152/programming-journal.org/2025/10/10}},
doi = {{10.22152/programming-journal.org/2025/10/10}},
volume = {{10}},
year = {{2025}},
}