Understanding Student Interaction with AI-Powered Next-Step Hints: Strategies and Challenges

Anastasiia Birillo, Aleksei Rostovskii, Yaroslav Golubev and Hieke Keuning

February, 2026. Accepted to SIGCSE'26 (A).

Abstract. Automated feedback generation plays a crucial role in enhancing personalized learning experiences in computer science education. Among different types of feedback, next-step hint feedback is particularly important, as it provides students with actionable steps to progress towards solving programming tasks. This study investigates how students interact with an AI-driven next-step hint system in an in-IDE learning environment. We gathered and analyzed a dataset from 34 students solving Kotlin tasks, containing over 6 million lines of code and detailed hint interaction logs. We applied process mining techniques and identified 16 common interaction scenarios, along with analyzing transitions between specific actions. Semi-structured interviews with 6 students revealed strategies for managing unhelpful hints, such as adapting partial hints or modifying code to generate multiple variations of the same hint. These findings, combined with our publicly available dataset, offer valuable opportunities for future research and provide key insights into student behavior with hint systems, helping to improve hint design for enhanced learning support.