Poster for LitterBox+: An Extensible Framework for LLM-enhanced Scratch Static Code Analysis

AnExtensibleFrameworkforllm-enhanced ScratchStaticCodeAnalysis Demo https://scratch.fim.uni-passau.de/scratch/ �https://github.com/se2p/LitterBox �https://arxiv.org/abs/2509.12021 Benedikt Fein, Florian Obermüller, Gordon Fraser ASE 2025: Tool Demonstration Track Command Line Interface All features are available via the cli, for example to allow for batch processing of student submissions: java -Dlitterbox.llm.openai.api-key=KEY -jar LitterBox.jar llm fix ^--path=submissions/ ^--target=Boat Via the cli users can also access additional features not yet integrated into the ui: • request the llm to find both new issues (smells) and comment on good code (perfumes), and • ask the llm to fill in missing parts of the code. Problem and Idea The problem: • Programming learners require feedback on their code • False positives are unacceptable in an educational context •llms underperform with visual Scratch code •llms cannot produce valid Scratch code The solution: • Use llms to explain issues found by static analysis (LitterBox) •LitterBox+converts Scratch code to textual scratchblocks for prompting •LitterBox+converts llm responses back to Scratch code • Enables conversations about code issues and programs LitterBox: Generic static code analysis feedback LitterBox + : Extended with program-specific feedback generated by the llm User Interface The Scratch gui is extended with a code analysis interface: • Lists code issues found by LitterBox • Users can ask for explanations • Users can ask for fix suggestions • Users can ask their own questions about the code Scratch Program Representation LitterBox + prompts the llm to explain the observable behaviour, the underlying programming concept the student might have misunderstood, and steps the student can follow to fix their specific issue. When requesting extended feedback from the llm for a LitterBox static code analysis warning, we add both the original code and the generic issue description to the prompt. LitterBox converts the visual code from the Scratch-internal json representation into scratchblocks syntax as used on the public Scratch community forums: when clicked move -1 steps if touching color then ^// Script: "internal script-ID" when green flag clicked if <touching color (#5ce581)> then move (-1) steps end In its response, the llm should again use the scratchblocks syntax. The scratchblocks elements are rendered as inline images in the llm explanation. However, for the ‘GPT: Fix!’ feature the fixed script needs be integrated directly into the current program and loaded into the editor so that the student can continue to make changes. To enable this, our newly developed scratchblocks-parser converts code blocks in the response into the ast representation of LitterBox, which in turn is mapped to the native representation of Scratch, which then can be loaded as editable code. The Scratch-internal json is not directly suitable for llms since it contains abstract opcodes and contains ‘next’-pointers to the following blocks, which may however be stored in far-away locations within the json and need to be updated consistently when making changes. System Design L itter B ox I ssues R equest H int R equest F ix A sk Q uestion S tatic C ode A nalysis LLM P rompt D esigns S cratch B locks P arser REST API O llama langchain LitterBox + is implemented as two main components: The user interface integration and a backend running the static code analysis and the communication with the llms. By using generic interfaces it is designed to be configurable and extensible with • custom prompt designs (e.g., to support other natural languages), • alternative llm providers, and • new llm-based features making use of the static code analysis and scratchblocks parsing infrastructure. Besides the integration in the Scratch gui and the cli,LitterBox + can also be used as a library providing a Java api to support the integration with other tools. 1