Programming By Scaffolded Demonstration with Perpend

Paper Title

Programming By Scaffolded Demonstration with Perpend

Publication Info

• Topic area: Output-centric programming for exploratory and creative coding.
• Keywords: Programming by Scaffolded Demonstration, Perpend, creative coding, p5.js, output-centric programming, exploratory programming, programming languages as materials, user interfaces, program modification, visual programming.

Background and Problem

• Problem / challenge: Existing output-centric programming paradigms (e.g., Direct Manipulation, Programming By Example) assume users have a target output in mind, which limits their applicability in exploratory or creative contexts where users aim to discover outputs through programming.
• Significance: Supporting exploratory programming is essential for domains like creative coding and data science, where the process of programming is intertwined with discovering the intended output.
• Motivation and related work: Prior work in output-centric programming focuses on aligning programs with user-provided outputs. However, this approach does not address situations where users lack a predefined output. This paper builds on concepts of programming as a material, inspired by Tim Ingold’s ideas, to propose a new interaction paradigm.

Solution

• Proposed approach: Programming By Scaffolded Demonstration (PBSD), a programming paradigm that scaffolds user exploration by displaying a range of small, valid program modifications and their corresponding outputs.
• Novelty:
  1. Introduction of PBSD as a new output-centric programming approach that does not require user-provided outputs.
  2. Development of Perpend, a programming environment for p5.js that implements PBSD by presenting galleries of code edits paired with output previews.
  3. Qualitative analysis of Perpend juxtaposed with a Direct Manipulation editor, highlighting how PBSD supports programming as a manual and exploratory practice.
• Procedure and key techniques:
  1. Users interact with Perpend by clicking on locations in their program text.
  2. Perpend generates and displays a gallery of small program modifications (patches) and their visual outputs.
  3. Users select modifications to apply, which updates the program and output iteratively.
  4. The system ensures modifications are syntactically small and visually meaningful, leveraging the structure of the programming language.

Results

• Concrete findings:
  • Perpend helped users explore new programming possibilities, expand their creative vocabulary, and maintain a sense of program continuity.
  • Users leveraged Perpend to discover unexpected directions, bound exploration to specific program regions, and regain creative momentum when stuck.
  • Participants used Perpend’s modifications to learn new programming constructs and integrate them into their practice.
• Advantage over baselines:
  • Unlike Direct Manipulation systems, Perpend supports exploratory programming by visualizing a space of possible program modifications rather than requiring predefined outputs.
  • Perpend enables users to work flexibly between visual-to-program and program-to-visual styles, fostering deeper engagement with both program text and output.
• Experiments / evaluation:
  • Nine experienced p5.js artists participated in a user study, using Perpend and a Direct Manipulation editor for ~45 minutes each, followed by ~30-minute interviews.
  • Participants explored self-directed creative tasks, and their interactions were analyzed using reflexive thematic analysis.
• Limitations and future work:
  • Limited study duration and single-session design constrained observations of long-term tool use.
  • Future work could explore PBSD applications in other domains (e.g., data science, web design) and support for novice programmers.
  • Enhancements like user-defined rewrite rules and bespoke output visualizations could improve PBSD systems.

Summary

This paper introduces Programming By Scaffolded Demonstration (PBSD) and its implementation in Perpend, a programming environment for p5.js. PBSD supports exploratory programming by displaying small, valid program modifications and their outputs, enabling users to navigate the possibilities of their programming language as a material. A user study with nine artists demonstrated that Perpend fosters creative exploration, helps users expand their programming vocabulary, and supports both visual-to-program and program-to-visual workflows. While designed for creative coding, PBSD has potential applications in other domains where programming is exploratory. Future work could refine PBSD systems for broader use cases and longer-term engagement.