--- title: "Notational Programming for Notebook Environments: A Case Study with Quantum Circuits" authors: - Ian Arawjo - Anthony J. DeArmas - Michael Roberts - Shrutarshi Basu - Tapan Parikh abstract: "We articulate a vision for computer programming that includes pen-based computing, a paradigm we term notational programming. Notational programming blurs contexts: certain typewritten variables can be referenced in handwritten notation and vice-versa. To illustrate this paradigm, we developed an extension, Notate, to computational notebooks which allows users to open drawing canvases within lines of code. As a case study, we explore quantum programming and designed a notation, Qaw, that extends quantum circuit notation with abstraction features, such as variable-sized wire bundles and recursion. Results from a usability study with novices suggest that users find our core interaction of implicit cross-context references intuitive, but suggests further improvements to debugging infrastructure, interface design, and recognition rates. Throughout, we discuss questions raised by the notational paradigm, including a shift from ‘recognition’ of notations to ‘reconfiguration’ of practices and values around programming, and from ‘sketching’ to writing and drawing, or what we call ‘notating.’" summaryType: survey sourceStatus: null sources: - https://doi.org/10.1145/3526113.3545619 --- # Notational Programming for Notebook Environments: A Case Study with Quantum Circuits [Original paper PDF](https://doi.org/10.1145/3526113.3545619) ## Background and research problem The paper investigates how handwritten diagrams can participate directly in computer programs alongside typed code. Its starting point is the gap between diagrammatic reasoning and the keyboard-centered environments used to implement that reasoning. Quantum computing makes this gap concrete: a programmer may develop an algorithm using mathematical expressions and quantum circuit diagrams, translate it into an API such as Qiskit or Cirq, and inspect a generated circuit diagram to check the translation. The authors ask whether executable handwritten notation can remain part of the program instead of serving only as an intermediate representation that must be transcribed. This is a new combination of ideas within a longstanding problem of integrating visual and textual programming. The paper situates its approach among block-based and tangible programming, heterogeneous visual languages, computational notebooks, bidirectional interfaces, embedded graphical editors, and pen-based systems such as SILK, DENIM, and AlgoSketch. Earlier systems supported diagram recognition, conversion from sketches to code, or linked graphical and textual editing, but the authors identify a less explored combination: editable drawing canvases embedded within typed expressions, with implicit communication between handwritten names and the surrounding program scope. Their historical framing, illustrated by circuit equalities in Figure 2, also challenges the assumption that programming must inherently be typewritten. The quantum case study addresses both translation effort and abstraction. The circuit editors discussed in the 2022 paper allowed users to assemble concrete circuits, but offered fewer ways to express arbitrary-sized families of circuits and reuse abstract components. A useful handwritten interface therefore needs more than recognition of isolated gates: it needs a notation for subcircuits, bundles of qubit wires, and repeated or recursive structures. The authors present their system as an exploration of these possibilities and of programming practice, rather than establishing that handwriting is universally preferable to typed code or graphical editors. ## Notate and implicit cross-context references Notate is a Jupyter notebook extension that lets users insert drawing canvases inside lines of Python code. A canvas can be resized, edited in place, opened in a fullscreen drawing view, and copied or deleted alongside surrounding text. The fullscreen toolbar provides freehand drawing, geometric shapes, erasing, undo, redo, and deletion. Users can also paste an external image into a canvas; Figure 3 demonstrates this with a circuit image placed inside a function call and subsequently rendered through Qiskit. Figure 1 shows the actual notebook interface and the enlarged drawing mode, including the handwritten circuit and the interpreted circuit output below the cell. The authors define a notational programming system through three components: a host programming environment, a pen-based interface, and a communication protocol between them. In their implementation, Python sees a canvas as image data represented by a NumPy array together with metadata such as strokes, pressure, timestamps, and a snapshot of the local scope at execution. A domain-specific interpreter recognizes the drawing, parses its semantics, and applies a policy governing which host names it may read, modify, or declare. The canvas is consequently more than an illustration attached to code; it can be an argument that contributes executable meaning. The central interaction is an implicit cross-context reference. For example, a user can assign an interpreted subcircuit to a typed Python variable `A`, then draw a box labeled `A` inside another circuit without explicitly passing `A` into the interpreter as an additional argument. The general framework also permits an interpreter to introduce variables into the host scope, which goes beyond the behavior of an ordinary Python function and requires an explicit communication policy. Figures 4 and 5 use geometry examples to explain this broader vision: handwritten labels can refer to typed vectors, or an interpreted diagram can return a parameterized object whose unknowns are filled later. These examples explain the paradigm; the empirical case study concerns quantum circuit construction, particularly references from handwritten gates to typed circuit variables. ## Qaw's visual notation and abstraction mechanisms Qaw, short for quantum-draw, adapts familiar circuit notation for pen input. Horizontal wires represent qubits and are read from left to right; lettered boxes denote gates or named subcircuits; connected control marks specify controlled operations. Its design draws on circuits in papers, textbooks, tutorials, class notes, and programming examples, with particular attention to how practitioners depict abstraction. The authors iterated the notation through handwritten versions of quantum programming examples, balancing expressiveness, drawing effort, and recognition ambiguity. They omitted ellipses as a recognized construct and used recursive definitions to express some of the patterns that ellipses informally suggest. A slash across a wire represents a bundle of qubits. An unspecified bundle size can remain abstract until a circuit is instantiated with a particular number of inputs. A bundled input to a standard gate such as $H$ can spread that gate over the bundle, representing $H^{\otimes n}$ without requiring the user to write the tensor-product expression. Gate input and output sizes are inferred where the wiring is unambiguous, and splitters or rearrangements can separate an individual wire from the rest of a bundle. The interpreter checks mismatches and ambiguities instead of treating every drawing as valid. Recursion combines these bundles with named subcircuits. A typed circuit variable can appear as a handwritten gate inside its own definition, applied to fewer input wires. The notation defines an implicit identity base case, and zero-sized bundles disappear under rewrite rules. Figure 9 shows a participant's two recursive definitions, `A` and `C`, and the resulting five-input circuit. This output resembles the structure of the quantum Fourier transform's body, but the study task omits the QFT's parameterized gates and final swaps; it is not a complete implementation of that algorithm. The appendix further illustrates the progression from handwritten abstractions to expanded Qiskit circuit diagrams. The broader Qaw specification contains features beyond the evaluated subset. Figure 6 illustrates superdense coding with classical control variables, ket initialization, and stoppered output wires for measurement. Figure 7 illustrates Grover-style composition using a diffusion subcircuit, an oracle placeholder, wire bundles, and a repetition box. These examples show the intended notation and its relationship to conventional circuits and typed code. They must not be read as evidence that every depicted feature was supported in the study: initialization, measurement, repetition boxes, and uncomputation were not supported in that configuration, and an earlier implementation of classical-bit control was removed for the study. The evaluated bundle handling permitted at most one input bundle and no handwritten symbols above a slash; recursive definitions required typed left-hand-side names and did not support alternative base cases. ## Recognition and execution pipeline The implemented `qcr` interpreter handles a subset of Qaw using deep learning and classical computer vision. Figure 8 traces an actual participant's Bell-state drawing through the pipeline: a YOLOv4 multi-object detector, character classifiers, and computer vision identify symbols, while heuristic sketch recognition labels wires. Recognized objects and connections are converted into a directed multigraph. Type inference and semantic checks then validate the structure and convert it into either a Qiskit `QuantumCircuit` or an `AbstractQuantumCircuit` wrapper. The wrapper postpones concrete circuit generation until missing parameters, such as the number of input wires, are supplied. The final diagram in Figure 8 is Qiskit's rendering of the interpreted circuit, which gives the user a way to inspect what the drawing became. Recognition adds a distinct source of failure to normal syntax and semantic errors. When the recognizer detects a problem, it can display an annotated plot showing detected objects and labels. Figure 11 records a participant debugging a `Z` gate misread as `B` while also misunderstanding a handwritten size annotation. Figure 10 separately shows errors involving a missing output wire, an unsupported slash placement, and a compact controlled-gate depiction that the implementation did not accept. These examples explain why better handwriting recognition alone cannot solve the entire debugging problem: users also need to distinguish a valid drawing that was misread from a drawing that violates the notation's rules. ## Study design and findings The evaluation used a between-group design with 24 participants drawn from a common pool, with 12 assigned to Notate and 12 to a typed Qiskit condition. Participants had Python and notebook experience, lacked prior quantum programming knowledge, and were comfortable drawing by hand. After a tutorial, they completed six increasingly demanding circuit-construction tasks within sessions capped at two hours. The tasks covered a Bell-state circuit, a larger concrete circuit, an abstract diffusion circuit, subcircuit reuse, recursion, and a final circuit combining these concepts. Data included screen and audio recordings, notebook interaction logs, observations, a five-item Likert survey, and interviews. This was a study of notation use and circuit construction, not a controlled recognition-accuracy benchmark or a test of quantum algorithm understanding. Participants could receive researcher hints. For Notate, a researcher could accept a correct final drawing despite a recognition failure and let the participant advance. All 12 Notate participants completed the first five tasks, while nine completed the final task within the available time. All Qiskit participants completed all tasks, although one needed substantial researcher assistance on the last task. Most Notate participants readily understood that a handwritten gate label could refer to a typed circuit variable. Three even thought inline drawing canvases might already be a standard Jupyter feature. Recursion and the details of the notation were harder: some participants misunderstood a circuit's self-reference, placed slashes incorrectly, or created definitions that did not terminate. Recognition failures also differed substantially across participants and sometimes prompted them to abandon freehand drawing in favor of line and rectangle tools. The Notate survey medians were 4 out of 5 for ease of use, enjoyment, and confidence, but 3 for ease of correcting errors. Participants often preferred fullscreen drawing, used scratch paper on difficult tasks, and requested a lasso tool for moving or copying parts of a drawing. The timing comparison gives a task-dependent result. For Task 3, which introduced wire bundles and required a multi-controlled gate, Notate's median was 265.79 seconds versus Qiskit's 422.07 seconds, with a reported Mann–Whitney result of $p=0.0404$ and Cliff's $\delta=-0.50$. For Task 6, Qiskit was faster: its median was 386.58 seconds versus Notate's 1619.33 seconds, with $p=0.0024$. The tutorial and Tasks 1, 2, 4, and 5 did not show significant between-condition timing differences at the paper's $0.05$ threshold, although Task 5 had significantly different variances. Thus, the results support the usefulness of particular visual abstractions, not a general speed advantage for notational programming. The final-task comparison also reflects the prototype's editing affordances. Qiskit users often copied their previous solution into the next task's notebook and adapted it, whereas Notate could copy canvases within a notebook but could not copy them between notebooks. The analysis included the recorded times of the three Notate participants who did not finish Task 6, so its timing summary should not be interpreted as completion time for twelve successful solutions. Qiskit participants rated their interface easier and more enjoyable, with medians of 5 rather than 4 for both questions. Their observed errors frequently involved qubit indices; such indexing errors were absent in the Notate condition, but Notate introduced other errors involving recognition, wiring, and abstraction. ## Contributions, limitations, and future directions The main contributions are the articulation of notational programming, an implemented notebook interface, the Qaw notation and partial interpreter, and empirical evidence about how programmers use and understand this combination. The paper also contributes a design argument: handwritten notation may be executable code in its own right, and its introduction can change users' assumptions about what counts as programming. Interviewees sometimes distinguished drawing from coding, but some reconsidered that distinction when discussing abstraction and recursion. The authors describe this as a possible reconfiguration of programming practice, rather than demonstrating a lasting change in professional behavior. The study's strongest limitation is its novice population, which could not establish how Notate would fit into expert quantum programming workflows. Prior Python and Jupyter experience may favor the typed condition, the comparison covers one historical API and prototype configuration, and the tasks involved much more drawing than typing in the Notate condition. The findings therefore do not settle how the modalities compare for experts, other APIs, or workflows that regularly combine handwriting and typed code. Recognition reliability and unclear debugging feedback further limit the prototype's usability. At the notation level, the authors identify difficulties expressing classical data encoded into quantum operations and repeated parameterized operations, and acknowledge that typed code may remain preferable for particular subtasks. Proposed next steps include participatory design with quantum experts, studies with more balanced use of both input modes, richer drawing and copying tools, and error feedback that visualizes the recognizer's interpretation even when failure appears semantic. The authors suggest Wizard of Oz studies to refine notation before implementing recognition, better tools for defining domain-specific interpreters and communication policies, and exploratory systems whose notations could adapt to users' practices. Their distinction between notating and sketching is deliberate: these drawings are intended to approach a final executable expression and require precision, while informal early-stage exploration can still take place on paper. Broader claims about reusable drawing interfaces, new programming postures, and portable image-based programs remain research directions motivated by this prototype and study.