NeuralGrid: A Physical Interface for Live Sequencing and Improvisation

Emory Smith · Ajay Kapur · Jake Cheng

California Institute of the Arts

Draft — NIME target · mid-build

Abstract

NeuralGrid is a sensor-dense physical controller whose models speak a symbolic vocabulary before they speak sound. Pads, ribbons, lidar, and contact microphones are smoothed and interpolated into a coherent output format defined by the dimensional space the sensors represent. A first model interprets gesture through a symbolic vocabulary borrowed from tabla — where bols are spoken strokes, theka holds a cycle, and tihai and kaida act as geometric transforms over stroke sequences — so phrases can be played, combined, varied, and spoken aloud. A second model translates that symbolic stream onto any synthesis engine through seven perceptual axes. The translator is deaf by design: it never hears what it renders, which keeps model latency structurally outside the audio path.

Paper not yet written

Structure

  1. Introduction — the Machine Lab, and the case for a universal controller
  2. Related work — grid controllers, gestural mapping, and computational models of tabla
  3. The symbolic layer — bols as vocabulary, theka as cycle, tihai and kaida as transforms
  4. The translator — seven perceptual axes, and what it means for a model to be deaf
  5. Instruments as constraint — hardcoding regions of translator behavior
  6. Hardware — pads, FSRs, ribbons, lidar, contact mics, and the sensor-to-space mapping
  7. Architecture — keeping every model out of the hot path
  8. Evaluation — latency of the fast plane; whether the axes are independently steerable
  9. Future work — Machine Lab renderer, browser controller/renderer pair

Resources