Here at SensiLab we are designing, implementing and evaluating improvisational interfaces - computer interfaces for improvising in creative arts domains. Ultimately we are interested in realising the notion of machines as creative collaborators, and as part of this overall goal we seek to implement improvisational behaviours in machine agents, facilitate improvised exchanges between humans and computers, and create guidelines for designing interfaces that support improvised interaction.
Improvisation is well established mode of creative practice across many different domains: from music, through painting to drama, performance and beyond. Traditionally, improvisation research has mostly considered improvisation between human artists. However, in recent years, interest in improvising with non-human intelligences or systems has grown. We feel that the field has reached a level of maturity that now requires detailed understanding of both the mechanics and psychology of improvisation with non-human partners.
One of the new interfaces we have designed is called the Dodecahedrum, an embodied artificial musical agent, designed for improvising music in a creative human-machine partnership. It is a hand-held dodecahedron (12-sided regular solid) with faces made from wood panels. Any of the faces may be struck by hand to create a percussive sound akin to an orchestral wood-block. Several of the faces have tongues etched to alter the pitch and timbre when struck. In this sense it resembles an acoustic percussion instrument.
Inside the body of the Dodecahedrum is a micro-computer system designed for real-time sensor processing and actuator output. The system drives three piston solenoids arranged to strike the wooden faces from inside. Inputs to the system comprise an audio microphone in the centre of the body, contact microphones on three of the faces, and a 3-axis accelerometer.
The Dodecahedrum is designed to invite interaction and to elicit musical engagement. It is intended to stimulate a sense of creative partnership with the human player: it wants to be played with. The combination of form, materials and algorithmic behavior address these design goals. The shared physical interface allows for joint timbral manipulation of a single resonating face, and the acoustic interaction combined with haptic feedback aim to enhance the sense of a physically present co-creative agent.
Beyond initial design features, we will investigate machine learning approaches that allow the Dodecahedrum to Learn to Play, so that over time it can adapt to a particular human interlocutor, and co-develop interaction mechanisms for sustained engagement. Stay tuned for the ongoing adventures of the Dodecahedrum, here on the Sensilab Creative AI blog!