Photo: J. Fagerlönn

RHYME member Anders-Petter Andersson won best paper presentation when he presented Birgitta Cappelen and his paper on AudioMostly2016 the 6th October 2016.  The motivation the committee gave was: “A very pedagogical, well-designed and passionate presentation with humorous and daring elements”

The paper is called ‘Health Improving Multi-Sensorial and Musical Environments’ and, “focus on the designed qualities of the Polly World environment, and specifically on the multi-sensorial and musical interaction design”. In the paper they present how they have designed the interactive music and multi-sensorial qualities of the environment, to make them health promoting, by combining knowledge from the field of interactive Music composition, Interaction Design and Tangible Interaction with knowledge from therapeutic disciplines such as Music and Health, Music Therapy and Sensory Integration. One of the main findings is the positive results moving from Multi-Sensory feedback, focusing on auditive, tactile, visual modalities, to Multi-Sensorial, motivating mastering over time, creating expectations, building on rhythmic, narrative, musical, relational and cultural.

The title of Birgitta and Anders-Petters paper is “Embodied and Distributed Parallel DJing” and focuses on the innovative potential of including people with special needs when designing and creating cultural artifacts and activities. In the paper they argue that “…because of the users’ extreme demands and rich contribution, we ended up creating both a new genre of musical instruments and a new art form. We call this new art form Embodied and Distributed Parallel DJing, and the new genre of instruments for  Empowering Multi-Sensorial Things.” Access the Proceedings and paper here.

Here is a video that demonstrate the new practice of Embodied Parallel DJing, where Anders and Birgitta are using their whole body when cutting, scratching (DJing) together in parallel with Polly Ocean and Polly Planet on familiar music tunes that they choose with the RFID-tagged objects.

RHYME member Karette Stensæth has worked very hard for a long time to write and edit the book  about RHYME called “Music, Health, Technology and Design”. The book is part of the Book Series from Centre for Music and Health and can be ordered here.

From the back side of the book (see the orange page above):
“Imagine that objects in your home environment – let us say a pillow, a carpet or a toy – became musical and interactive. Do you think that they could offer new ways of playing and being together? Could they even have the potential to reduce isolation and passivity and promote health and well-being for some of us?

This anthology, the eighth in the Series from the Centre for Music and Health, presents a compilation of articles that explore the many intersections of music, health, technology and design. The first and largest part of the book includes articles deriving from the multidisciplinary research project called RHYME (www.rhyme.no). They engage with the study of the design, development, and use of digital and musical ‘co-creative tangibles’ for the potential health benefit of families with a child having physical or mental needs.

Well-known international researchers broaden the picture on the book’s topic in the second part. They ask: How can video-based visualisation techniques of music-related body motion diagnose health problems? How can music therapy practice profit by digitalised improvisation analysis? What are the implications of gender and age in music technology for therapists and the people with whom they work? All together, this book
supplies a broad perspective on its topic, which should be of interest to a wider audience.
The Centre for Music and Health at the Norwegian Academy of Music was established in 2008. The centre conducts research and dissemination. Its goal is to develop knowledge about the connections between music and health.”

Table of Contents
Foreword
Natasha Barrett

Editor’s foreword
Karette Stensæth

Designing four generations of ‘Musicking Tangibles’
Birgitta Cappelen and Anders-Petter Andersson

Vocal and tangible interaction in RHYME
Anders-Petter Andersson and Birgitta Cappelen

An interactive technology for health:
New possibilities for the field of music and health and for music therapy?
A case study of two children with disabilities playing with ‘ORFI’
Karette Stensæth and Even Ruud

Potentials and challenges in interactive and musical collaborations involving children with disparate disabilities
A comparison study of how Petronella, with Down syndrome, and Dylan, with autism, interact with the musical and interactive tangible ‘WAVE’
Karette Stensæth

‘Come sing, dance and relax with me!’
Exploring interactive ‘health musicking’ between a girl with disabilities and her family playing with ‘REFLECT’ (A case study)
Karette Stensæth

‘FIELD AND AGENT’: Health and characteristic dualities in the co-creative, interactive and musical tangibles in the RHYME project 
Ingelill Eide

Health affordances of the RHYME artefacts
Even Ruud

PARTICIPATION: A combined perspective on the concept from the fields of informatics and music and health
Karette Stensæth, Harald Holone, and Jo Herstad

From experimental music technology to clinical tool
Alexander Refsum Jensenius

Technology and clinical improvisation – from production and playback to analysis and interpretation
Jaakko Erkkilä, Esa Ala-Ruona, and Olivier Lartillot

Using electronic and digital technologies in music therapy: the implications of gender and age for therapists and the people with whom they work
Wendy L. Magee

During November we user tested the 4th generation of the RHYME Co-creative Tangibles, the Polly World, at the Haug School and Resource Centre. This is the last user tests the RHYME project conduct at Haug with the children that have followed the project for 4 years .  We like to take this opportunity to thank all Haug personnel, children and parents for their unique contribution and engagement in the RHYME project.

During the user study both children, parents, personal assistants, teachers, music therapists and occupational therapists participated and gave their responses to the Polly World. With the Polly World we have tried to put together all the demands, suggestions and wishes from users and experts related to the three earlier generations of RHYME’s Co-creative Tangibles, in addition to new Social Media functionality.
In the 4th generation of tangibles, family and friends explore interaction with social media, sending musical messages, using the PollyCompose App with a graphical interface and text in twitter. The family and other persons interacted and sent messages with picture projected in the physical tangibles Polly Land and Compositions to the mobile tangibles Polly Fire, Polly Ocean and Polly Planet

Testing the tablet interface on an iPad.

First you choose the Polly Compose APP on your desktop. The first, Scenes & Things-screen, of three  in the APP opens (see picture to the left).
There you choose Scene (music and visuals) and Thing (Polly Ocean, Polly Fire, Polly Planet). Then hit the [play] button at the bottom (see picture left).

The second screen with the chosen thing in the middle opens (see picture in the middle). Here you can test out the and learn how to play on the thing and how it sounds, before sending the composition to be played on the physical thing at home. You build up a composition by hitting the sensors on the screen representing the sensors on the interactive physical thing. You can also specify how you want the action to be performed (fast, slow, slow etc.).  A composition is being built up and visualized over the play and send buttons at the bottom of the screen (see right upper picture).
You play your composition by hitting the [play] button. You send the composition by hitting [send], and you edit  (see left lower picture) and save and give it a name by hitting [edit] .
You can go to the first screen and choose another Scene or Thing by hitting  [back].

The sent Twitter Composition is received in the Interactive thing. In our example here Birgitta sends a Composition to Polly Planet (see picture lower right) and Polly Planet plays the music and light Composition.

In his presentation Andersson argues that music is not longer only the sound object, instrument or piece of art, but the services around it. These are services in the form of actors with characters that behave coherent as they answer back musically, take initiative and develop over time. With perspectives from drama, game theory, actor-network-theory as well as from learning theories on motivation, Andersson explores two design cases from RHYME.

• Dynamic and interactive change of microphone input in mobile Co-creative Tangibles. So the users continuously can change their singing while  interacting with diverse sensors of the wireless Tangibles.
• Closer and more intimate and embodied relation to the video projection, compared to traditional wall projection, TV and Computer Screens.
• Better sound quality in the mobile Tangibles, regarding both sound frequency range and volume.
• More varied musical choices and tunes ( for expanded the self-regulating possibilities), regarding forms of musical interaction (musicking), type of music and sound experiences (music tunes, genres and soundscapes), so the users can easily handle and choose music, music related activity and intensity level.
• More robust transparent textile surfaces, since light is an important response dimension both in the interactive surfaces and part of the whole audio-visual experience.
• Better touch sensor solutions, both regarding interaction possibilities and surface qualities like colour, softness, light response, responsiveness and durability.
• More engaging and sensorial stimulating surfaces, to engage and motivate interaction with non-computer surfaces to gain a more fluent tactile, sensorial and interactive (computer based ) experience.
• Easier grips and more possibilities to handle input sensors like microphone and bend sensors.
• Possibility for the user to include their own things into the musical experience and make their own things sing along and expand the self-regulating possibilities.
• Easier battery charging of the mobile Tangibles, so that the users easily can do it themselves.
• Easier start and stop of the system, so the user can handle it themselves.
• Movable Tangibles, so the whole Polly World can be moved, installed and stored rather easy. These practicalities took days and were in several cases impossible with earlier generations of the Co-creative Tangibles…
• Social Media functionality  for distributed interaction on Smartphones and Tablets over the Internet (will be described in detail in a later post).

All these user and expert wishes (and many more) we have tried to put together in the new generation of the Co-creative Tangibles, Polly World. The Polly World consists currently of 4 Interactive, networked things, Co-creative Tangibles. The biggest , Polly Land, is wired , and 3 are mobile and wireless.  Polly Land contain a close projection using back ground projection onto a acrylic surface included in the soft textile  landscape. Polly Land has three “arms” containing RFID-reader, camera and microphone. It has a stand alone Wi-Fi system. The three other Polly interactive things is Polly Planet, Polly Fire and Polly Ocean.

They have their own separate colour  so they are easy to identify and interact with using a graphical interface on Smartphone and iPad. But the different sensors has the same shape, surface and interaction form. For instance is all touch sensors soft velvet triangles with the same output light pattern, but in Polly Land they are blue, in Ocean they are turquoise, in Fire they are orange and in Polly Planet they are yellow.  The same is the case with all input sensors like the bend sensors, the microphones, the RFID-readers and inductive charger.

The different shapes are built on traditional shapes within the field, such as Polly Planet that is bases on a ball shape. The shapes are constructed of triangles joined together with strong bands like trails the users can follow around the shapes to find more  sensorial challenges and experiences. The input areas are coloured sports fabric that contrasts the passive grey wool patches. Every Polly Tangibles, interactive thing has big light output patches included, some with speakers embedded.

Here are some more situations from the pilot test 16. June 2014:
       

In the 4th generation we try to accomplish, develop and realize as many of the ideas, suggestions and wishes as possible, that we have received from the users, care persons, families and experts from several fields throughout the whole project. These many ideas, suggestions and wishes have been articulated through participation in the many workshops, tests, interviews, discussions and observations during the whole RHYME project.

Some of the users’ wishes, such as dynamic microphone input to the music together with different forms of sensorial stimulating input sensors in a mobile interactive thing, has been a major technological challenge. These user wishes have compelled us to develop six different hardware platforms throughout the project, which was not something we planned to do, but had to do because of its importance for the users and music therapists.

Including direct light response and projection in the tangible objects has also been an important user demand that has been a great design challenge. We try to include and prioritize as many user wishes as possible into a new 4th generation of distributed and social co-creative tangibles with new shapes, structures, surfaces, hardware,  software, music, set design, positive experiences and mastery levels, personalization and interaction forms. Here are some pictures from the ongoing development process:

New hardware solutions under development:
   

New shapes under development:
   

New materials, surfaces and joinery under development:
   

New light solutions under development:
   

New APP with graphical user interface under development: