Photo: J. Fagerlönn

RHYME member Anders-Petter Andersson won best paper presentation when he presented Birgitta Cappelen and his paper on AudioMostly2016 the 6th October 2016.  The motivation the committee gave was: “A very pedagogical, well-designed and passionate presentation with humorous and daring elements”

The paper is called ‘Health Improving Multi-Sensorial and Musical Environments’ and, “focus on the designed qualities of the Polly World environment, and specifically on the multi-sensorial and musical interaction design”. In the paper they present how they have designed the interactive music and multi-sensorial qualities of the environment, to make them health promoting, by combining knowledge from the field of interactive Music composition, Interaction Design and Tangible Interaction with knowledge from therapeutic disciplines such as Music and Health, Music Therapy and Sensory Integration. One of the main findings is the positive results moving from Multi-Sensory feedback, focusing on auditive, tactile, visual modalities, to Multi-Sensorial, motivating mastering over time, creating expectations, building on rhythmic, narrative, musical, relational and cultural.

During November we user tested the 4th generation of the RHYME Co-creative Tangibles, the Polly World, at the Haug School and Resource Centre. This is the last user tests the RHYME project conduct at Haug with the children that have followed the project for 4 years .  We like to take this opportunity to thank all Haug personnel, children and parents for their unique contribution and engagement in the RHYME project.

During the user study both children, parents, personal assistants, teachers, music therapists and occupational therapists participated and gave their responses to the Polly World. With the Polly World we have tried to put together all the demands, suggestions and wishes from users and experts related to the three earlier generations of RHYME’s Co-creative Tangibles, in addition to new Social Media functionality.
In the 4th generation of tangibles, family and friends explore interaction with social media, sending musical messages, using the PollyCompose App with a graphical interface and text in twitter. The family and other persons interacted and sent messages with picture projected in the physical tangibles Polly Land and Compositions to the mobile tangibles Polly Fire, Polly Ocean and Polly Planet

Testing the tablet interface on an iPad.

First you choose the Polly Compose APP on your desktop. The first, Scenes & Things-screen, of three  in the APP opens (see picture to the left).
There you choose Scene (music and visuals) and Thing (Polly Ocean, Polly Fire, Polly Planet). Then hit the [play] button at the bottom (see picture left).

The second screen with the chosen thing in the middle opens (see picture in the middle). Here you can test out the and learn how to play on the thing and how it sounds, before sending the composition to be played on the physical thing at home. You build up a composition by hitting the sensors on the screen representing the sensors on the interactive physical thing. You can also specify how you want the action to be performed (fast, slow, slow etc.).  A composition is being built up and visualized over the play and send buttons at the bottom of the screen (see right upper picture).
You play your composition by hitting the [play] button. You send the composition by hitting [send], and you edit  (see left lower picture) and save and give it a name by hitting [edit] .
You can go to the first screen and choose another Scene or Thing by hitting  [back].

The sent Twitter Composition is received in the Interactive thing. In our example here Birgitta sends a Composition to Polly Planet (see picture lower right) and Polly Planet plays the music and light Composition.

• Dynamic and interactive change of microphone input in mobile Co-creative Tangibles. So the users continuously can change their singing while  interacting with diverse sensors of the wireless Tangibles.
• Closer and more intimate and embodied relation to the video projection, compared to traditional wall projection, TV and Computer Screens.
• Better sound quality in the mobile Tangibles, regarding both sound frequency range and volume.
• More varied musical choices and tunes ( for expanded the self-regulating possibilities), regarding forms of musical interaction (musicking), type of music and sound experiences (music tunes, genres and soundscapes), so the users can easily handle and choose music, music related activity and intensity level.
• More robust transparent textile surfaces, since light is an important response dimension both in the interactive surfaces and part of the whole audio-visual experience.
• Better touch sensor solutions, both regarding interaction possibilities and surface qualities like colour, softness, light response, responsiveness and durability.
• More engaging and sensorial stimulating surfaces, to engage and motivate interaction with non-computer surfaces to gain a more fluent tactile, sensorial and interactive (computer based ) experience.
• Easier grips and more possibilities to handle input sensors like microphone and bend sensors.
• Possibility for the user to include their own things into the musical experience and make their own things sing along and expand the self-regulating possibilities.
• Easier battery charging of the mobile Tangibles, so that the users easily can do it themselves.
• Easier start and stop of the system, so the user can handle it themselves.
• Movable Tangibles, so the whole Polly World can be moved, installed and stored rather easy. These practicalities took days and were in several cases impossible with earlier generations of the Co-creative Tangibles…
• Social Media functionality  for distributed interaction on Smartphones and Tablets over the Internet (will be described in detail in a later post).

All these user and expert wishes (and many more) we have tried to put together in the new generation of the Co-creative Tangibles, Polly World. The Polly World consists currently of 4 Interactive, networked things, Co-creative Tangibles. The biggest , Polly Land, is wired , and 3 are mobile and wireless.  Polly Land contain a close projection using back ground projection onto a acrylic surface included in the soft textile  landscape. Polly Land has three “arms” containing RFID-reader, camera and microphone. It has a stand alone Wi-Fi system. The three other Polly interactive things is Polly Planet, Polly Fire and Polly Ocean.

They have their own separate colour  so they are easy to identify and interact with using a graphical interface on Smartphone and iPad. But the different sensors has the same shape, surface and interaction form. For instance is all touch sensors soft velvet triangles with the same output light pattern, but in Polly Land they are blue, in Ocean they are turquoise, in Fire they are orange and in Polly Planet they are yellow.  The same is the case with all input sensors like the bend sensors, the microphones, the RFID-readers and inductive charger.

The different shapes are built on traditional shapes within the field, such as Polly Planet that is bases on a ball shape. The shapes are constructed of triangles joined together with strong bands like trails the users can follow around the shapes to find more  sensorial challenges and experiences. The input areas are coloured sports fabric that contrasts the passive grey wool patches. Every Polly Tangibles, interactive thing has big light output patches included, some with speakers embedded.

Here are some more situations from the pilot test 16. June 2014:
       

In the 4th generation we try to accomplish, develop and realize as many of the ideas, suggestions and wishes as possible, that we have received from the users, care persons, families and experts from several fields throughout the whole project. These many ideas, suggestions and wishes have been articulated through participation in the many workshops, tests, interviews, discussions and observations during the whole RHYME project.

Some of the users’ wishes, such as dynamic microphone input to the music together with different forms of sensorial stimulating input sensors in a mobile interactive thing, has been a major technological challenge. These user wishes have compelled us to develop six different hardware platforms throughout the project, which was not something we planned to do, but had to do because of its importance for the users and music therapists.

Including direct light response and projection in the tangible objects has also been an important user demand that has been a great design challenge. We try to include and prioritize as many user wishes as possible into a new 4th generation of distributed and social co-creative tangibles with new shapes, structures, surfaces, hardware,  software, music, set design, positive experiences and mastery levels, personalization and interaction forms. Here are some pictures from the ongoing development process:

New hardware solutions under development:
   

New shapes under development:
   

New materials, surfaces and joinery under development:
   

New light solutions under development:
   

New APP with graphical user interface under development:
   

Chilli Net

Cove

Dynamic Rock

Ebbe

Stump Drum

Whispering Willow