The project is to develop a new textile for sensorial and musical interaction. Textile which has a soft and sensorial stimulation qualities,  in combination with transparency to transmit light and roughness to handle the rough use. In addition the textile has to have the ability to embed electronics into the textile.

We have worked from different angles, providing aesthetically and sensorial qualities by exploring combinations of yarn like cotton, wool, viscose, polyamid, polyester to high tech yarns like mono filament, pemotec and thermoformable yarns.

We have worked with different structures/patterns to embedd the technology like light, sensors, actuators and wires.

We have worked with different forms of knitting techniques from jacquard knitting to complex and simpler forms of tubular structures.

We have explored different forms of knitting machines from tubular machines to complex flat beds.

We have worked in different scales from many small samples to larger moch-ups and prototypes to be.

The title of Birgitta and Anders-Petters paper is “Embodied and Distributed Parallel DJing” and focuses on the innovative potential of including people with special needs when designing and creating cultural artifacts and activities. In the paper they argue that “…because of the users’ extreme demands and rich contribution, we ended up creating both a new genre of musical instruments and a new art form. We call this new art form Embodied and Distributed Parallel DJing, and the new genre of instruments for  Empowering Multi-Sensorial Things.” Access the Proceedings and paper here.

Here is a video that demonstrate the new practice of Embodied Parallel DJing, where Anders and Birgitta are using their whole body when cutting, scratching (DJing) together in parallel with Polly Ocean and Polly Planet on familiar music tunes that they choose with the RFID-tagged objects.

Testing the tablet interface on an iPad.

First you choose the Polly Compose APP on your desktop. The first, Scenes & Things-screen, of three  in the APP opens (see picture to the left).
There you choose Scene (music and visuals) and Thing (Polly Ocean, Polly Fire, Polly Planet). Then hit the [play] button at the bottom (see picture left).

The second screen with the chosen thing in the middle opens (see picture in the middle). Here you can test out the and learn how to play on the thing and how it sounds, before sending the composition to be played on the physical thing at home. You build up a composition by hitting the sensors on the screen representing the sensors on the interactive physical thing. You can also specify how you want the action to be performed (fast, slow, slow etc.).  A composition is being built up and visualized over the play and send buttons at the bottom of the screen (see right upper picture).
You play your composition by hitting the [play] button. You send the composition by hitting [send], and you edit  (see left lower picture) and save and give it a name by hitting [edit] .
You can go to the first screen and choose another Scene or Thing by hitting  [back].

The sent Twitter Composition is received in the Interactive thing. In our example here Birgitta sends a Composition to Polly Planet (see picture lower right) and Polly Planet plays the music and light Composition.

RHYME contributed with two keynote lectures at the 10th Nordic Snoezelen Conference 2014 in Helsingør. The two lectures were Birgitta Cappelen’s Designing Next Generation Interactive MSE and Anders-Petter Andersson’s lecture Music in RHYME.
In Birgitta’s lecture she argued based on a democratic and humanistic perspectives and a resource oriented approach for a design of Multi Sensory Environments  (MSE) that offers continuous positive experiences and challenging mastery curve for diverse users. This demands a design that is both sensorially stimulating and easy in use and complex at the same time. This demands a computational and software solution that offers many ways to interact and develop actability and mastery over time, not only simple switch solutions.

In Anders-Petter’s lecture Music in RHYME he explained how the musical simplicity and complexity is built up, based on musical sounds and programmed rules that are activated in different user-situations.

The Nordic Snoezelen (also called Multi-sensory Environment – MSE) Conference is a biannual conference and the most important community in the Nordic Countries for use of sensory stimulation in therapy and health promotion.
The Norwegian Community is called Norske Sanserom, the American Community is called AAMSE and the international organisation ISNA. Within MSE very little has been done regarding use of advanced computational possibilities, interactive music and professional design and aesthetic competence when creating the Multi-Sensory Environments. In this context  RHYME is a unique project internationally. If you  want to read more about MSE research you will find a list here.

• Dynamic and interactive change of microphone input in mobile Co-creative Tangibles. So the users continuously can change their singing while  interacting with diverse sensors of the wireless Tangibles.
• Closer and more intimate and embodied relation to the video projection, compared to traditional wall projection, TV and Computer Screens.
• Better sound quality in the mobile Tangibles, regarding both sound frequency range and volume.
• More varied musical choices and tunes ( for expanded the self-regulating possibilities), regarding forms of musical interaction (musicking), type of music and sound experiences (music tunes, genres and soundscapes), so the users can easily handle and choose music, music related activity and intensity level.
• More robust transparent textile surfaces, since light is an important response dimension both in the interactive surfaces and part of the whole audio-visual experience.
• Better touch sensor solutions, both regarding interaction possibilities and surface qualities like colour, softness, light response, responsiveness and durability.
• More engaging and sensorial stimulating surfaces, to engage and motivate interaction with non-computer surfaces to gain a more fluent tactile, sensorial and interactive (computer based ) experience.
• Easier grips and more possibilities to handle input sensors like microphone and bend sensors.
• Possibility for the user to include their own things into the musical experience and make their own things sing along and expand the self-regulating possibilities.
• Easier battery charging of the mobile Tangibles, so that the users easily can do it themselves.
• Easier start and stop of the system, so the user can handle it themselves.
• Movable Tangibles, so the whole Polly World can be moved, installed and stored rather easy. These practicalities took days and were in several cases impossible with earlier generations of the Co-creative Tangibles…
• Social Media functionality  for distributed interaction on Smartphones and Tablets over the Internet (will be described in detail in a later post).

All these user and expert wishes (and many more) we have tried to put together in the new generation of the Co-creative Tangibles, Polly World. The Polly World consists currently of 4 Interactive, networked things, Co-creative Tangibles. The biggest , Polly Land, is wired , and 3 are mobile and wireless.  Polly Land contain a close projection using back ground projection onto a acrylic surface included in the soft textile  landscape. Polly Land has three “arms” containing RFID-reader, camera and microphone. It has a stand alone Wi-Fi system. The three other Polly interactive things is Polly Planet, Polly Fire and Polly Ocean.

They have their own separate colour  so they are easy to identify and interact with using a graphical interface on Smartphone and iPad. But the different sensors has the same shape, surface and interaction form. For instance is all touch sensors soft velvet triangles with the same output light pattern, but in Polly Land they are blue, in Ocean they are turquoise, in Fire they are orange and in Polly Planet they are yellow.  The same is the case with all input sensors like the bend sensors, the microphones, the RFID-readers and inductive charger.

The different shapes are built on traditional shapes within the field, such as Polly Planet that is bases on a ball shape. The shapes are constructed of triangles joined together with strong bands like trails the users can follow around the shapes to find more  sensorial challenges and experiences. The input areas are coloured sports fabric that contrasts the passive grey wool patches. Every Polly Tangibles, interactive thing has big light output patches included, some with speakers embedded.

Here are some more situations from the pilot test 16. June 2014:
       

In the 4th generation we try to accomplish, develop and realize as many of the ideas, suggestions and wishes as possible, that we have received from the users, care persons, families and experts from several fields throughout the whole project. These many ideas, suggestions and wishes have been articulated through participation in the many workshops, tests, interviews, discussions and observations during the whole RHYME project.

Some of the users’ wishes, such as dynamic microphone input to the music together with different forms of sensorial stimulating input sensors in a mobile interactive thing, has been a major technological challenge. These user wishes have compelled us to develop six different hardware platforms throughout the project, which was not something we planned to do, but had to do because of its importance for the users and music therapists.

Including direct light response and projection in the tangible objects has also been an important user demand that has been a great design challenge. We try to include and prioritize as many user wishes as possible into a new 4th generation of distributed and social co-creative tangibles with new shapes, structures, surfaces, hardware,  software, music, set design, positive experiences and mastery levels, personalization and interaction forms. Here are some pictures from the ongoing development process:

New hardware solutions under development:
   

New shapes under development:
   

New materials, surfaces and joinery under development:
   

New light solutions under development:
   

New APP with graphical user interface under development:
   

Chilli Net

Cove

Dynamic Rock

Ebbe

Stump Drum

Whispering Willow