The website for Ear Trading is http://eartrading.moon.net.au

If you are interested in beta testing the application, please send your iPhone UDID (Unique [...]]]> Hearing the Unseen has evolved into an iphone game and generative music machine called Ear Trading.  It is now ready for beta testing after which it will be released as on the iTunes store as a free application.

The website for Ear Trading is http://eartrading.moon.net.au

If you are interested in beta testing the application, please send your iPhone UDID (Unique Device Identifier) to eartrading@moon.net.au.

\

Barry Moon Arizona State University Brenda Moon The Australian National University

This project started as a exploration of data sonification techniques. The abstract published in the conference program is a testament to this. [...]]]> This is based on a Poster/Demo presented at the “Humanities + Digital Visual Interpretations Conference” hosted by HyperStudio – Digital Humanities at MIT 20-22 May 2010.

Barry Moon
Arizona State University
Brenda Moon
The Australian National University

This project started as a exploration of data sonification techniques. The abstract published in the conference program is a testament to this. As it progressed, the idea of using real-time data to create a game for mobile devices became more alluring. More specifically, a game where sound plays a major role in decision making, or even a game that produces interesting music with minimal interaction. For this application, the meanings of the data become far less important than its trends and time basis. We are using stock market data which wakes up and goes to sleep at fixed times of the day. Music, being predominantly time based, suits this kind of predictable framework upon which to drape its material. Although the creation of a game is the direction our sonification research has taken us, similar techniques could be applied to data to not only reveal details, but make the exploration of those details interesting and fun.

As you will see in the conference program, our early model was programmed in Processing and Max/MSP. While this gave us a great deal of flexibility over visualization and sonification, it would not be usable on mobile devices in the near future. For our game, the iPhone/iPad was chosen as the device, and Unity as the programming solution. Choosing to use the iPhone/iPad over Macintosh should generate a greater amount of feedback from users to aid in development and create a stronger sense of intimacy with the interface via touchscreen.

Most approaches to producing sound in interactive media follow precedents set in film. So far, in our work, we have combined two sonic elements traditionally used to enhance visual media: music and “off-screen” sound. Music reveals details in the data in two ways:

1, Amount of rhythmic activity relates to a share’s trading volume – the greater trading volume per sample (6 seconds), the greater the number of beats per measure

2, Pitch change across the duration of the “measure” relates to price changes of the share -downward pitch sequence for falling share price and upward pitch sequence for increasing share price.

Figure 1: Typical bar of music with moderate trading volume (9 beats) and share increase in share price.

When data is off-screen, it continues to be heard, spatialized according to its position in the corresponding visual space:

Figure 2 – Position of sounds relative to visual space

Figure 2a: Game View

Figure 2b: Top View

Our main concern in creating a musical sonification (as it is in most music creation) is the balancing of repetition with variation. We generate music statistically by choosing beats via weighted probabilities. This extends a drum machine programming analogy, which if simplified (to create really horrible techno for example), the kick drum is heard on the downbeats and snare drum on upbeats.

Figure 3: Beat Tables

Audio samples are triggered at the statistically chosen beats. Audio synthesis would allow greater sonic variety, but there are no methods presently available in Unity. Sixteen percussive samples of different loudness were created for each sound. There needed to be some balance between percussiveness (which aids in direction perception), and “pitchedness” (since we are using pitch as a perceptual referent for price change). Samples needed to be quite short (less than one second), again because of limitations in the Unity environment.

We ultimately wanted to call our game “Our Tax Dollars at Work”, but found some of the bailout recipients have extremely low trading volume. Instead we chose shares with large trading volume (which does include CitiGroup and Fannie Mae), and our working title is now “b-trade”.

Enhancements we would like make to our game before letting it loose on the public (as much as iPhone/iPad users represent “the public”) are enhanced visual interface, greater variety of sounds, the potential for users to design their own “beat tables”, and for the game to morph between multiple “beat tables” to create greater musical variety. We would also like the user to be able to choose which company’s shares they follow.

Max/MSP: http://www.cycling74.com
Processing: http://processing.org/
Unity: http://unity3d.com/

Johnny Lee demos his wiimote whiteboard and wiimote headtracking/3D applications at TED 2008.

Uwe Schmidt has made an excellent Mac OS X version of wiimote whiteboard in Java (he also has a cross-platform version available).  It is available from his website - http://www.uweschmidt.org/wiimote-whiteboard - and includes TUIO support for multitouch.

Lots more information about Johnny Lee’s projects and other peoples additions to [...]]]>

Johnny Lee demos his wiimote whiteboard and wiimote headtracking/3D applications at TED 2008.

Uwe Schmidt has made an excellent Mac OS X version of wiimote whiteboard in Java (he also has a cross-platform version available).  It is available from his website - http://www.uweschmidt.org/wiimote-whiteboard - and includes TUIO support for multitouch.

Lots more information about Johnny Lee’s projects and other peoples additions to them is available on the Wiimote Project site.

I’ve just read a book called Presentation Zen by Garr Reynolds about how to make more effective presentations. Like many good ideas, it seems very simple and straight forward when you read it. Some ideas from Presentation Zen:

…keep these three words in mind always: simplicity, clarity, brevity. (p43)

If your presentation is successful, the audience will have no [...]]]>

I’ve just read a book called Presentation Zen by Garr Reynolds about how to make more effective presentations. Like many good ideas, it seems very simple and straight forward when you read it.


Some ideas from Presentation Zen:

…keep these three words in mind always: simplicity, clarity, brevity. (p43)

If your presentation is successful, the audience will have no idea how many slides you used, nor will they care. (p60)

…remember that there are three components to your presentation – the slides, your notes, and the handout… (p67)

Information plus emotion and visualization wrapped in unforgettable anecdotes are the stuff that stories are made of. (p82)

Today’s presentations increasingly share more in common with a documentary film than an overhead transparency. (p135)

He quotes from an article by Anna Patty in the Sydney Morning Herald to support his view that the current use of Powerpoint bullet point, text presentations is "Really Bad Powerpoint". Anna’s article is available online (Research points the finger at PowerPoint by Anna Patty, SMH, 4 April 2007) and I liked these quotes from it:

"The use of the PowerPoint presentation has been a disaster," Professor Sweller said. "It should be ditched."

"It is effective to speak to a diagram, because it presents information in a different form. But it is not effective to speak the same words that are written, because it is putting too much load on the mind and decreases your ability to understand what is being presented."

15 Stunning Cutting-Edge Gadgets and Technologies | Monday Inspiration | Smashing Magazine In this post we present stunning examples of cutting edge technology which is already reality today or will become reality in 2008. Please notice that some videos might not provide [...]]]> Today’s post on Smashing Magazine has interesting examples of cutting edge display and user interface technology.

15 Stunning Cutting-Edge Gadgets and Technologies | Monday Inspiration | Smashing Magazine
In this post we present stunning examples of cutting edge technology which is already reality today or will become reality in 2008. Please notice that some videos might not provide you with a concrete idea of what the technology offers, e.g. it’s impossible to display the advantages of 3D-TV using videos produced by a 2D-video-camera.

I particularly liked the “Funky Forest” installation and seeing how the children enjoyed interacting with it.

Funky Forest
Funky Forest’ is an interactive ecosystem where children create trees with their body and then divert the water flowing from the waterfall to the trees to keep them alive. The health of the trees contributes to the overall health of the forest and the types of creatures that inhabit it.

The article also mentions some of the possibilities for using a Nintendo Wiimote to make interesting user interfaces for computers. As well as showing Johnny Chung Lee’s Wiimote experiments it links to a demo by Mike Rouse building on Johnny’s idea of tracking infrared lights on your hands. He has made an interface similar to Microsoft Surface using gloves with infrared lights in them and the wiimote to track them.

The Coolest Gloves Ever?