Mixed Realities: Ars Virtua & Huret & Spector Gallery

Summary

We will construct two devices, one in Ars Virtua, one in Huret & Spector Gallery, which, via the EnvironmentXML framework, will enable people around the world to build remote devices and environments that respond in realtime to the local environmental conditions of the two galleries. A Java applet and historical data repository, residing on the turbulence.org server, will enable the tracking of realtime and historical sensory data from the two locations as well as the connections that people make over time.

Proposal

This proposal takes as its starting point that the distinction between 'virtual' and 'real' is as quaint as the nineteenth century distinction between 'mind' and 'body'. As such, the space of Ars Virtua and Huret & Spector Gallery are to be treated equivalently.

We have already built a simple web-based framework, EnvironmentXML (see below) that enables people to tag and share data feeds, in order to connect remote environments and devices across the internet. This framework will be extended to include the capability of accessing and responding to Ars Virtua environmental conditions and for Ars Virtua devices to access and respond to Huret & Spector Gallery environmental conditions. Since the Environment XML protocol is relatively easy to implement, particulary for any device built using Processing or Arduino, we hope that over the course of several months more and more other artists and designers will be able to build their own local devices that respond in realtime to both the Ars Virtua and Huret & Spector Gallery remote environments.

This will entail the construction of two devices and a java applet:

• The first device will reside in Ars Virtua. Its 'sensors' will track local environmental data [requires further research into the extent of Linden Scripting Language; we hope for example to construct a "proximity" detector that returns values for how close people are to it, or their compass direction]. Its 'actuators' will display similar data for the corresponding data that are simultaneously measured in Ars Interactive. The device may also display environmental data from some of the other Environment XML data feeds around the world, e.g. London and Tokyo.
• The second device, constructed around the open source Arduino hardware platform, will reside in Huret & Spector Gallery and will similarly be composed of 'sensors' [measuring for example temperature, sound, light level, people proximity] and 'actuators' displaying the concurrent values for Ars Virtua's device and other environments.
• A final artefact will be a java applet, to reside on Turbulence's server; this will display the data states of the two devices and also the data states of any remote devices or environments that members of the public have connected to the two devices using Environment XML. If possible, a server-side application will also monitor and store historical data from the two environments so that over the course of several months it will be possible to see how they have altered or been affected by each other.

Further information

1. Details of how the project will be realized, including what software / programming will be used.

• EnvironmentXML (see below) has already been implemented using PHP/MySQL. This may change over the course of the next few months as the core applications are defined. LSL will be necessary for constructing the Ars Virtua device; Arduino and Processing will be used for the Ars Interactive device.

2. Enumerate the specific techniques/products/programs/languages/toolkits that you will be employing to realize the project

• The project will be constructed around EnvironmentXML which is already built and functioning with several existing devices and environments.
  
  The basic idea of Environment XML is that if someone has constructed an object that is sensing environmental data (temperature, light level, people proximity, sound level, pollution, whatever) then by networking the object it should be possible to share this data in realtime without needing to negotiate protocols. It works in much the same way as flickr does for photos: the website provides a means of browsing and selecting data feeds, that are served in a simple dialect of XML. In effect the server is a proxy for devices and environments that have been registered by people around the world and that ensures that the XML is served in Environment XML format. (At the moment, in early stages only 12 feeds have been registered). A Processing library is also available to make this as straightforward as possible for beginners to incorporate in their own projects.

3. If your project requires the procurement of a domain name, please specify why this is necessary, and where you plan to have the domain(s) hosted.

• EnvironmentXML is already hosted at www.haque.co.uk. The java applet and historical data will reside on Turbulence.org.

4. What portion of the project will be realized on the turbulence.org server?

• A java applet will reside on the Turbulence server; it displays current data states of the devices and the remote devices and environments that are connected to the Ars Virtua and Huret & Spector Gallery devices. It will also display historical data from the devices.
  
  The historical data from the two devices will be grabbed by and reside on the Turbulence server and may be constructed using PHP/MySQL (to be confirmed). This application will poll the two devices approximately 10 times per minute and store the values. This historical data will also be used in the applet described above.
  
  One person on our team would require shell/ftp access to the Turbulence.org server.

5. What portion of the project (if any) will be realized on server(s) other than turbulence.org?

• See above.

6. If possible, please state the approximate monthly bandwidth you anticipate using for this project.

• Unknown, but since no image/video traffic is required we do not expect bandwidth to be particularly high.

7. Will you work within the existing SL structure or do you plan to create something unique?

• We may have to develop some new techniques. Matt Biddulph has demonstrated a method for bringing external data into Second Life, which we will refer to. We will also need to explore ways to make an extensible method for extracting Second Life data.

8. How will the Ars Virtua component of your project interact with the Turbulence and Huret & Spector Gallery components?

• Please see above.

9. Will the realization or your piece fit into the 2000 m sq of Ars Virtua's main Gallery and within 500 prims?

• The devices situated in Ars Virtua and Huret & Spector Gallery should be quite small (approx. 20cm x 20cm x 20cm). We think of them much on the scale of the museum humidity sensors.

10. Will you be streaming video or audio into the space? How many streams will you be using, and have you done this before?

• No, we will not.

11. What are the dimensions of the piece?

• See above.

12. What equipment will it require? Are you able to supply the equipment?

• The Huret & Spector Gallery device will be constructed around a bluetooth-enabled Arduino device. It will be tethered (via bluetooth) to a computer that is connected to the internet, which will be running a Processing applet that serves and grabs the EnvironmentXML data. Ideally this computer would be provided for us by the venue. It is possible that later this year a new Ethernet-enabled Arduino will be released in which case the computer would not be required.

13. Other installation materials?

• No.

14. Will anything have to be suspended from the ceiling?

• No.

15. How will the piece be delivered to Huret & Spector Gallery?

• Delivered by UPS.

16. How will the piece be installed and by whom? How long will the installation take?

• Ideally the unit will be plug-and-play (if the Arduino ethernet is available in time). If we use Arduino BT, then some configuration may be necessary with the local computer. In addition, there might be some configuration needed locally to enable the router to serve EnvironmentXML data via the appropriate port. We hope that this can be accomplished by the gallery's technical staff.

Project Budget

Hardware

• Arduino BT (4): 320 euros
• Misc electronics & sensors: 150 euros

Software

• All open source: 0 euros

Time

• 3 people, 12 weeks, 1 day/week @ 100 euros/day: 3600 euros

Shipping (UPS)

• Small packet: 60 euros

Total

• 470 + 3600 + 60: 4130 euros = $5500.00
  (deficit of $500 to be sourced locally)

Bio/CVs

• Usman Haque has created responsive environments, interactive installations, digital interface devices and mass-participation performances. His skills include the design of both physical spaces and the software and systems that bring them to life. He has been an invited researcher at the Interaction Design Institute Ivrea, Italy, artist-in-residence at the International Academy of Media Arts and Sciences, Japan and has also worked in USA, UK and Malaysia. As well as directing the work of Haque Design + Research (www.haque.co.uk) he was until 2005 a teacher in the Interactive Architecture Workshop at the Bartlett School of Architecture, London.
  
  He is a recipient of a Wellcome Trust Sciart Award, a grant from the Daniel Langlois Foundation for Art, Science and Technology, the Swiss Creation Prize, Belluard Bollwerk International, the Japan Media Arts Festival Excellence prize and the Asia Digital Art Award Grand Prize. His work has been exhibited at the Institute of Contemporary Arts (London), Ars Electronica, Transmediale, Hillside Gallery (Tokyo), The National Maritime Museum Greenwich, the Tokyo Metropolitan Museum of Photography, Itau Cultural, Sao Paulo, NTT Inter-Communication Centre, Tokyo and the Singapore Biennale.
• Georg Tremmel studied Media Art in Vienna and Interaction Design at the Royal College of Art in London. His body of work mainly consists of interactive installations, of both the physical and meta-physical kind. Although he was trained as a designer, he is not afraid to make use of programming biotopes like Max/MSP, Processing, Xcode and Flash/Flex to realize his projects.
  
  Georg was awared the 'Product of the Future' Award for the Science Museum London, the Creative Pioneer Award from the National Endowment for Science, Technology and Art (NESTA), and the 2nd Prize at the Cambridge Biology in Business - Pitch Your Idea Competition. His work also won the grand prize of the VIPER Art Festival in Basel, Switzerland. In the 2005 he received a grant from the Art Section of the Office of the Austrian Chancellor to live and work in Japan for 6 months. He has been there ever since, and pretends not to care how people pronounce his first name.
• Too lazy to practice piano, Neill Zero instead taught himself to program in order to advance his music. And he got distracted. 10 years later, he has worked on everything from fun small wireless and telematics prototypes for IBM, through to production financial information systems. He's nuts about sharing knowledge, and about automating things. By day Neill works with Ruby on Rails for a startup in London. By night he works with Ruby on Rails for a startup in London. If he had more time, he'd return to his beloved arduino board, or maybe his interactive toys in Second Life. He has a piano keyboard in a loft, somewhere. Proficient in Java, MySQL and back-end programming in general.

Up to five examples of prior work accessible on the web.

• Open Burble
• Haunt
• Scents of Space
• Sky Ear
• Low Tech Sensors and Actuators for Artists and Architects (With Adam Somlai-Fischer)