At Britelite, most of our projects involve dramatic and unusual presentations of interactive works. In order to build these projects, we use designs and software that go beyond those used in typical web and application development. This article describes some of these tools, how we use them, and how we integrate with other systems to create powerful and stable solutions.

Why creative technology needs its own tools

Creative technology tools are often designed to enable the “creative” side of the equation, by making powerful technology available to artists, designers and creators whose primary focus may not be writing code.

There are three things they have in common:

1. Easy to get started: they all present high level modules that can be used directly, without a lot of setup
2. Very rich and responsive multimedia output: they all support high quality, real-time video and audio presentation
3. Easy to extend and connect: they can all be extended to support new capabilities, in particular new and unusual input devices such as depth cameras and large scale touch interfaces

Trying to do these things from scratch using traditional coding environments is a lot more work, since a lot of setup needs to happen. (Similar challenges do exist in software development in general: frameworks and libraries are getting a lot of traction overall. Creative tech tools are especially focused on multimedia and interactivity.)

Tools that we use

I’ll go over three different categories of tools that we use:

1. Multimedia creation environments, like Touch Designer
2. Game development systems, like Unity
3. Coding frameworks, like Cinder

I’ll also talk about how we integrate these systems with each other, and with other application development environments.

Multimedia creation environments

These are software tools designed to allow creators to build interactive works without writing a lot of code. They include ways of defining installations that use a graphical user interface, where blocks and modules can be connected on a screen to create an installation.

We use Touch Designer, by Derivative Inc. This tool is particularly well known for its visual quality and high performance realtime graphics. Touch Designer was first released in 2008, and is closely related to the Houdini visual effects program. It is able to integrate a large variety of physical sensors and other systems. When the built in blocks are not enough, a developer can write programs in Python that extend Touch Designer.

We’ve used Touch Designer to build large-scale interactive video walls, and for playback of very high resolution videos. This giant living wall would respond as users walked in front of it:

Other examples of multimedia creative tools include the famous Max/MSP, which has been around for 30 years and is now part of the Ableton family of creative tools. Another system that we are very interested in is Notch, by Disguise, which is able to produce impressive realtime 3D visuals.

Game development systems

Video game developers have many of the same challenges as creative technologists, trying to express a creative vision that is immersive and responsive. In recent years, tools used for developing video games have become increasingly used for building interactive installations and even business applications.

At Britelite we are big fans of Unity, a very popular game development system. Unity is famous for letting a developer build software once that can be used on a number of different devices, such as mobile phones, computers and video game consoles. Unity is also well-known for its extremely well-stocked Asset Store, an online market place for software snippets, 3D models and everything else that you could need to build a game. A strong point of Unity is the very strong support for AR (augmented reality) and VR (virtual reality) systems. It is possible to quickly build an interactive 3D scene and make it available to users of many different systems.

We used Unity to build this robotic surgery simulation game, that was presented at the opening gala for a new hospital in San Francisco. For this project we used a special display called a Multitaction screen that responds to touch and to objects placed on it.

We also used Unity to create automatic beer pouring kiosks for a very high profile event. On this project, we integrated Unity with a variety of sensors, including a pressure mat to detect when a user approached, an NFC payment sensor and a depth camera for taking AR (augmented reality) selfies of a user wearing their favorite team’s colors as facepaint.

There are other game systems as well. Unreal Engine by Epic is very well known, and we use it in situations where very realistic rendering of 3D systems is required. We are also intrigued by Amazon’s Sumerian system, which is particularly good for building AR and VR applications.

Coding frameworks

When ultimate flexibility is required, there is no substitute for writing code. Many systems have emerged to make it easier for creators to build code-based projects.

A very well-known example is the Processing environment, which provides an easy-to-use development environment for writing code in Java. It is free and open-source, and it includes a vast number of extensions for different creative effects and interfaces. However, Java is subject to significant performance limitations, so there is a lot of work to make higher performance languages accessible to creators.

We have had good experiences using the Cinder framework, which is a library of modules that support graphics and interactivity using the C++ language. While harder to use than purely graphical systems, C++ is able to provide very high performance in the finished apps, and frameworks like Cinder save a lot of development time.

We used Cinder to build the control software for this elaborate LED illuminated tree:

Another framework similar to Cinder is OpenFrameworks, which also allows creators to quickly build media rich applications.

Finally, we often have to program tiny computers called Arduinos that will connect to specialist sensors and devices. The Arduino environment, originally developed for education, has grown over the years to support a vast number of embedded systems. We frequently use Arduino to build software to integrate physical sensors.

For this project, we built a series of tiny circuit boards with optical sensors, that allowed for accurately establishing the physical position of an architectural model, in order to project video imagery.

Integration

There are many creative technology projects that are limited to a specific event or performance. At Britelite, however, we are building projects to last. Our clients expect our projects to run reliably for a long time, to be able to change content, and to integrate with other systems.

We have developed an extensive skill set and software library for integrating creative technology tools and techniques with classical software development platforms. For example, we can build a simple website that allows a customer to change the content that is shown on various different displays and installations. Another key capability is being able to deploy new versions of software to interactive installations, without having to be onsite. We can collect usage data to prepare analytics reports – very useful to show the impact of an interactive piece.

For a recent gala to celebrate the opening of a new hospital, we integrated the following types of systems:

• Multi-touch table with surgical simulation game
• Large touchscreens for wayfinding
• Reactive living wall
• Interactive holographic displays
• Smart wearable bracelets to allow for customized experiences
• A registration microsite for the event
• Realtime analytics

Britelite’s own proprietary infrastructure managed software for more than 60 separate computer systems, allowing deployment of software and configuration changes centrally. This software backbone allows to connect immersive and interactive installations with front-end and back-end software tools that make our projects much more robust and cost effective. Britelite was able to manage all the technology at the event with a very small and efficient team, and was able to provide the event organizers with detailed analytics almost immediately.

If you are interested in creating powerful interactive experiences for your organization that go beyond the demo, please get in touch!