Sequencing and VJing: Iannix, Duration and my OSC to MIDI converter

Hi there,

I have just discovered the existence of two great tools: Iannix and Duration.

They call themeselves timelines or sequencer, the idea behind both is the same: offering a tool for synchronizing and orchestrating programs for music and video through MIDI and OSC messages.

Iannix seems to be an advanced and complex instrument. As its website says:

IanniX is a graphical open-source sequencer, based on Iannis Xenakis works, for digital art. IanniX syncs via Open Sound Control (OSC) events and curves to your real-time environment.

A screenshot of Iannix

It allows building bidimensional and even tridimensional waveforms that can be used to ‘play’ a virtual instrument, i.e. an OSC or MIDI channel, mapped to whatever you want. You can create these forms manually or you can generate them synthetically through a scripting language.

Duration according to its website is:

a timeline for creative coding. Create live performances, interactive installations, and music visualizations by synchronously composing servos, lighting, and projection

A snapshot of Duration

It is a simpler version of Iannix, it hasn’t any scripting language and the interface is more reduced. Though more limited than Iannix, it has a very interesting feature: it can import mp3 tracks so that you can synchronize things with the music, while in Iannix you only have time as a reference.

So I decided to test Duration more carefully. One problem I have though is that Duration only sends OSC messages and no MIDI, so if I want to use it with Resolume 2.4 I need a OSC to MIDI conversion.

I have tried a couple of tools out there: Moco and Konkreet but I have to say that they did not work they way I expected.

So I decided to make a tool myself in Java, the OSC2MIDI, available for download at my code repository.
It is called OSC2MIDI and can be downloaded as a jar executable here.

OSC2MIDI interace

Arduino to MIDI implemented !


I have finally reached one of the objectives I have had for a long time since I have started this blog: converting physical signals captured by Arduino into MIDI messages that I could use in some program on my laptop (like Resolume for VJing).

First of all I would like to tell you what DID NOT work for me.

I have seen on many blogs that what usually people do is a operating system-level trick: they install a serial to midi driver that reads the serial port and generates midi messages as it was a midi device connected to the system.
This is the way it is actually done by many midi devices manifacturers, and actually the trick just consists in downloading and installing one of their drivers.
The most famous one is the Roland serial midi driver, which is seems not available on some operating systems (e.g. Windows 7). A much more interesting driver is offered by Yamaha on this site, but I have tried it and it wouldn’t install either.

So in the end I thought: why don’t I just create a Java application myself that reads the serial port and routes the MIDI messages to some midi device?

The first problem was finding a reasonable solution for reading serial ports in Java.
You would think “hey! this is pre-history of computer science!”, or at least this is what I thought, but being Java anti-low-level-stuff it doesn’t have a “normal” way for doing it.
There is a standard API, the Java Communications API, that was once created by SUN and then discontinued. Actually the standard implementation, which used to work also on Windows, is now only available for some Unixes. Fortunately some guys created the RTXT library which is an alternative implementation of the same API, and which offers support for all operating systems. So I have adopted this last one and made it sniff my Arduino messages.

Then I had to implement a protocol to send midi messages from Arduino. I could have used the MIDI itself, but it was then difficult to separate messages one from another as there is no standard separator, so I just made an extremely simple protocol myself.

The protocol follows these rules:

  • one message per line (i.e. packets are separated by a carriage return)
  • Control changes are represented as: cc ch 10 ctrl 12 val 130 (where ch stands for channel, ctrl stands for control and val for value)
  • Notes ON are represented as: non ch 10 key 12 vel 120
  • Notes OFF are represented as: noff ch 10 key 14 vel 120
  • Aftertouches are represented as: at ch 10 key 14 tch 100

Then I created an utility that interprets these packets and sends them back to some MIDI interface you may have on your computer.
If the objective is to use these messages in some program, like in my case, then you would also need some MIDI “router” that receives messages and sends them back to another interface. For this objective I have used the LoopBe1 free tool, which works on my Windows 7 64 bits (i.e. it is updated). For other operating systems I am sure there are similar alternatives.

The result is some code for both Java and Arduino.

Let’s see the example for Arduino first:

void setup() {

void loop() {
float sensorValue = analogRead(0);
int intValue = (sensorValue / 1023) * 127;
String value = intValue;
Serial.println("cc ch 10 ctrl 12 val "+value);

As you may see, it is just a simple example that reads the value of pin 0 and sends it as a control change on channel 10, control number 12. The value is normalised to 0-127 (as the MIDI standard requires) and then put into the string.

More interesting may be the Java code, which can be seen here.

I have also resumed the functionalities in the wiki page.

In order to make it simple to use I have created a GUI for it. It lets you choose the midi output, the serial port and then it has just a start/stop button to activate/deactivate it.

In order to work, the proper native library of RTXT must be put into the Libraries folder when compiling it from Eclipse (see the original instructions here). The current native library is for Windows 64 bits.
If you have any problem don’t hesitate to contact me.

So, the proof of the concept is completed, now there are thousands of ideas that come to my mind on how to improve/generalise this.

Here you can see a demonstration that I am not lying:

Any comment is welcome !

MidiWii released !

MidiWii is a small program written in Java that translates the messages sent by a WiiMote into MIDI messages.
I have released the source code in my repository and also a downloadable executable jar here.

Things you can do with MidiWii:

– control some of your Resolume commands with the wii mote
– play some notes with the wii mote
– control some effects on audio and/or video by moving the wii mote

And here a couple of screenshots:



I have just started a new project called MidiDraw.
Check the code here.
Here there’s the description (taken from the repository wiki):

The aim of this project is to provide a technical mean to translate drawings into musical messages. The idea is not the one of creating another Midi sequencer, but something like an “interpreter” of a free hand drawing.

Technically the program will draw mouse inputs and will generate “features” from the drawn points.

The generated midi messages are most likely to be controller messages (although in principle it is possible to generates notes as well).

There is also the possibility to use live images from an external video source, like webcams.

The project is still at its very first stage, but will grow quickly!

Arduino Midi controller project


for my VJing I use the Resolume software (www.resolume.com), after trying many programs I’ve found out that it’s the most intuitive and powerful one for live video processing (at least for me !).

Cool, but as any live manipulating software it has a big limitation: you only have one mouse.
In order to allow a more direct interaction Resolume, as many more programs, can be controlled via MIDI.
Actually it allows you to map some parameters to effect channels on the MIDI interface.

Now you can buy a normal Midi controller form the market. They are usually thought to be used in music, so they present an interface that might be similar to a keyboard, or a mixer. It would be cool to have something more similar to the Resolume interface (or any other software). Actually people from resolume used to sell a midi device that was tight coupled to the program graphical interface, but as newer versions of the program changed that interface, the product was dismissed (a part that it was quite exepnsive).

As I’ve heard about Arduino (www.arduino.cc) from a friend I thought that maybe it would give me the possibilit to build up my own midi controller for Resolume.

So here it begins my approach to this electronics, I have a mission: building my own midi controller.

For doing that I will need two things:

  1. make the appropriate hardware
  2. write the code to send midi messages from the arduino board

Next posts will describe how I have reached this objective (if I manage to)…