Utilizing Bluetooth Low Energy (BLE) to send MIDI packets

Author: Tushar Agarwal, iOS Developer

This article will throw light on the utilization of Bluetooth Low Energy to send MIDI packets containing musical information which is generated according to the angle of rotation and linear velocity of the device being tracked by the smart sensors like Accelerometer and Gyroscope.

At FreshWorks Studio we recently developed an iOS application for CanAssist that used this functionality and is live at AppStore as Waveband Pro.

This video with CanAssist shows a Central device (iPad) connected to an audio speaker for sound output and multiple Peripherals (External Hardware Devices) sending musical notes via MIDI over BLE.

Overview of the Technologies

MIDI (short for Musical Instrument Digital Interface) is a technical standard that describes a communications protocol, digital interface, and electrical connectors that connect a wide variety of electronic musical instruments, computers, and related audio devices. A single MIDI link can carry up to sixteen channels of information, each of which can be routed to a separate device. MIDI carries event messages that specify notation, pitch, velocity, vibrato, panning, and clock signals (which set tempo). Advantages of MIDI include small file size, ease of modification and manipulation.

Bluetooth Low Energy (colloquially BLE, formerly marketed as Bluetooth Smart) is a wireless personal area network technology designed and aimed at novel applications in the healthcare, fitness, beacons, security, and home entertainment industries. Compared to Classic Bluetooth, Bluetooth Low Energy is intended to provide considerably reduced power consumption and cost while maintaining a similar communication range. Mobile operating systems including iOS, Android, Windows Phone and BlackBerry, as well as macOS, Linux, and Windows, natively support Bluetooth Low Energy.

Accelerometers in mobile phones are used to detect the orientation of the phone. The Gyroscope adds an additional dimension to the information supplied by the accelerometer by tracking rotation or twist. An accelerometer measures linear acceleration of movement, while a gyroscope, on the other hand, measures the angular rotational velocity. In practice, that means that an accelerometer will measure the directional movement of a device but will not be able to resolve its lateral orientation or tilt during that movement accurately unless a gyroscope is there to fill in that information.

What is MIDI over BLE on iOS?

Apple introduced support for MIDI over Bluetooth Low Energy in iOS 8 and OS X Yosemite in the summer of 2014 and added features in 2015. A number of products have already been introduced for sale supporting Apple’s MIDI over BLE protocol. That being said, sending and receiving MIDI data is supported using Bluetooth Low Energy connections on any iOS device or Mac that has native Bluetooth Low Energy support. All established connections are secure which means that pairing is enforced and connections cannot be made to the devices without their explicit consent. After a connection is established, it simply appears as an ordinary MIDI device that any MIDI application can communicate with.

How does it work?

There are two key roles involved in a Bluetooth connection; the Central and the Peripheral. The Peripheral’s job is to become discoverable and advertise that it has some functionality you can connect to. For Bluetooth MIDI, the peripheral side will advertise its MIDI capabilities. The Central may scan for a Peripheral device that is advertising MIDI capability and then establish a connection. There are two view controller objects which are part of the CoreAudioKit framework that can be used to manage Bluetooth MIDI connections on iOS. Both of these Bluetooth MIDI view controllers inherit from UIViewController.

1. CABTMIDICentralViewController allows an application to play the role of the Central, which means the application can scan and connect to a Peripheral device.

2. CABTMIDILocalPeripheralViewController allows an application to play the role of Peripheral, which allows the application to advertise its services and wait for a connection.

In order to have a Bluetooth connection one Central and a minimum of one Peripheral device is required.

After a Bluetooth connection has been established, MIDI data can be transferred bi-directionally between the Central and the Peripheral. Both iOS devices and Macs can play either role allowing Mac to Mac, iOS to iOS, Mac to iOS, or iOS to Mac connections. When two devices are not communicating for a while over an established Bluetooth MIDI connection and the connection is unused by the application, the Bluetooth connection will automatically terminate after several minutes to save power.

How to use it in conjunction with Accelerometer and Gyroscope?

Apple provides Core Motion framework which reports motion and environment related data from the onboard hardware of iOS devices, including from the accelerometers and gyroscopes, and from the pedometer, magnetometer, and barometer.

A CMMotionManager object is used to start the services that report movement detected by the device’s onboard sensors and receive it’s motion data:

1. Accelerometer data as CMAccelerometerData, indicating the instantaneous acceleration of the device in three dimensional space.

2. Gyroscope data as CMGyroData, indicating the instantaneous rotation around the device’s three primary axes.

The live sensor data can be received at specified update intervals, or can be collected and stored for its retrieval later. With both of these approaches, the appropriate start method (startAccelerometerUpdates(), startGyroUpdates()) and the stop method (stopAccelerometerUpdates(), stopGyroUpdates()) is required to be invoked when the data is no longer needed.

Once the data is available, mathematical calculations are done to calculate the value of the musical note to be sent in the MIDI packet over BLE.

This empowers you to play musical notes according to the angle and velocity of the device. SOUNDS COOL RIGHT?

We did a similar implementation in Waveband Pro which has been designed to assist people with disabilities and improve their quality of life. The application facilitates people with a wide range of disabilities express themselves musically through movement. Through the movement of an iOS device, users convert motion into the sound of their choice of musical instrument. Users can roll, pitch or turn the device to play a piano scale, shake the device to play the tambourine, and so on. Waveband Pro is completely customizable to ensure that each user’s ability to move can be used to best advantage.

Why MIDI over BLE?

With an ever increasing number of gadgets, for example, the phenomenal of Bluetooth MIDI connectors accessible to us, it’s a great opportunity to use the energy of iOS gadgets and PCs as sound modules without the restraints of MIDI and USB links. MIDI over Bluetooth is a rising and developing technology which will continue to grow at an astonishing rate due to its key advantages like Simplicity, Low Power, Robust Radio and Real-time operation.

It’s not only the native iOS devices that can be utilized to take an advantage of this nifty protocol, but a wide variety of External Hardware Devices can also be configured as peripherals to interact in this environment, as we have done in Waveband Pro. These touch-sensitive peripheral devices send information packets containing musical notes to the central device when they are touched.