One thing that’s super cool about Bluetooth Low Energy is that it’s completely ubiquitous, and these days, super cheap to implement. That means, in addition to Garmins and Fitbits and Google Homes and other expensive devices, there’s an entire universe of way cheaper devices that can be made more useful (or, arguably, more frustrating) by throwing in a $1 BLE chipset to talk to your smartphone.

Case in point, the pulse oximeter pictured above. Pulse oximeters are used primarily by doctors in a hospital setting, or by patients with chronic respiratory diseases like chronic obstructive pulmonary disease (COPD) to check SPO2. That is the proportion of oxygen-saturated hemoglobin in the blood – the number of occupied oxygen binding sites on red blood cells, expressed as a percentage of the total. A healthy person at sea level will typically have about 99% SPO2. A sick patient might have more like 92%. Much lower than 85% is typically cause for concern, or so the internet says – I’m not a COPD patient. These devices are available in a handful of designs, cloned hundreds of times by as many sellers, and all available for between $15 and $30. Some have OLED screens, some LED segment displays. Some are sold on Amazon from reputable-looking manufacturers and some on eBay with no branding but identical molds. And now, at least two hardware designs I can find come with BLE built in, for the low low price of $25. I find that very interesting, so I bought one.

The BLE versions come with apps that log SPO2 and pulse over time. Respiratory patients tend to be interested in this functionality to get a longer-term idea of their respiratory health, sometimes overnight. Since SPO2 tends to vary with environmental factors by a few percentage points, a single reading doesn’t always tell the whole story. But what do I care?

Remember how I said SPO2 is typically 99% or so for a healthy person at sea level? All sorts of respiratory things change when you increase altitude and decrease atmospheric pressure. What I’d like to build for myself is a dataset of respiratory performance versus altitude, over time.  I have some curiosities about what I might find, although at this point, the ends don’t have to justify the means – I’m really just in it for the fun of it.

The proliferation of BLE into cheap commodity consumer devices has one notable disadvantage: manufactures of cheap commodities don’t usually consider the full user experience like Apple or Garmin or Fitbit. While a commodity device like a scale just does what it does, getting that data onto a digital platform doesn’t mean anything, really, unless you can do something useful with the data. On a computer, the world is your oyster. But on a phone, everything has to happen inside the app that talks to the device. There has to be a cohesive, thought-out user experience. What use is a food scale that simply tells you the same weight on your phone screen as on the LCD itself?

Rather than take manual readings with my pulse oximeter, and record altitude and location by hand, I’d like an app that data-logged all of this automatically whenever the data is available. The stock appstore app for this particular model pulse ox (branded Jumper) does datalogging, but that’s the end of it. Worse, there’s no way to export the data as a CSV or any such thing, to then pair up after the fact with contemporaneous altitude and location data from another source. I need to write my own app to do this, and for that, I need to figure out how to pull data off the pulse oximeter. While we’re at it, I want to do the same with the food scale, for reasons I’ll save for a different post.

First Stop: Exploring

BLE is pretty cool because it defines a stack from the physical layer up where you only need to fill in so many blanks. That is, if your device looks like a ton of other very similar devices, you can use an off-the-shelf protocol and just plug in your data – a heart rate monitor, for example. If your device doesn’t fit the mold, you go just a little lower and fill your own data into slightly less specific blanks. Unfortunately, it’s often the case that very generic devices still, for whatever reason, don’t use the standard profiles they could. For example, my food scale doesn’t use the scale profile, and my pulse oximeter doesn’t use one of (two, I think?) available pulse ox profiles. For this reason, we need to poke around to figure out what they’re doing. But because the BLE standard defines the intermediate layers, this isn’t so hard.

To get started, I downloaded LightBlue for my iPhone, which is a tremendously useful app for prototyping and exploring BLE systems. It lets us connect and probe the pulse ox connection without doing any sort of coding work whatsoever.

We’re immediately greeted with a bunch of information about the device. This is because the device uses the BLE Generic Attribute Profile (GATT), so the app knows exactly what to go looking for without knowing anything about the device. GATT is the structure that defines how every application protocol built on top of it transfers data. Application protocols include the heart rate profile, battery profile, and custom profiles such as the one we’re dealing with here. Because this custom profile builds on GATT, it’s not too hard to figure out quickly how it ticks.

The info above isn’t super useful – the name of the device in plain text (which doesn’t need to be included, and many devices don’t), the serial number (very helpfully, the serial number for this device is “Serial Number”), and so on. If we look inside ADVERTISEMENT DATA up there, though, we see a Service UUID listed – CDEACB80-5235-4C07-8846-93A37EE6B86D. Because the app saw that, too, it already probed it, so we can just scroll right down to see what’s inside it:

GATT defines Services, which contain Characteristics. A Service might be the “Heart Rate Service,” which contains the Characteristics “Heart Rate Measurement,” and “Body Sensor Location” to determine what kind of a heart rate sensor this is. (Since this is a defined service, you can read all about it in the official spec at the GATT specification page of the BLE website.) In this case, the service and its characteristics, being non-standard, don’t have names. But we can make a few solid guesses. Characteristics can be Read, Write, Notify, or Read/Write (or maybe some other combinations?). We can make a guess that the data we want will be sent on a “Notify” characteristic, since it’d be weird to spam a device with requests for frequently-changing data. Indeed, when we open Characteristic 4 and click “Listen for notifications” to subscribe to that characteristic, we immediately get a stream of interesting looking data. We’re on the right track.

I notice two things immediately. First, there are two different kinds of message coming in, a long one and a short one. Second, they have two different headers: 0x80 (binary 128) and 0x81 (129). I then notice a third thing: the longer messages sort of go all over the place, where the shorter ones have mostly the same data, or data that changes by a single bit at a time, and that only changes when the readings on-screen change.

Let’s take a sidebar quickly and look at the pulse oximeter screen and app to get an idea of what they might talk about:

We’ve got the SPO2 reading, pulse rate (in BPM), and the Perfusion Index (a measure of pulse strength, sort of). We’ve also got a line across the bottom called a plethysmograph – a graph of the relative blood volume in the fingertip at that instant. A visual measurement of pulse, really. What immediately jumps out at me is that SPO2 is between 0-100, which fits in one byte. Pulse rate is definitely within 0-255, so also one byte. And then perfusion index isn’t a medically useful measure anyway, more just an indication of whether you’ve got a good reading (as I understand it) – that only gets measured between 0 and 20. Or, (000 to 200)/10.0. So, also one byte, fixed point. Sure enough, if we look at the screenshot above and focus on the 0x81 messages: 0x81(37)(62)(6A). In decimal, that’s (55)(98)(106). That looks a whole lot like (PR)(SPO2)(PI). And at this point, I bet the 0x80 messages that seem to stream in continuously are points on that plethysmograph updating along the bottom of the app.

At this point, I could probably just assume I’m right and dive right into iPhone app development. But 1) I’m not registered at the moment, 2) it’s been a while since my last iOS app so the next incremental validation is probably a long way off, and 3) where’s the fun in that? A cool interim next step would be to see the actually decoded data streaming on my computer screen, rather than blobs of hex I have to manually convert.

Second Stop: Prototyping With Script

Ideally at this point, I’d like to use some kind of scripting language (non-compiled, anyway) to process incoming BLE data in real-time. Even if I did jump right to iOS development, there’s a compile/debug/repeat cycle that I’d like to avoid for the moment. But what language to choose?

Idea 1: Python

I really like Python. It’s fast, well-loved so there’s a library for everything, and I’ve already got a toolchain set up to use it on all of my machines, which include a Surface Pro 4 and a 2011 MacBook Pro 17″ I desperately need to replace once I can get more than 16GB of ram for my $3K. So anyway, cross-platform Windows/Mac compatibility is a plus. Unfortunately, the Windows 10 bluetooth stack apparently sucks, and therefore there basically aren’t any libraries to support it. I found the following options:

• Adafruit’s Python BluefruitLE library. Unfortunately, Windows support here is a casualty of its lateness to the party. Not until Windows 10 was support even plausible, so Windows wasn’t targeted, and therefore Windows support is unexpected. But worse, it seems that this library really only targets UART over BLE, which isn’t useful for this project. There does seem to be support for interacting with characteristics, but I couldn’t find any examples of subscribing to notifications or anything, so I gave up on this one.
• PyGATT (Github link). The first link there gives a thorough overview of the motivation of this library. Long story short, native Linux support was easy, but to build in Mac and Windows support, the author chose to go a different route, circumventing the operating system stack. BlueGiga supports a UART based interface for their BLE chipsets called BGAPI. By integrating one of these chips into a USB dongle that appears to the OS as a UART/TTY/COM port, you can talk BLE without having to worry about the OS at all. I bought the Silicon Labs BLED112 from DigiKey, for example. Of course, this is a bit of a bummer because who wants an extra dongle? But such is the price of easy cross-platform support, sometimes.

So now I had to wait for my BLED112 to show up. I kept looking for other options.

Idea 2: JavaScript and Web Bluetooth

While digging around, I came across Web Bluetooth, an API being driven by Google to support connections to BLE devices from the browser. I’d seen it before, but it had only beta support in Chrome on Mac, so I sort of left it there. Well, by now, it’s got main line support in Chrome on Mac, some level of support for Chrome on Linux, and… basically no support for Chrome on Windows. Whomp. And besides, I’m way more comfortable in Python than Javascript. But I still had days to kill before my dongle would arrive, so I opened up the Mac and dug around the examples.

It turns out, they’re super cool! This page of sample code lists lists demos of how to do a bunch of useful first steps with a device, like list device information and discover services and characteristics. It’s kinda cool to go to the “Device Info” sample, select “all devices” and check out how Chrome presents the UI for connecting to BLE devices. But that’s also not too interesting. Let’s go straight to the meat of it: the notifications sample. From LightBlue, we already know the service UUID as well as the characteristic UUID we want to subscribe to: cdeacb80-5235-4c07-8846-93a37ee6b86d (service) and cdeacb81-5235-4c07-8846-93a37ee6b86d (characteristic). Let’s plug those into the sample and see what we get when we hit “Start notifications.”

Awesome! Now we just need to clone this code locally, serve it on localhost, and modify to taste to process the data!

But I didn’t do that, for a couple reasons: I tried to install RVM to install the right version of Ruby on my Mac to get the Jekyll-using sample repo up and running, but ran into issues necessitating I upgrade my OS. At about the time I started this highly non-trivial process (I like to start with a fresh install when I can, and rebuild from there), my BLED112 dongle showed up in the mail, so I went back to working in Python.

Idea 1: Python (Again)

So Web Bluetooth proved too troublesome for my needs, but if you’re on Mac or Linux, I suggest you start there. I went back to my Windows machine, fired up a python virtualenv, installed PyGATT, and opened my favorite editor (Visual Studio Code). Annoyingly, PyGATT seems to suffer a timing issue on some Windows systems. The fix is quick, but you’ll probably need to install from git rather than PyPi. Rather than leave the adapter in an unknown state, PyGATT resets it at the beginning of each usage. But this causes it to eject and reenumerate in Windows, so the code needs to wait a minute before continuing. The issue is outlined here.  Anyway, my procedure (in Powershell, assuming you have Virtulaenvwrapper installed for python):

mkvirtualenv python-ble
git clone https://github.com/peplin/pygatt.git
cd pygatt
python setup.py develop
cd ../
mkdir python-ble-test
code .

At this point, I opened pygatt\pygatt\backends\bgapi\bgapi.py  and added “time.sleep(.25) ” on line 202 per the github issue above.

Back over in code, I open up spo2.py and get hacking. Here’s where I ultimately landed:

import pygatt
import logging
import os
import time
import binascii

# Uncomment this line to send log messages from the pygatt module to stdout
#logging.basicConfig(level=os.environ.get("LOGLEVEL", "INFO"))

# The BGAPI backend will attemt to auto-discover the serial device name of the
# attached BGAPI-compatible USB adapter.
adapter = pygatt.BGAPIBackend()

# Make an array to gather all the points from the 0x80 messages.
points = []

def handle_data(handle, value):
    """
    handle -- integer, characteristic read handle the data was received on
    value -- bytearray, the data returned in the notification
    """
    # Here's where we process message data we're subscribed to. Since I'm only poking around, I was only doing one thing at a time.
    
    # if value[0]==129:
    #     print("Received data: {} BPM:{} SPO2:{} PI:{}".format(binascii.hexlify(value), value[1], value[2], value[3]/10.0))

    if value[0]==128:
        for x in value[1:]:
            points.append(x)
        print("Received data: {}".format(' '.join(format(x, '08b') for x in value)))

try:
    # Find all devices named "My Oximeter"
    adapter.start()
    devices = adapter.filtered_scan(name_filter="My Oximeter")
    # Print to the screen the list of found devices, for a spot check.
    for device in devices:
        print "{} ({})".format(device["name"], device["address"])
    # Connect to the first device
    device = adapter.connect(devices[0]["address"])
    # This finds the Service UUID we're interested in (there's only one). But it seems unnecessary, as with PyGATT you just subscribe
    # to whatever characteristic you want. I haven't yet figured out how to find a service, then list it's characteristics.
    service = devices[0]['packet_data']['connectable_advertisement_packet']['incomplete_list_128-bit_service_class_uuids']

    # Subscribe to the characteristic UUID we got earlier using LightBlue, and call "handle_data" whenever a message comes in.
    device.subscribe("cdeacb81-5235-4c07-8846-93a37ee6b86d", callback=handle_data)
finally:
    # Since the subscription happens out of this flow, we need to wait doing something else. I suppose you could sleep forever, or
    # wait for some breaking condition like a keypress. But I'll just log for 15 seconds.
    time.sleep(15)
    adapter.stop()

# Print finished (mostly to have somewhere to break)
print "Finished."
# Spit out the list of plethysmograph points in a comma separated list to graph with Excel.
print ','.join(format(x, 'd') for x in points)

That code does two things, depending on what section you uncomment inside handle_data: it either processes the 0x81 messages assuming the format I guessed at, or it processes the 0x80 messages by adding every individual byte to a list of points to then plot later in Excel (to confirm that they’re pleth points). Running it with the 0x81 handler uncommented (opposite of above), we get the goods:

Bingo!

I’ll leave out the screenshot of the other processor running since it’s just a long comma separated list of ints spammed to the console. But what does it look like in Excel? I copied the output into test.csv, opened it in Excel, and plotted:

And there’s my second format suspicion confirmed! Fwiw, this means there’s no timing data or anything included in the message. I guess we just assume the samples are taken with deterministic timing. If that’s true, we could establish a timebase by collecting a large number of seconds of data, and measuring the number of points we got in that time. Points/s = total points/total seconds. But I haven’t bothered yet.

Next Steps

Well obviously, at this point, I sort of have to go write my iPhone app. There’s no avoiding that any longer. But as far as the RE is concerned, there’s still more going on here – 4 other characteristics I have no insight into. I can make some guesses: one may plausibly be a firmware update mechanism. There are some alarm settings on the device (alarm on/off, high and low limits on SPO2 and PR, etc), but I’m not sure if the app can read/write them, or if it just uses its own. Looking at it quickly, I think it’s the latter – app limits have nothing to do with on-device alarms. Sadly, since I’ve got what I need, there’s almost no way I ever bother to figure out what the other characteristics do.

Wait, what about the scale?

Yeah, I never really did dive in, did I? It’s basically totally the same. There’s a service with a notify characteristic that’s very clearly the weight coming through. A couple of bytes are the weight reading, and one byte indicates the “units” setting on the scale (grams, lbs-oz, fluid oz, and fluid oz of milk). One thing I found interesting, the scale knows when someone is subscribed to that characteristic, and indicates that on-screen by a solid bluetooth icon. When connected, the behavior is that the scale gets a settled reading, beeps, flashes the value 3 times, THEN sends the notification message with weight. If it’s not connected, no beep and flash. And annoyingly, since it sends the message AFTER the 3 screen flashes, it’s pretty slow to update the phone and makes you think the app isn’t working right. Bad design.

But maybe you’ll see more about that one later!