Monday, July 6, 2009



BodyMedia Reverse Engineering

centibenzo @t gmail d.t com

What

I am documenting this work in the hopes that it spurs BodyMedia to create a
Linux driver, or a third party to develop a similar Open Source Linux application.

A partial reverse engineering of the BodyMedia ArmBand Mini device protocol was accomplished.

A Python library was written to communicate with the device, and to parse the data.
This library is compatible with both Linux and Windows (barely tested on Windows).

There is a lot of work to do before this system becomes useful.

Accomplished



One night of RAW graphed data records, left-to-right, showing brief
activity before sleep, quiet, and more activity after waking.


  • Correct Linux driver and modprobe line discovered to make device function as a USB Serial port.

  • Basic structure of serial protocol decoded.

  • Memory dumps from device can be obtained by serial protocol.

  • Most data structures in memory, including data tables, can be parsed.

  • Crude "activity graph" can be made from the raw data tables which does show activity over time
To do

  • Find and interpret data timestamps (presumably in each data table header)

  • Identification of type and size of individual data record fields

  • Identification of actual MEANING of individual data record fields

  • Development of heuristics to translate data into indentification of sleep, active, etc (Ph.D thesis)

Discoveries

  • Device uses an FTDI USB<->Serial chip, compatible with Linux ftdi_sci driver.

  • Under Windows, system uses a stock FTDI driver with changed Vendor and Product
    ID's to present a virtual com port to Java(?) software integrated with Web
    Browser.

  • Protocol is far more complex than the USB/Serial application would
    require. It is geared for the wireless RF link (sync bytes, etc). I suspect
    it derives from some RF vendor protocol.

  • ??? Device logs a data record only once per minute (?). A little disappointing - perhaps the clinical models log more. Or maybe it is variable rate???

  • ??? Device has 200 kByte of memory for data logging???



Platform

  • Device:

  • Ubuntu Linux 9.04

  • Windows XP

Usage

Python Code: SEE NEWER VERSION ABOVE
Command List: SEE NEWER VERSION ABOVE
(Command list must be renamed to known_requests.cpickle for FullSerialDump() function.)

Linux: How to force the ftdi_sci driver to recognize the BodyMedia Vendor/Product IDs:


sudo modprobe ftdi_sio vendor=0x11f8 product=0x0007


Verify that a new '/dev/ttyUSB*' device has been created.

Windows: You don't need to install an FTDI driver if you already have the
GoWearSense driver installed. When you plug in the device, you will see a new
COM14 (or similar) port appear which you can use within Python.

Examples of how to use the python library.


import bodylib as bl
import cPickle

# Open, query, and close the device
packets=bl.FullSerialDump(serialName="/dev/ttyUSB0")

# Let me print all the packets sent and received
bl.PrintPacket2(packets)

# Parse packets and print all data in appropriate format in text tab-delimited format to file
# This is a good human and spreadsheet readable dump of most of the device state.
bl.SaveStructTabDelim(packets, "/tmp/foobar.txt")

# If filename not specified, outputs to stdout for your inspection
bl.SaveStructTabDelim(packets)

# I usually save my device dumps in cPickle format for future reference
cPickle.dump(packets,open("mydump.cpickle","w"))

# Assemble the received data packets into a unified memory buffer
mem=bl.AssembleDataFromPackets(packets)

# Pretty Print the memory buffer. We use 46 bytes/line to be periodic with the most common data record
bl.HexPrintMod(mem, 46)

# Parse memory data structures, including data records
tables=bl.ReadAllStruct(mem)

# Create image from data table
bl.ToImage(tables.s4).show()



Protocol Structure

Serial baud rate must be set to 921,600 bps.

The protocol is packet based, with packets of length 66 bytes including headers
and sync bytes.

The computer sends a Request packet ("Req" in my code), to which the device
responds with one or more answer ("Ans") packets.

All request types expect only a single "Ans" answer packet, except for requests
to read from the data memory. Data memory requests specify a starting memory
offset and a length, and receive as many 66-byte packets as required in
response. I call this "Burst" mode in the code.

All packets have a simple modulus 256 checksum byte. This is a simple sum and
modulus of all bytes in the packet, excluding sync bytes (leading or tailing
"AB" or "BA").

The device appears to ignore the checksums sent to it (I don't bother computing
it). However, the device always gives me correct checksums.

The packet begins with a leading sync byte 0xAB. Then a rather elaborate header
almost all of which is unused or invariant (RF protocol related?). Most
importantly, a "command" byte defines the type of request. There is also some
form of sequence number byte on Req packets, but not device-generated Ans
packets. The sequence number appears to be ignored by the device.

The "command" byte specifies the requestion type, and is the key element of the
header. Note that in my code, I call this the "Bank" byte, because I initially
thought it was a device selection address.

After the command and (sequence number if Req), the packet payload contains
command-specific data.

The command byte to read from the main data memory is 0x82. The command byte
for the Answer packets from the device is 0x02 (in my code I stip the high bit
from the command/bank for clarity, so you may see 0x02 Requests).

The payload to read from the main data memory consists of a 4-byte LSB offset
uint32 and a 2-byte LSB length uint16.

In addition to the block of main data memory, there are a number of other
values which can be queried using other command types. These usually return
short strings or single binary values. Examine the packet lists to find some
examples.

Tools

"Free Serial Port Monitor" by HDD Software is GREAT - you can use this to snoop
on the virtual serial port under Windows, and export the conversation to a file.

Under Python I've created a parser for the "Free Serial Port Monitor" text
export of the "RequestView" window. See bodylib.ParseFile().

Debug FS is very useful for monitoring USB traffic under Linux.

sudo mount -t debugfs none_debugs /sys/kernel/debug
cd /sys/kernel/debug/usbmon/
cat 0u


I use ipython, Python 2.6.2, numpy, PIL python libraries. If you install on
Windows you'll have to fetch each of them, or comment them out of the bodylib library.

See Also http://bodybugghacks.blogspot.com/
(apparently an attack at the Java API level, but the author is not forthcoming on details)

Units of Measure - from http://bodybugghacks.blogspot.com/2008/07/code.html?showComment=1225132980000#c1498831675573385775

Useful to determine figure out field structure?

Accelerometers - The SenseWear® WMS Armband (2-axis) - The SenseWear® WMS Armband Mini (3-axis) Calibrated range is +/- 2.0g
The minimum resolution is 0.01g
Two-standard-deviation accuracy of +/-0.05g (longitudinal axis) and +/-0.06g (transverse & forward axis), up to 1.0g
Two-standard-deviation accuracy of +/-12.0% of expected value otherwise
Heat Flux
Calibrated Range is 0.0 W/m2 to 300.0W/m2
A minimum resolution of 1.0W/m2
Two-standard-deviation of +/-10.0W/m2 at heat flux less than 50.0W/m2
Two-standard-deviation of +/-35.0% of expected value otherwise
Galvanic Skin Response
Calibrated range is 56.0KΩ to 20.0MΩ (50.0 nSiemens – 17.00 uSiemens)
Two-standard-deviation accuracy of +/- 7.0 nSiemens up to 233.34 nSiemens reading
Two-standard-deviation accuracy of +/- 3.0% of expected value otherwise
Skin Temperature
Calibrated range is 20.0ºC to 40.0ºC
A minimum resolution of 0.05ºC
Two-standard deviation accuracy of +/- 0.80°C
Classification of the device, as per 93/42 directives: IIa (rule 10)
Certification procedure : 93/42/EEC, Annex VI, VII.
Transmit Power Class 8 - Less than 10mW output power
Duty Cycle Class 4 - permitted to operate at 100% duty cycle
Receiver Class 3 - Standard reliable SRD communication media

----

20 comments:

  1. I'm pretty sure the device logs data more than once per minute. Based on GoWear's marketing docs and their whitepapers, the device uploads all the detailed sensor data (accelerometer, heat flux, GSR, skin temp) to their website for more detailed analysis, while the device itself contains only a simplified version of the calorie algorithm. This would make sense to me, since it's to their advantage to collect as much raw data as they can from their users for more analysis. (And it's consistent with what Henry at the BodyBugg Hacks blog found with version 1 of the device.) So I think there's still more to find.

    Question: Are you signed up for a service plan, and you're snooping the USB data when it's connected to the website? If so, which service are you using--bodybugg's or GoWear's? I'm thinking about getting a GoWear for the sole purpose of getting the raw data (I don't care so much about the calorie data), but I'm trying to get a better idea of what people have tried already and how likely it is that this will be successful.

    Great to see some new efforts on this!

    ReplyDelete
  2. Hi Kenneth,

    I'm using the GoWear service. It is a good service - I use my GoWearFit primarily to monitor sleep, and the service does a fantastic job (although more downloadable data and better reports would be nice).

    I snooped the initial data while the GoWear service ran to get an initial sample of the protocol, using "Free Serial Port Monitor" by HDD Software (see above).

    Now the Python library is fully capable of obtaining (at least) the data memory from the device itself, under Windows or Linux, without snooping or any BodyMedia software.

    As for data logging once per minute - I can't claim that as fact, but it seems to be about what I am seeing in the very few tests I've done so far. I only see 200kbytes of memory being retrieved, with 46 byte records - but I have not probed around much yet.

    I actually suspect data records may be saved at a variable rate, but that is only conjecture.

    I'm sure you are right that almost all of the algorithmic work is being done server side. The analysis of that data is a Ph.D thesis of work. BodyMedia has little to fear from people pulling raw data.

    ReplyDelete
  3. BTW - If anyone wants some samples of captured data to play with, e-mail me. centibenzo @t gmail d.t com

    ReplyDelete
  4. Note I've just made some processed tab-delimited sample data available.

    http://bodybugglinux.blogspot.com/2009/07/sample-data.html

    ReplyDelete
  5. you could have a look at this:

    http://www.sics.se/~annas/inner/files/

    an older windows stand alone version (4.1) of the software used to read the data off the device.
    A newer stand alone version (6.1) that I happen to have, has 30 days trial timer although it doesnt stop afterward, only limits the kind of data available.

    ReplyDelete
  6. I have a java application for windows that I wrote that uses the windows dll (bmusbapex4.dll) to communicate with the device. It has the ability to download all the datapoints as well as manage all the configuration parameters of the unit.

    if anybody wants it.

    ReplyDelete
    Replies
    1. can you send me the application please?

      Delete
    2. I would appreciate a copy. Thanks, Paul

      Delete
  7. for anyone wanting to try my utility to manage the data on the bodybugg, please send email to

    FreeTheBugg@gmail.com

    and I will give you access to my google docs repository

    ReplyDelete
  8. I can't seem to get this working..
    Traceback (most recent call last):
    File "C:\Python26\bmhack-z718b.py", line 33, in
    from numpy import array, ndarray, fromstring, zeros, resize
    ImportError: No module named numpy

    Any help?

    ReplyDelete
  9. download my current bodybugg application and drivers

    https://docs.google.com/leaf?id=0B2o6Z-umHv-hNTQwYTUwZDMtNzJiYS00OGYwLTljZDctMjIwMGY1ZjFhNGQ3&hl=en

    https://docs.google.com/fileview?id=0B2o6Z-umHv-hYTlhZjAwNDUtOTBiMS00NDgxLTllMzYtNThjMzM0MWJkYWI5&hl=en

    https://docs.google.com/leaf?id=0B2o6Z-umHv-hNWE3NGM4ODAtNzM2NS00ZjZjLWE4OGUtNDIyYTIyNWFlYjUx&hl=en

    ReplyDelete
  10. We try to get the software working with a newly purchased Armband Pro2. After using vendor=0x11f8 product=0x0005 we got the device /dev/ttyUSB0. (note that there seems to be a change in the product id!)
    When using the bmhack-z718b.py software, we get an assertion error:
    File "bmhack-z718b.py", line 1179, in
    sys.exit(main())
    File "bmhack-z718b.py", line 1120, in main
    SaveStructTabDelim3(packets, dopts['--toCsv'])
    File "bmhack-z718b.py", line 983, in SaveStructTabDelim3
    fields, records = RecordTable(packets)
    File "bmhack-z718b.py", line 951, in RecordTable
    assert len(recs) > 3, "No sensor data is currently on the device (or dump)"
    AssertionError: No sensor data is currently on the device (or dump)

    We know for sure that the device has data on it.

    Does anybody know whether there is a new and incompatible product line? and if there is the possibility to retrieve the data from this particular device type?

    ReplyDelete
  11. Hi Guys
    I have a KiFit armband. Basically bodymedia bug but in the UK. Was astonished (and jealous of their business model). Found your hack. Very impressed
    loaded python-2.6.6,exe
    numpy-1.5.0rc1-win32-superpack-python2.6
    pyserial-2.5.win32.exe
    PIL-1.1.7.win32-py2.6
    in that order. Just followed default instructions. All very painless.

    used activity manager to identify the com port. Is com4 on my laptop.

    Had to edit the bmhack file to include a ser.close() after the first ser.open() in def OpenSerial( or I got an access denied error

    see above

    The following writes a csv file
    c:\python26\python.exe bmhack-z718b.py --fromSerial=COM4 --toCsv=testal290810.csv

    Checked with my wife's display and EE field is almost identical to the calories burned.

    I have written a .net application to automate the extraction and collation of the data. The charts it produces look very very similar to KiFits activity manager. I have also started building in a meal logging system which is pretty straight forward. I already had a prototype in Excel.

    Its a pity you have to do all this. The armband is I think an excellent idea. The website should be free and they should look to add value some other way.

    Thanks again for all your hard work
    Steve

    ReplyDelete
  12. is anyon still reading here.. i would love a program to access mt bodybugg can anyone plz reply and let me know if this exists.. thanks
    cheryl

    ReplyDelete
  13. WHile using

    the command to get the serial driver to work on Ubuntu Natty is with a BodyBuggSP is

    sudo modprobe ftdi_sio vendor=0x11f8 product=0x0010

    Notice the "10" not "07" at the end

    and the result was a serial port known as

    /dev/ttyACM0

    bmhack-z718b.py seems to work well with this combo.

    ReplyDelete
  14. Has anyone tried hacking the Radio Frequency? I have a bodymedia sensor and am interested in knowing if the RF is hackable

    ReplyDelete
  15. For anybody still looking at this, there is a polished app that does this and everything it doesn't do. Google up freethebugg, and you'll need this for the drivers: http://downloads.bodymedia.com/BodyMedia_Activity_Manager_JRE_20090715.exe

    ReplyDelete
  16. Just wanted to thank Anon from Mar 18 2012; I couldn't get freethebugg to download data because I was missing these drivers.

    ReplyDelete
  17. Does anybody have a Tool to read data from the KiFit Armband (Bodmedia Core Fit)?
    In CSV or other would be sufficient to see the calories burned...

    ReplyDelete