Saturday, July 18, 2009

Searching for a Calorie Graph

Kenneth did some academic paper research and found that our EE column = Energy Expenditure which he thought was the crude calorie calculation on the device.

I've plotted EE for an ~12 hr period in the top half of the graph above, and in the lower half is the calorie graph from the GoWearFit web service. I'd say Kenneth is right!

The web service obviously does slightly different processing. I'm also not sure of the units for EE - the values range from ~113 - 1050, whereas the calorie chart range from ~1.5 - 7.5 calories-per-minute. Maybe it is just a divide-by-100 and the more sophisticated web service processing is simply that much more accurate, but I'm not sure. A summation over EE/100 gives a much larger calorie count than the web service.

35 comments:

  1. It could be a fixed point value?
    Perhaps, 7 bit whole and 5 bit fractional?
    Dividing 113 and 1050 by 128 gives a range of about 0.88 to 8.20.

    ReplyDelete
  2. I do believe the EE numbers stored in the armband are a rough estimate.

    Then once the data is uploaded to the web site they are processed to be slightly more accurate.

    The reason I say this is the bodybugg watch will show the estimated numbers, but once the data is uploaded there is a slight discrepancy between the watch readings and the numbers displayed on the website.

    ReplyDelete
  3. Actually I had a point. It was...

    The upper graph and lower graph won't be exactly the same because one is, I believe, more precise than the other.

    The graph generated by the data directly retrieved from the armband will be a bit less accurate than the graph displayed on the website.

    ReplyDelete
  4. I am a complete newb with python. I have the script and python installed with the numpy rc2 version installed but when I try and run the module it says there isn't a serial module. did I miss something I had to install? Is that a standard library?

    ReplyDelete
  5. edit, ok I was trying to mess with this on a computer that doesn't have any of the body media software installed. I imagine that is my problem.

    ReplyDelete
  6. I've got bodymedia s/w installed and it still doesn't find 'serial' -- can anyone help?

    ReplyDelete
  7. Your MediaFire account is empty. Could you re-upload your tools?

    ReplyDelete
  8. @Arkiel the download works for me. Is anyone else having trouble??

    Sorry for delay, I've been away for over a month.

    ReplyDelete
  9. Downloaded worked for me.

    Folks,

    I have had some success getting my Bodybugg v2 to work under Mac OSX 10.6 (Snow Leapord). I used the following software compenents (in order of install):

    FTDIUSBSerialDriver_v2_2_14.dmg
    python-2.6.3-macosx.dmg
    numpy-1.3.0-py2.6-macosx10.5.dmg

    The FTDI driver doesn't need any modifications to as it reconizes the PID of the BBv2.

    I am using bmhack-z718.py; I can get the cpickle dump; but the csv output gives this error:

    python2.6 bmhack-z718b.py --fromSerial=/dev/tty.usbserial-0000201A --toCsv=bb102109.csv

    ...

    assert RecPack.has_key(id), "Unknown record ID %i at offset %i" % (id, offset)
    AssertionError: Unknown record ID 0 at offset 675

    would appreciate any advice to get this to csv output.

    Thanks,
    J

    ReplyDelete
  10. opps. Forgot about the pyserial install:

    FTDIUSBSerialDriver_v2_2_14.dmg
    python-2.6.3-macosx.dmg
    pyserial-2.5-rc1 - python setup.py install
    numpy-1.3.0-py2.6-macosx10.5.dmg

    ReplyDelete
  11. Jayme, I'm not sure what the problem is. Have you tried it with data on the device? It might be running into a zero record if it is cleared (although it should detect that).

    If you e-mail me a cpickle dump I can try to diagnose it. centibenzo@gmail.com

    ReplyDelete
  12. Finnerty,

    My gmail has had a couple hickups; I wanted to make sure you received the dumps I sent you.

    Thanks,
    Jayme

    ReplyDelete
  13. I'm curious, has there been any further progress on this?

    ReplyDelete
  14. I'm trying to get my GoWearFit to work with this under OS X 10.6 and the same software as Jayme above, but it keeps failing. Manual experimentation shows that, if I try to open the serial port at baudrates lower than about 400,000, the serial port will open (the handle passed back from serial.Serial() shows open=True after an open() call).

    But, at 921600, or any high baudrate, calls to open() return error-free, but the handle shows open=False, and there's no other diagnostic info to be had.

    Using GoWearFit's website to sync the armband works just fine on the same Mac.

    Ideas?

    ReplyDelete
  15. Hm. It seems that on OS X, since Tiger, termios doesn't support anything higher than 230400 baud. And trying to communicate with the device at that speed via pyserial fails to yield any data.

    ReplyDelete
  16. As far as I know, the 921,600 baud rate is an arbitrary requirement of the Windows FTDI virtual serial driver. I did discover that no other baud rate seemed to work for me under Linux or Windows (this was a major stumbling block). I found that speed by system call trace during a device transfer.

    It might not apply to a Mac, or might be something different.

    ReplyDelete
  17. Any chance for a .NET port? I was going to attempt it myself, but then realized that I don't understand 1/2 of the Python code (0 Python skills). I figure I have to understand completely in order to duplicate functionality. Any takers?

    ReplyDelete
  18. Regarding the discrepancy between EE values and "real" values from the site... if you can get me data like the data you have posted (but add "real" values), I can reverse engineer the calculation to come up with it.

    ReplyDelete
  19. Fantastic code... works great under Ubuntu 9.04's live CD. 9.10 is available here (and I'm sure if you hunt, you'll find 9.04, too):
    http://www.ubuntu.com/GetUbuntu/download

    For those who are linux n00bs (like me) here's the set of commands I ran from the live CD to get the bodybugg data - and then clear it:
    operations:
    become root
    get python support libraries (numpy, imaging, serial)
    get linux to ID the bodybugg (make sure it's connected first!)
    dump the bodybugg data to dump.csv
    clear the bodybugg memory

    commands:
    sudo -i
    apt-get install python-numpy
    apt-get install python-imaging
    apt-get install python-serial
    modprobe ftdi_sio vendor=0x11f8 product=0x0007
    python bmhack-z718b.py --fromSerial=/dev/ttyUSB0 --toCsv=dump.csv
    python bmhack-z718b.py --fromSerial=/dev/ttyUSB0 --clear


    For those working under windows and getting "Access Denied" when trying to attach to a COM port, CB's code is missing a Windows-specific line. No fault against CB -- he did a *phenomenal* job with this!!

    in bmhack-x718b.py:
    add a line before line 88 from:
    def OpenSerial(fname="/dev/ttyUSB0"):
    ser=serial.Serial(fname,baudrate=921600,timeout=.01)
    ser.open()

    to:
    def OpenSerial(fname="/dev/ttyUSB0"):
    ser=serial.Serial(fname,baudrate=921600,timeout=.01)
    ser.close()
    ser.open()

    All I did was add ser.close() right before the ser.open(). It's a Windows quirk but it makes the script work like a charm!

    so then all you need to do is put bmhack-z718b.py in the c:\python26 folder and (from a command prompt) run:
    python bmhack-z718b.py --fromSerial=COM3 --clear

    Note that you'll need to try other COM ports if your system has more than 1 comm (serial) port. Check the device manager (run 'devmgmt.msc') under 'ports' to see what COM port your bodybugg is hooked up to. Could be COM3, COM4, COM5, etc.

    In windows, you'll have to download & install python 2.6.4 (http://www.python.org/download/), PIL - python image library (http://www.pythonware.com/products/pil/), numpy (http://sourceforge.net/projects/numpy/files/) and pyserial - python serial (http://pyserial.sourceforge.net/pyserial.html#installation).

    ReplyDelete
  20. to add abit to this conversation.

    to convert EE to calories is to divide by 128

    ReplyDelete
  21. Where did this blog go? I would really like to get a BodyBugg but I refuse to pay a subscription for a device that's already costly and while these efforts look amazing they seem to just dead-end in 2009...

    Did something happen or did the original creators just lose interest?

    ReplyDelete
  22. Hey! I run http://www.openyou.org, and I'm trying to pull together resources for as many development libraries as I can right now. I was thinking I might make a github repo out of your python code for the bodybugg? Or is there another repo available somewhere that you're updating? Please feel free to throw me email at kyle at openyou dot org.

    ReplyDelete
  23. Nice work on this OP (or OB?). The python code works great. Has anyone done any more work on decoding the data. I'd like to figure out what is used to calculate sleep efficiency.

    As far as the EE to calorie formula, it looks to me like the divisor should be ~120 (i.e. EE/120 for the on body minutes gives me the same total calories for the day as the bodybugg website if I use the following for off body time:
    if 11PM < time < 6AM, use 1.5cal/min
    if 6AM > time > 11PM, use 1.6cal/min

    ReplyDelete
  24. I think it takes into account your age and weight on the website. But I'm guessing the device probably doesn't. If someone could reverse engineer the formula that'd be awesome.

    ReplyDelete
  25. I just finished downloading and clearing data for my girlfriend using these wonderful scripts. We've been using the software posted here for nearly a year now in Ubuntu (9.04 for reference).

    My girlfriend did buy the Bodybugg digital display to use in conjunction with the data pulled from these scripts, and we clear the data using the scripts as well. After these many months the clock has begun to drift a few minutes off (as its not synced during the data download), but that's a small price to pay.

    Thank you to everyone who contributed to this. We are a very happy household because of it!

    ReplyDelete
  26. Is this blog still active or do people know of other sites / blogs focused further on a subscription free alternative for the hardware? Would be interested in seeing what visualizations people have come up with hte outputted data and the formulas to produce them.

    Thanks for all the great work. It is very appreciated.

    ReplyDelete
  27. I just got a second-hand bodybugg from my wife and trying to dump data to .csv file under Linux. It does not work for me. I can dump to .hex or .cpickle, but I can't dump to csv, nor can I convert from .cpickle to .csv. Anyone have any ideas?

    # ./bmhack-z718b.py --fromSerial=/dev/ttyUSB0 --toCsv=dump.csv
    THIS CODE IS DECLARED BY THE AUTHOR TO BE IN THE PUBLIC DOMAIN.
    NO WARRANTY EXPRESSED OR IMPLIED OF ANY KIND IS PROVIDED.
    ....Traceback (most recent call last):
    File "./bmhack-z718b.py", line 1179, in
    sys.exit(main())
    File "./bmhack-z718b.py", line 1101, in main
    packets, mem = MemoryDump(ser)
    File "./bmhack-z718b.py", line 146, in MemoryDump
    packets.append(WriteAndReadSerialPacket(ser, pp))
    File "./bmhack-z718b.py", line 127, in WriteAndReadSerialPacket
    return ReadSerial(ser, minLen=n*66, maxLen=n*66)
    File "./bmhack-z718b.py", line 115, in ReadSerial
    raise Exception("ReadSerial timeout after %f sec, expected %i bytes, got %i bytes: %s" % (time.time() - t0, minLen, len(s), s))
    Exception: ReadSerial timeout after 3.005803 sec, expected 13200 bytes, got 0 bytes:

    ReplyDelete
    Replies
    1. Try upping the Timeout from 3 to 10 in the script at this point:
      >> def ReadSerial(ser, minLen, timeout=3.0, maxLen=2**15):

      It worked for me. Good luck!

      Delete
  28. Is there a update on the code? Or any type of update to get this to work better?

    ReplyDelete
  29. Nice work! I have a bodymedia fit link. It is the latest version of the bodybugg. is there a possibillity to use the Fit Link with freethebug?

    thx a lot

    ReplyDelete
  30. I'm looking for a way to still use my old BodyMedia device which is currently useless since Jawbone bought the company and turned it off.

    ReplyDelete
    Replies
    1. I'm working on a graphical display of the data on a web server. All output and code interpretation will be PHP/HTML5 with the data being stored to a MySQLdb. I can share most of the code with you when I get it completed.

      Delete
    2. Hi cl, any update on your progress?

      Delete
  31. I like the helpful information you provide in your articles. I'll bookmark your weblog and check again here frequently. I am quite sure I'll learn plenty of new stuff right here! Best of luck for the next!

    ReplyDelete
  32. Hey there just wanted to give you a quick heads up and let you know a few of the pictures aren't loading properly. I'm not sure why but I think its a linking issue. I've tried it in two different web browsers and both show the same results.

    ReplyDelete