While I was at QS 15, I heard that Fitbit had made their data export free. I have been using Zenobase for a while now to store and export my hourly Fitbit data, but I was interested to see what FitBit was now offering their users. I am not sure whether this new feature was added in connection with the recent Fitbit IPO or in response to customer complaining about lack of support for syncing to the iOS Health app. Previously, Fitbit charged customers $50/year to export their data and this was the source of many gripes from customers. Whatever their reasoning, I commend Fitbit for their simple statement that “Your data belongs to you!” (Better late than never.)
If you are looking for the Fitbit data export options, you can find them through your Fitbit dashboard page. When you log in to your online account from the web, locate the cog icon on the top toolbar, click on it, and choose Settings. When the settings page comes up, choose the Data Export section.
As you can see, you can select a time period and what kind of data you wish to export. I was disappointed not to see heart rate data as an option, but perhaps it is coming.
Since I have been using my Fitbit Charge HR since the end of March, I selected Custom for my Time period and choose all the various Data types to see the structure of the exported data. This was when I ran into my first issue, which turned out to be a known Fitbit bug. I used the calendar widget to choose March 27, 2015 through July 2, 2015 as my time period, but when my data exported, it only contained data from March 27 through April 26.
I searched the Fitbit forums and found a few different threads on this issue. A customer had posted the helpful observation that the calendar widget was inserting the wrong date format, and his work-around was to enter the date directly in the correct format (MM/DD/YYYY), which worked for him. Later users mentioned that this helped them, but more recent comments indicated that this broke again, perhaps when Fitbit recently added new tiles. I followed all the suggestions, but it didn’t work for me in Chrome or IE.
Before deciding if I wanted to go through the hassle of combining monthly data or wait for them to fix this bug, I decided to look at the output for combined data export for a single month. Unfortunately, when including all supported data types, the Excel export option created a bunch of separate worksheets in a single workbook and the CSV formatted output dumped various differently formatted tables into a single document. I was hoping to avoid needing to parse this output with regex so exporting types separately seemed the most sensible course of action.
To get my first month of Activity data, I entered my date range manually to begin on March 28 and chose Activities under the Data section. Here is the first week of the data I exported as CSV.
So far, so good. I also exported the same data as XLS, and thought it was a bit odd that the dates in the CSV export showed up as MM/DD/YYYY and the date format in the XLS export showed up as YYYY-MM-DD. Even trying the dashed date format for XLS export in the dialog didn’t get the software to export a longer time frame.
To avoid further frustration, I decided to give up on trying to export more than a month at a time for now. If you are a Fitbit user, you can search for data export in the forum to add your own comments and likes to Fitbit threads about this issue. Basically at the moment you are left to export one month at a time and combine your data, or choose a third party provider like Fitabase or Zenobase that will let you export all your data at the same time.
I also exported my sleep data using the Fitbit data export. Again, I only got a month of my data and it was pre-summarized to include information about the minutes I slept, minutes I was awake, and total time in bed. I currently get sleep segment information from Zenobase, but this is useful pre-summarized additional data that I may want to export and track in the future.
One advantage to this format is that it summarizes data for a specific night of sleep. I have run into issues with the correct night of sleep being assigned to a segment depending on where it falls relative to the midnight border, however, and it’s not yet clear to me if this issue also exists here.
I don’t currently log my foods in Fitbit’s software. After leaving BodyMedia’s food logging interface and exploring MyFitnessPal, I liked the idea of trading up to a database with a broader selection of food items that either Bodymedia or Fitbit offered. Since I had logged food data for a few days in Fitbit as I evaluated the two side by side, I exported my Food data to see what Fitbit’s format looked like.
After I swtiched to using MyFitnessPal, I had set it up to export summary data to Fitbit. Those days looked like this.
The CSV Food output included a single table with calorie summary data followed by the table listing details of foods eaten and days with imported summary data from MFP. I found this format to be less useful than it could have been. To make the food log part of the table more analysis-ready, it should instead contain a date column filled in for all rows, a separate column labeled Meal with the name listed in every row, and no partially blank lines.
As an example of what I mean, here is the table format I created from a report in similar format that I exported from Bodymedia’s software.
The XLS format separated different kinds of information into tabs, with summary data on the first tab and detailed food log information for specific dates on later tabs. It would take some work to get this processed into an analysis-ready format.
If I was going to use this detailed information, I would definitely restructure it in JMP to match my standard format. However, since I already get this information from exports of my MyFitnessPal data, I don’t need to do that at the moment. Perhaps I will save my suggestions on reformatting this data this for a future JMP blog.
I thought I had set up my Withings wireless scale to sync to Fitbit, but I noticed that this doesn’t appear to be working. For that reason, only my starting weight shows in the Body export. I am going to need to fix that or import my daily weight data data directly from Withings or MyFitnessPal (whose sync with Withings does appear to be working for me).
All in all, I’m a bit disappointed by Fitbit’s data export options. The bug with the Custom date range that would force me to export monthly data is especially annoying, since I had to deal with monthly export of many files (~50 in the end) when using Bodymedia’s native export tools and I was very glad to leave that behind when I found Zenobase. I much prefer being able to pull my whole table whenever I want to without needing to write any additional code or concatenate files manually, so I will stick with Zenobase.
However, exploring my Fitbit export options wasn’t a waste of my time as the Sleep data offers some additional useful summary information. I’ll probably work on exporting that data for use at some point.
weighthistory 
Thank you for an excellent summary of FitBit’s data export short-comings. I have been struggling with the same issues and cannot understand why a large company like Fitbit would not fix this issue. To shackle users with a 30-day limit on each export is pointless. I think the best one can hope for is to provide negative feedback on review sites (e.g. Amazon) and hope that the company can be coerced into fixing this irritating issue. So far, Fitbit has been reluctant to address users’ complaints on their support forum.
LikeLike
Hi Bob, unfortunately this is a pretty common experience with tracking gadgets/software that the data export isn’t where you want it to be. Whether it’s a bug like this one (which they say they will fix soon) or exporting data only partially by design, without a data sharing standard in place, you never know what you’re going to get. I have a few other devices I have been using recently (see my most recent blog on the Skulpt Aim and also various apps for measuring HRV) which totally lack data export features so you’re left to do manual data input. I still find the tools highly useful but the process is just not as automated as it should be. I actually wrote a script to import all my BodyMedia data into JMP a year and a half ago because BodyMedia only allowed me to export my 4+ years of data in 28 day chunks. What a pain! The BodyMedia food log data was even more frustrating-PDFs that I had to save to text and use regex parsing to get them formatted correctly! I blogged about those projects last year on jmp.com/blog (search Fitness and Food there). Getting data out of MyFitnessPal is relatively straightforward and I have a blog about that too. Data export and data ownership his is a big focus for QSLabs, so you might check out their site for more info. I have been really lucky to meet several people from that small company and attended their conference in June. They are passionate about this topic and I’m making a point to join in by requesting features from software I use that doesn’t export in a straightforward way. Thanks for your comment!
LikeLike
Hi Shannon, I find it frustrating that a company does not routinely purse excellence or common sense in its business. As a case in point, the FitBit export feature has a bug in the date format. You pointed this out above and I experienced the same difficulty. FitBit is well aware of this bug through the many complaints on their forum. Both you and I know that this could be fixed in 10 minutes with a few lines of re-written code, yet FitBit has failed to act for many months on this known issue. The limitation of data-export to 30-day chunks is another example of an easily fixable problem. Should I be writing Perl scripts or VBA macros to handle this because FitBit will not address this issue? Seems lame to me. I did notice, however, that FitBit’s webpage now says “Each export file can contain 30 days worth of data”, Perhaps they have succumbed to the old trick of claiming a bug to be a ‘feature’? A third example is the FitBit iPhone app — when I plot my weight for the year, the weight-axis is scaled from 0 to 180. So all of the data looks like a straight line clustered at the top of the graph. With a little bit of thought, the FitBit developers could easily have ignored zero-values in the data and then auto-scaled the axis to be something reasonable. I think the FitBit hardware is outstanding. I have been impressed with the FitBit Zip and HR. The company really flubbed it, though, on the software side. In the grander scheme of things, these are minor irritants. But then, again, so is that grain of sand in your shoe that seems to grow bigger and bigger as you walk.
LikeLiked by 1 person
I agree, Bob, the whole situation with FitBit export is totally lame. I dealt with a similar limitation when exporting/scripting import of my 4+ years of BodyMedia data into JMP (see early entries in my Fitness and Food series on the JMP blog) and my patience with stupid data export limits is gone. I use Zenobase, an aggregation database developed by Eric Jain, which allows me hourly data info instead of the daily info FitBit provides. It’s a tiny fee ($5/mo) and IMHO well worth it.
LikeLike
I’m with you on the hardware. I have been very happy with my Charge HR and the data it collects for the most part. The whole “data export is now free” seemed very promising at first, but I’m thinking it wasn’t well thought out. I don’t know what kind of software update schedule they are on, but it seems clear they’re not doing continuous deployment so we’re at the mercy of whatever their deployment schedule is. Hopefully the 30 day limitation and date issues will be resolved in a larger update that deals with additional bugs. In the mean time, there are plenty of work-arounds, including signing up for a free account with Zenobase where you can get access to your hourly FitBit data. This is a big improvement over the daily date. I now pay $5/month for that software (developed by Eric Jain) as a data aggregation tool and it’s totally worth it to me. I was not a FitBit user until after I’d already discovered Zenobase, and I was used to paying BodyMedia for their software so the small cost seemed justified for the extra resolution.
LikeLike
They are just being obstructive because the data is there. I too was looking to get my weight data (going back to 2012) saw the 30 day limit, groaned, googles and found your blog. Anyway I saw how the dashboard was working and i’ve figured out a workaround, you can use the api directly.:-
https://www.fitbit.com/weight?apiFormat=xml&pages=200&pageNumber=0&viewRecentMeasurements=on
you need to be logged in but this will return an XML of all you data (200 pages of it, increase this number if you need more), you can then save it and then open this in excel, (new book>DATA>From Other Sources>From XML Data Import.
Or if you don’t have excel etc. you can get the data in json format with this:-
https://www.fitbit.com/weight?apiFormat=json&pages=200&pageNumber=0&viewRecentMeasurements=on
and use https://json-csv.com/ to convert it to csv.
sorted
LikeLike
Thanks for the comment, Stewart! It’s always good to have multiple ways to get at the information in case one stops working (case in point the Chrome extension I was using to get MyFitnessPal food info recently went poof). I’ve been using Zenobase as my data aggregation tool because I can get hour-resolution data on many of the Fitbit metrics including heart rate. I met Eric Jain (Zenobase’s developer) at QS15 and he’s been very helpful.
LikeLike