{"id":167,"date":"2022-05-18T17:00:00","date_gmt":"2022-05-18T17:00:00","guid":{"rendered":"https:\/\/blogs.oregonstate.edu\/davidkaff\/?p=167"},"modified":"2022-05-19T09:52:32","modified_gmt":"2022-05-19T09:52:32","slug":"telemetry-test-of-flutter-app","status":"publish","type":"post","link":"https:\/\/blogs.oregonstate.edu\/davidkaff\/2022\/05\/18\/telemetry-test-of-flutter-app\/","title":{"rendered":"Testing the Location Telemetry of a (Flutter) Mobile App"},"content":{"rendered":"\n
This is a report on a comparative field test I planned and performed, which tested the “Location Telemetry” of my Senior Software Engineering Capstone team’s Flutter App.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Route walked for test in Balboa Park<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n
Motivation<\/h2>\n\n\n\n
<\/div>\n\n\n\n
For the last 8 months I have been collaborating on the development of a cross-platform evacuation drill app. This app instructs and guides participants through an evacuation drill. It also records and eventually uploads the location<\/strong> and survey datasets<\/strong> that are generated by participants. These datasets go on to support Evacuation Traffic research.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Our app is replacing a previous workflow for drill participants which included:<\/p>\n\n\n\n
Following written instructions during the drill<\/li>
Using an exercise tracking app to record and export a location dataset<\/li><\/ol>\n\n\n\n
<\/div>\n\n\n\n
My team had previously tested and ensured that the app could administer surveys individually and record the results. We had also tested and ensured that the app could deliver novel instructions as prepared by the drill coordinators. <\/p>\n\n\n\n
<\/div>\n\n\n\n
However, we had not yet tested the app’s ability to generate and upload location<\/strong> datasets (also known as trajectory<\/em> datasets: location<\/em> + time<\/em>). In order to test this functionality of the app I planned and carried out a field test. To determine the efficacy of our app in comparison with the previous workflow, I planned the field test as a comparison with the previously used exercise tracking app.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Tools<\/h2>\n\n\n\n
<\/div>\n\n\n\n
I currently have three operational smartphones; <\/p>\n\n\n\n
an iPhone 11<\/li>
an iPhone SE (1st gen)<\/li>
a moto g(7)<\/li><\/ul>\n\n\n\n
<\/div>\n\n\n\n
I used all three devices in the test. <\/p>\n\n\n\n
<\/div>\n\n\n\n
To load our app onto the devices I used USB links to my development machine and installed the app using the flutter run<\/code> command. <\/p>\n\n\n\n
<\/div>\n\n\n\n
To load the exercise tracking app onto the devices I visited the relevant App Store pages and installed the app.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Procedure<\/h2>\n\n\n\n
<\/div>\n\n\n\n
There were several concerns which shaped my testing procedure:<\/p>\n\n\n\n
I needed to compare the results I generated against each other.
To generate a collection of datasets which could be meaningfully compared, I would need to track my location while repeatedly walking an identical route. <\/li><\/ul><\/li>
A known issue was causing early test results to show the output from the moto g(7) as approximately 60 ft below what should have been recorded.
To determine the exact offset I would need to gather results from a route which had varied elevation to produce signal despite potential low-level noise. <\/li><\/ul><\/li>
The was danger of a device battery running out of charge part-way through the test.
I needed to mitigate the possibility of gathering much more data with one app versus the other. So each lap I would need to alternate which app I was testing.<\/li><\/ul><\/li><\/ol>\n\n\n\n
<\/div>\n\n\n\n
These concerns informed my search for a test location. My priorities for the test location were for it to be:<\/p>\n\n\n\n
near public transit for ease of access<\/li>
a ~0.5km loop that I could walk repeatedly in a reasonable amount of time<\/li>
a minimum 10 meters of elevation change (which would create datasets that I could determine the Android elevation offset from)<\/li><\/ul>\n\n\n\n
<\/div>\n\n\n\n
I eventually settled on this route in Balboa Park (I currently live in San Diego, CA):<\/p>\n\n\n\n
<\/div>\n\n\n\n
Route walked for test in Balboa Park<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n
To comparatively test both apps, I planned to walk the route 10 total times: 5 times running each app, exporting the results after each complete lap.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Results<\/h2>\n\n\n\n
<\/div>\n\n\n\n
In the end, I generated the following quantity of location datasets:<\/p>\n\n\n\n
<\/div>\n\n\n\n
Device<\/th>
Exercise Tracking App<\/strong><\/th>
Evac. Drill App<\/strong><\/th><\/tr><\/thead>
iPhone 11<\/td>
1<\/td>
1<\/td><\/tr>
iPhone SE<\/td>
2<\/td>
3<\/td><\/tr>
moto g(7)<\/td>
3<\/td>
3<\/td><\/tr><\/tbody><\/table>Number of datasets generated, by device and app.<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n
Upon returning to my development machine I compiled the results from both apps and began to visually inspect the GPS data. I generated several comparative displays, the most interesting of which I have included below.<\/p>\n\n\n\n
<\/div>\n\n\n\nThere is clearly something awry with our telemetry!<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\nThese results look ok, but there is a small error in our app’s data. I only gathered one dataset for each app, as this phone was my bus pass home and the battery was nearly dead!<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\nHere we see clear errors in our app’s data.<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\nWow, here we can see our app’s data is really wacky. But the exercise app’s data is a bit wonky as well\u2026<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n
I then extracted the elevation data and created elevation vs. time plots for each device:<\/p>\n\n\n\nOur data is less smooth, but generally looks ok\u2026<\/figcaption><\/figure>\n\n\n\nA false-start trial is evident in this view (in green)<\/figcaption><\/figure>\n\n\n\nThe level of noise in this data was completely unexpected, but it was replicated in later trials.<\/figcaption><\/figure>\n\n\n\n
<\/div>\n\n\n\n
These results led to the following conclusions:<\/p>\n\n\n\n
<\/div>\n\n\n\n
Conclusions<\/h2>\n\n\n\n
<\/div>\n\n\n\n
Our app needs an \u201cAcquiring GPS Signal\u201d feature to filter out initial noise.<\/li><\/ol>\n\n\n\n
<\/div>\n\n\n\n<\/figure>\n\n\n\n
<\/div>\n\n\n\n
This is a feature which the exercise tracking app employs, and is evidently the reason for the erroneous GPS data at the start of the iPhone SE datasets. Such a feature will need to begin the streaming of location data, and determine when the signal is settled by using a decaying timer which resets upon any data point that is significantly different from the previous data point.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Noise in GPS data varied more by device than by app.<\/li><\/ol>\n\n\n\n
<\/div>\n\n\n\n<\/figure>\n\n\n\n
<\/div>\n\n\n\n
While the trajectory smoothing algorithms do set the exercise app results apart, this difference is smaller than the difference between results from different devices. This is significant because it means that the data our app generates is worth using for the intended application of Civil Engineering Evacuation Traffic research.<\/p>\n\n\n\n
<\/div>\n\n\n\n
Researchers may need to post-process calculation elevation data from GPS data if device altimeter outputs poor quality data.<\/li><\/ol>\n\n\n\n
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/BalboaParkRoute-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/GPSdataComparisons_byApp-1-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/GPSdataComparisons_11-1-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/GPSdataComparisons_SE-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/GPSdataComparisons_g7-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/elevVsTime_11-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/elevVsTime_SE-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/elevVsTime_g7-1024x640.png\")
![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/SEbadData-1024x640.png\")
<\/figure>\n\n\n\n<\/div>\n\n\n\n![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/GPSdataComparisons_byDevice-1-1024x640.png\")
<\/figure>\n\n\n\n<\/div>\n\n\n\n![\"\"](\"https:\/\/osu-wams-blogs-uploads.s3.amazonaws.com\/blogs.dir\/4766\/files\/2022\/05\/elevVsTime_g7-1-1024x640.png\")
<\/figure>\n\n\n\n<\/div>\n\n\n\n