The Challenge
The main objective of the competition revolved around developing innovative solutions for improving the infrastructure of cities. The Smart Infrastructure Hackathon was hosted at The Cannon, in partnership with Microsoft. Thankfully, we had a team of really bright cloud architects for guidance.
After juggling several ideas we settled with the issue of road potholes. We found that potholes cost American drivers $6.4 billion dollars per year, that is in both repair and insurance cost's. Additionally, the average price to repair a pothole including labor is about $30 - $50 per pothole, while the damage that single pothole may cause are up to $300/year per vehicle (2016). Thus the effort to aiding this issue can be indeed cost-effective.
Our Proposal
The idea of the application to be a simple as possible to minimize driving distractions. Therefore, we intended to create an interface that would show a dialog box only when our system predicted there to be a pothole, from there the user would confirm if so. Also, we designed for a 2-layer confirmation of a road pothole, one by the active camera, and secondly using either the user's phone telemetry data.
Analyzing Mobile Data
Every modern smartphone is equipped with very accurate telemetry systems. For instance the GPS and accelerometer, which we leveraged for location and to track any abrupt movement indicating a pothole on the road. One of the difficulties with this approach was that logging every high acceleration movement may result in false positives of a road pothole.
| Datetime | Accel.X (m/s²) | Accel.Y (m/s²) | Accel.Z (m/s²) | Latitude | Longitude | Altitude (m) |
|---|---|---|---|---|---|---|
| 2019-12-14 16:28:30:975 | -1.03 | -0.18 | 9.69 | 29.7955 | -95.568 | 6 |
| 2019-12-14 16:28:31:475 | -1.01 | -0.03 | 9.70 | 29.7955 | -95.568 | 6 |
| 2019-12-14 16:28:31:975 | -1.07 | -0.02 | 9.67 | 29.7955 | -95.568 | 6 |
| 2019-12-14 16:28:32:475 | -0.91 | -0.03 | 9.75 | 29.7955 | -95.568 | 6 |
| 2019-12-14 16:28:32:975 | -0.96 | -0.20 | 9.67 | 29.7955 | -95.568 | 6 |
Live Computer Vision
This part was a little more difficult, but this was our second layer for prediction of potholes, implementation done with some guidance from the MS architects. We used Azure Logic Apps to import our road image frames to the cloud, and with OpenCV where able to register a model to start analyzing our content with the conveniently named Azure microservice Computer Vision.
Geocoding / Mapping
Finally we needed to actually plot our GPS data so the city could take action. Our collected data included latitude, longitude, altitude and location accuracy. Location accuracy is important to pinpoint multiple occurrences of the same pothole from different users.
The Final Result
Having won the competition 🥇, after our minimum-viable-product, granted us credits to Microsoft Azure services, and even seats to the Cannon co-working space. Which we could use to continue development of the application.
