6 ms·

Basically someone didn’t do the load testing before prod release.

6nf 7y ago
People getting trained on the app last week reported manhy issues with it but the reply was 'don't worry you can always phone in the results too'
- runawaybottle 7y ago
  The company leadership basically didn’t take the Iowa caucus seriously. It’s obvious to me they wanted to bang out a quick frontend mobile app with a skeleton backend (Skeleton frontend for that matter) as a proof of concept to say ‘hey we did it! Now give us all the other primary contracts’, at which point perhaps they’d invest more heavily on a more comprehensive tech team/platform. They did this with a minimal viable product approach for what should have been a critical production ready/tested app.
yellow_postit 7y ago
Wasn’t even a prod release. They used a Test Flight like distribution platform to bypass the app stores because the app wasn’t ready in time to go through App Store approval processes.
maxlybbert 7y ago
That’s what I thought originally, but there really shouldn’t have been much load (ignoring the load of downloading the app to begin with). There were fewer than 2000 precincts, and each precinct had to upload around 100 bytes of data. That shouldn’t have been hard. One news story I read quoted an error message about a “missing protocol.” I believe that the “coding error” was really a configuration error: somebody forgot to include https:// https:// in the web service URL. The app recorded the data, but couldn’t upload it. Load testing would have found this, but so would a single integration test/smoke test.
- runawaybottle 7y ago
  My thinking is they simply didn’t do any dry runs with possible simulated data (bad data, malformed data, partial data), they didn’t consider all the ways things can go wrong in real time. This is less about load testing but more about exception handling/fallbacks.
  - maxlybbert 7y ago
    I definitely agree with you there.
- SyneRyder 7y ago
  There's a screenshot of that "Unknown Protocol" error message in this Vice article: https://www.vice.com/en_ca/article/y3m33x/heres-the-shadow-inc-app-that-failed-in-iowa-last-night https://www.vice.com/en_ca/article/y3m33x/heres-the-shadow-i... It (seems to) occur while trying to access a "2nd Factor Authentication" page at auth0.com, and there's a cross through the padlock icon at the bottom of the window.
  - maxlybbert 7y ago
    Thank you for tracking that down and sharing it!
alasdair_ 7y ago
A big part of the issue was that the app needed to be downloaded first and many of the precincts were in rural areas of Iowa with particularly spotty cell service.
flukus 7y ago
There was no load to test, even the most inefficient code running a web server off a mobile phone could have handled this load.

6nf 7y ago

People getting trained on the app last week reported manhy issues with it but the reply was 'don't worry you can always phone in the results too'

runawaybottle 7y ago
The company leadership basically didn’t take the Iowa caucus seriously. It’s obvious to me they wanted to bang out a quick frontend mobile app with a skeleton backend (Skeleton frontend for that matter) as a proof of concept to say ‘hey we did it! Now give us all the other primary contracts’, at which point perhaps they’d invest more heavily on a more comprehensive tech team/platform. They did this with a minimal viable product approach for what should have been a critical production ready/tested app.

yellow_postit 7y ago

Wasn’t even a prod release. They used a Test Flight like distribution platform to bypass the app stores because the app wasn’t ready in time to go through App Store approval processes.

maxlybbert 7y ago

That’s what I thought originally, but there really shouldn’t have been much load (ignoring the load of downloading the app to begin with). There were fewer than 2000 precincts, and each precinct had to upload around 100 bytes of data. That shouldn’t have been hard. One news story I read quoted an error message about a “missing protocol.” I believe that the “coding error” was really a configuration error: somebody forgot to include https:// https:// in the web service URL. The app recorded the data, but couldn’t upload it. Load testing would have found this, but so would a single integration test/smoke test.

runawaybottle 7y ago
My thinking is they simply didn’t do any dry runs with possible simulated data (bad data, malformed data, partial data), they didn’t consider all the ways things can go wrong in real time. This is less about load testing but more about exception handling/fallbacks.
- maxlybbert 7y ago
  I definitely agree with you there.
SyneRyder 7y ago
There's a screenshot of that "Unknown Protocol" error message in this Vice article: https://www.vice.com/en_ca/article/y3m33x/heres-the-shadow-inc-app-that-failed-in-iowa-last-night https://www.vice.com/en_ca/article/y3m33x/heres-the-shadow-i... It (seems to) occur while trying to access a "2nd Factor Authentication" page at auth0.com, and there's a cross through the padlock icon at the bottom of the window.
- maxlybbert 7y ago
  Thank you for tracking that down and sharing it!

alasdair_ 7y ago

A big part of the issue was that the app needed to be downloaded first and many of the precincts were in rural areas of Iowa with particularly spotty cell service.

flukus 7y ago

There was no load to test, even the most inefficient code running a web server off a mobile phone could have handled this load.