19 ms·

The TSA Randomizer App Cost $336k

486 points by andrewguenther 11y ago

noja 11y ago
"..that a beginner could build in a day.." Yeah... the clue is that an entire bureaucracy is required before even the first line of code was written. This thing probably took months and months of meetings with tons of people before the prototype was written.
- zappo2938 11y ago
  Do you know how many bike sheds they could have painted with the man hours?
- pink_dinner 11y ago
  Exactly. I used to work with a government contracting company. The simplest piece of software took many thousands of dollars in lawyer fees and compliancy. A beginner dev could build many simple web applications in a day. However, it will probably have security holes and spaghetti code.
  - gglover 11y ago
    I don't think lawyer fees imply anything about the code quality. See: HealthCare.gov fiasco.
    - jonesb6 11y ago
      I think OP meant that bootstrapped code, written as quickly as the developer feels like writing it on a weekend, has its own flaws as well.
      - ZanyProgrammer 11y ago
        Though an app showing a randomized arrow veers closely into, well, not Hello World, but an intro example for developing a mobile app. There's not going to be a lot of code to spaghetti-ize.
        AndyNemmity 11y ago
        In Enterprise software? You'd be surprised...
    - ori_b 11y ago
      It implies that you know who to sue if code quality isn't up to scratch. This is important to large organizations for some reason. Even though it may be cheaper to just do it 10 times over with cheap one person shops and throw out the worst 9.
      - scrollaway 11y ago
        And who was sued in the case of the Healthcare.gov debacle? I get the feeling this "knowing who to sue" saying is a bit of a myth.
        ori_b 11y ago
        I suspect that the myth is that anyone will follow through on it. The ass covering is probably real.
  - hiimnate 11y ago
    >security holes It's an arrow that points in one of two directions. I can not think of a single way this could be made insecure. Even if someone was able to influence the direction the arrow was pointing, everyone is still going through the same ineffective screening process.
    - odinduty 11y ago
      They could hack into the iPad to make it display this: http://www.thecleverest.com/countdown.swf http://www.thecleverest.com/countdown.swf High chance somebody gets shot.
      - rimantas 11y ago
        That's a lot of hacking to display Flash on iPad :)
    - Johnny555 11y ago
      One way it could be made insecure would be if it didn't use a good source of randomness, so the arrow direction could be predicted. A worst case example would be if the app just cycled through a fixed table of "random" choices like "L R L L R L R R", then the attacker could just watch the pattern and position himself in the line appropriately to make sure that his buddy Joe is the one to screen him. repositioning is easy, just dig through your bag and say to the guy behind you "I can't find my passport, you can go in front of me".
      - Scoundreller 11y ago
        That problem would be solved by making the app very unstable and require constant restarts ;)
      - gambiting 11y ago
        And all of this trouble to achieve.....what exactly?
        11y ago
- drdeadringer 11y ago
  What would happen if, some fraction of the way through the long stretch of bureaucratic meetings, someone just coded a prototype? If they sit on it and whip it out the moment the green light is given, then what? If they whip it out at the next red-tape meeting, then what?
  - tdicola 11y ago
    Their manager will have a stern talking to them about how 'this isn't how we operate' and 'you broke protocol' and 'made everyone look bad'. They'd be reassigned to the most boring and menial tasks until they quit.
  - grecy 11y ago
    I can tell you what happens. I worked at a very large telco. We went through months of daily meeting with 10+ very expensive execs, PMs and Business Primes from all over the business. One day the only other coder in the room leaned over to me and said "Are we still talking about <x,y,z> that you or I could code in 10 minutes?" Yes, we were. I coded it and showed my managers and so forth. I was completely ignored, and the meetings rolled on. In a place like that, there are so many people that have job titles that have nothing to do with "getting it done", which in a funny way makes them less interested in getting it done. Their job is to analyze, plan, document, process map, etc. etc. so that's what they're going to do, no matter how trivial the thing is.
    - AndyNemmity 11y ago
      Exactly! I would argue it creates MORE red tape and work to do that, than less. I've done it, I've seen it, it actually makes your life worse, not better.
      - grecy 11y ago
        When I stopped trying to make that place better (I'm banging my head against the wall) and started just going with the flow, I knew it was time to leave.
    - 11y ago
- AndyNemmity 11y ago
  Yep. You have the AE, and then the AE on a large project has another AE with him. Then you have a solution captain. Then you have the COE that comes in that actually knows something. Then an architect to design the thing. Then you have others who actually do the thing. Then you have a PM to schedule meetings, and bitch at the doers to do the thing. Then you have all the daily meetings to communicate to all of those people the messaging. Then you have the meetings to clarify all the wrong messaging that went in some of those meetings. Then you have the meetings for the customer facing communication. Then you have the customer communication. I'm not even close to done...
factorfractal 11y ago
They could have used a magic 8 ball with similar outcomes
- SandB0x 11y ago
  Or even a coin.
  - fabulist 11y ago
    For that matter, they could tell odd parties to go left and even parties to go right.
    - agf 11y ago
      I doubt this was meant seriously, but it's worth noting this actually doesn't work. You need each left/right decision to be independent so that positioning yourself at a certain place in line doesn't allow you to choose left/right.
      - fabulist 11y ago
        So you're saying this app operated by TSA agents is meant to defeat attackers looking to be screened by cooperative TSA agents? I can't help but feel something is lacking in this threat model.
        KMag 11y ago
        Defense in depth is a real thing.
        deleted 11y ago
        [deleted]
        deleted 11y ago
        [deleted]
        11y ago
seibelj 11y ago
I worked at a company that had to resell its software through a disabled, veteran-owned sole proprietorship so that the TSA could find budget to purchase it. The way it was described to me was that a portion of the budget is earmarked for these businesses, so if the primary technology purchasing budget runs out, you can still sell to the TSA if you resell through one of these businesses. Essentially, the reseller took a cut for doing nothing, and fleeced the American taxpayer for a few hundred thousand dollars. I'm sure this happens all the time.
- rhizome 11y ago
  To be fair, if the taxpayers are going to be victims of bureaucratic inefficiencies, I'd prefer that the beneficiaries be disabled veterans vs. that money going to Boeing or Raytheon or Palantir or whoever.
  - nefitty 11y ago
    Those companies could more effectively redistribute or reinvest that wealth than an individual ever could.
    - superuser2 11y ago
      Why? The money is ultimately just going to pay some individuals - they can be upwardly mobile elites from good schools (Palantir), stodgy defense establishment types (defense contractors), or groups that traditionally get less of this kind of attention. Why not?
      - nefitty 11y ago
        Are the down votes because of the content or style of my comment? Ultimately, it's a matter of the capital those companies have invested. These companies can more efficiently allocate the extracted wealth toward research and other projects that there is demand for. I'd rather have an efficient organization running off with my tax money than an individual who might go buy a yacht with that money. A corporation has little incentive to spend money the same way an individual would.
LoSboccacc 11y ago
Couldn't find in the contract if the hardware was provisioned or not.
slagfart 11y ago
Honestly, for a custom app from one of the largest corporations (IBM), for the largest of organisations (US Govt), $336k is pretty reasonable!
- AndyNemmity 11y ago
  Seriously. 336k is absolutely nothing. Think about how fast that gets used up adding up the numbers of the per hour of those involved, and how many people it is (not just the developer who does it) You can argue that you don't need that many people, and certainly for a start up that's often true. But in enterprise software, you actually do need a ton of people because of the expectation. I'm in the Business and came from start ups. I'd VASTLY prefer to do things in small groups to just get things done. But even though I want that, no one else does, including the customer. They want all the bells and whistles of a ton of people. It makes no sense at all. Hell, one of my current projects has about 20 people on a phone call, and 4 that actually understand anything going on. Think about all the money wasted in all of that, and yet... It's the way business "works".
abritishguy 11y ago
If this is what it seems then that is ludicrous.
jemfinch 11y ago
If this is truly the price to develop such simple software, it's going to be very bad for the government's reputation to be releasing code like this under open source licenses.
- Pxtl 11y ago
  Theyve already said they won't if there's a security or privacy risk, and you can find one of those in any code.
caseysoftware 11y ago
Notice this is also T&M (time & materials) which is not just a fixed priced bid.. so that means someone "used" the actual working hours and materials for the $336k bill. (Contract negotiation, etc is not explicitly built into these, they're reflected in a higher billing rate.)
- fisherjeff 11y ago
  No, this is just the contract, which would be based on IBM's cost estimate. Since it's T&M, the actual total cost and hours will only be reflected on IBM's invoices.
notmyemployer 11y ago
I've done some federal contracting and the issues seem to be cultural. The government hands out what they call "prime" contracts which are then subcontracted out to multiple other firms. The primes tend to be stodgy old companies filled with lawyers and MBAs that can win the contracts, who then view the engineers as replaceable cogs. You then interact with multiple other contractors that own particular parts of the stack, for example an independent testing contractor and another for infrastructure. It wasn't uncommon to have 5 project managers for each engineer on the project. Coming from tech startups this was completely shocking, I was so used to an engineer driven culture. That developers couldn't write their own tests, or manage their own infra, or deploy multiple times a day was super frustrating. There sounds like there are some great initiatives to change this old approach, but until then I can't imagine many talented devs would put up with the bureaucratic bullshit.
- jchrisa 11y ago
  And yet there's good code written in the midst of it sometimes.
- ViViDboarder 11y ago
  I worked in the past on an IBM project with the TSA, and they were indeed the prime on a large software contract. About half the team was IBM, the other half medium to small sized subs. Honestly, most of the wasted time wasn't the engineering teams being slow, but anytime something had to be run by the government, it halted. It was utterly depressing.
  - notmyemployer 11y ago
    This was my experience as well, it was mainly the arbitrary road blocks. When I'd be on a call and realize there was finally another dev on the line I'd immediately get in touch via email or chat and back channel while the swarm of project managers talked about who knows what.
- cmurf 11y ago
  These companies need to just be fired. Seriously. Send them back to the real world for a while. Makes me think of the Vogons from Hitchkiker's Guide to the Galaxy.
venomsnake 11y ago
It was 300000 for lobbying grease to land the contract, 30000 profit, 6k for hardware and 500$ for one intern to write it.
- AndyNemmity 11y ago
  Funny but ultimately wrong. The real distribution of where the money goes on projects is certainly funny from a sane persons perspective, but in a different way than this :)
TelAviv 11y ago
Not to mention the cost of all those iPads, which was probably millions of dollars. (They could have just invested in some 24-sided dice! (Search Amazon for the "D24") And I wouldn't trust the TSA with iPads anyway.... http://abcnews.go.com/Blotter/abc-news-tracks-missing-ipad-florida-home-tsa/story?id=17331937 http://abcnews.go.com/Blotter/abc-news-tracks-missing-ipad-f...
gojomo 11y ago
The PDF contract the author shares doesn't make it clear what deliverables it covers. Perhaps we can assume that if the FOIA process worked it is in fact the contract that covered that app... but it's not clear from the document itself, nor is it clear that the 'TSA Randomizer App` was the only deliverable of the contract.
- kevinburke 11y ago
  Yes, it was unfortunate I didn't get more data/information back from them :(
valine 11y ago
The government would be a lot more efficient if everyone who worked for them was a little more tech literate. I don't think the lack of education is entirely the US government's fault. Tech moves so insanely fast it can be a full time job just keeping up with the changes. My point being, this is not the price of building a tiny app. 336k for a randomized arrow is absolutely ridiculous. This is the price for making a change in a government run security system. A change built on technology that government officials don't fully understand.
- brown9-2 11y ago
  This assumes all inefficiency is accidental, or done out of a lack of knowledge.
  - CyberDildonics 11y ago
    Don't attribute to malice what can be attributed to ignorance
- sailfast 11y ago
  They're hiring. Good people needed. Please consider it. Quite an opportunity to actually embed a solid tech culture in some places that really need it: https://www.whitehouse.gov/digital/united-states-digital-service/apply https://www.whitehouse.gov/digital/united-states-digital-ser...
- rplst8 11y ago
  I've actually gotten to the point that I think the speed at which software tech moves (and we're all expected to move with it) has become a negative on the industry. No one has time to hone their craft when everyone making the architecture and language choice decisions has shiny new object syndrome. It used to be that people who worked at their jobs for 20-30 years or more became "elders" in an organization and were able to use that experience to guide the younger generation and avoid the same mistakes and design pitfalls. There are some good design practices that still do carry on, but having to learn a new language every year (plus all the frameworks that go with it) is not a pleasant or rewarding experience, nor does it allow for much time to become an expert. Note, I do understand that sometimes new ways of doing things allow one to solve a problem such it would have previously been near impossible. However, most of the problems that need solving aren't these situations. I.e. if you think you have a big data problem, you probably don't, etc. I think this especially detrimental in the government sector because the folks buying the services don't always understand the technology or the maintenance requirements that come with it. Then we all foot the bill (through taxes) for a lot of OJT for said new tech.
tootie 11y ago
Once again, contracts are rarely for just development. It likely included design, development, deployment, testing, training, accessibility testing, analytics.
spilk 11y ago
No invoices against the contract? This is only part of the picture. It's a time and materials contract which means the contract award is the "ceiling", but you have to perform work billed against it to get paid any of that. Plus, the second page details several extension options that could increase the ceiling to $1,176,280. Were those options exercised?
- kevinburke 11y ago
  We don't know; I've amended the post to reflect this. Thanks.
lowboy 11y ago
Why is there a ?lobsters qs on the submitted url? Seems to load fine without: https://kev.inburke.com/kevin/tsa-randomizer-app-cost-336000/ https://kev.inburke.com/kevin/tsa-randomizer-app-cost-336000...
- kevinburke 11y ago
  I submitted it to Lobste.rs with that on the end of the URL, and someone must have submitted the URL without stripping that :)
swanson 11y ago
Let's assume a hefty markup for the long sales/BD time for a government contract like this: $336k * 0.60 = $200k Since this deals with national security, could probably justify higher rate resources staffing (security clearance required?). Ballpark $250/hr @ 8 hr/day so $200k buys you 100 days of capacity. Project breakdown: * UI/UX Design: 10 days (lots of stakeholders+approvals needed, maybe field testing with TSA agents, accessibility audit) * Development: 15 days (this is very conservative but might be some business rules that aren't being shared -- remote monitoring, analytics, auditing, static analysis, etc) * Project Management: 25 days * QA: 15 days (multiple devices, high perceived security exposure) * Third-party security audit/pen testing: 5 days * Technical writer for documentation and training materials: 5 days Total: 75 days of capacity spent. Math doesn't work out exactly and some of these numbers are pulled from thin air, but it seems like it's at least in the right ballpark. Do you folks really know people that are doing government contract work (for the TSA no less) for less than $100k?
- AndyNemmity 11y ago
  PM would be higher than that. There'd also be a Manager above the PM scheduled some days as well. And the security, analytics, and monitoring are probably all smaller specifics days for individual experts. Point is though, you're thinking the right way on it. Very reasonable.
  - swanson 11y ago
    Good point, added a couple more line items.
- joering2 11y ago
  No, just like I don't believe any reasonable company could be allowed this contract for less money. Having to dealt enough with IBM on contract levels, I rather believe there has been bribes involved, but I can't prove anything.
  - AndyNemmity 11y ago
    I live in this world, and have been in the upper levels of all sorts of these discussions regularly. I've never seen a direct bribe. If you want to call lavish dinners, drinking, and entertainment bribes, okay, that happens constantly. But a bribe? No... You don't need bribes. It's quid pro quo all of the time, but bribes?! No... Maybe you can get hired at a higher pay scale/job title from your work getting someone a deal. There are certainly perks.. but not bribes...
jsprogrammer 11y ago
The page (2 of 8) that had the unit prices censored (exception (b)(4))[0], claims the total award amount was $1,176,280.72. Really, what we need is IBM's invoices and TSA's pay statements. Also, you shouldn't need to file an FOIA request to get this information. Shouldn't all bids and contracts be public? [0] Exemption (b)(4): Records that contain trade secrets and commercial or financial information obtained from a person that is privileged or confidential.
- AndyNemmity 11y ago
  Unit prices are trade secrets. If they published that, then someone else getting a higher price could point to it, and say they only charged X for them.
kra34 11y ago
Yo app got a $1,000,000 from Andreessen Horowitz, so they kind of got a deal if you think about it that way
brandonmenc 11y ago
If you're a consultant or freelancer, and you're not charging that much to build an equivalent app, you are undercharging. This is a completely reasonable price. Honestly, it seems cheap.
mikeryan 11y ago
A few things. 1. Kevin you may want to block out your address in the attached letter. 2. I'm missing the part where this amount is tied directly to the creation of this application and this application only. From my read this is a T&M contract for mobile development there's nothing specific to this application. It could span multiple engagements and apps. Considering this contract went into place over 2 years ago this could likely be the case. 3. This is a cap not an invoice. They could have billed 60k.
- sailfast 11y ago
  Some details of past IBM wins here via UsASpending: https://www.usaspending.gov/Transparency/Pages/AwardSummary.aspx?AwardID=17381939 https://www.usaspending.gov/Transparency/Pages/AwardSummary.... This may be the preceding contract but very similar scope and line items if you want more details, no FOIA required.
intrasight 11y ago
What is WAY worse is that they probably purchased 10,000 ipads to run this stupid app.
jonhohle 11y ago
A coworker and I were laughing about this at the airport one day. I literally wrote the app in the flight home and almost put it up on the App Store.
jqm 11y ago
Ya well, that's 10K development, 30K profit and 290K meeting time and paying people to fill out forms. Sounds about right.
ThinkBeat 11y ago
With a reference to another recent hackernews story: https://news.ycombinator.com/item?id=11415747 https://news.ycombinator.com/item?id=11415747 Can we expect to see the source code for this application pretty soon?
bbcbasic 11y ago
But no one got fired for choosing IBM
- FriedPickles 11y ago
  I came here to say this.
  - bbcbasic 11y ago
    Pfft! Yet another cliché slinger
kumarski 11y ago
Less worried about that and more worried about the fact that baggage scanners only scan from one angle combined with reality that they don't do material differentiation. Z-effective scores...findable.
transfire 11y ago
Hmmm... at the current rate of bureaucratic growth, I estimate in a 100 years or so, this app (or it's modern AR equivalent) will cost 20% of the entire federal budget. No worries though, 10% of all U.S. workers will be involved in its digital-paper-pushing development.
woodman 11y ago
Before we even go into line item mode we really need to ask ourselves if this is a $336k problem in the first place. They could have just asked the military how they handle this sort of problem, and the solution they would have been given is this: a $2 hand tally counter. Establish what the desired flow rate is, divert every nth person, reset the clicker, anybody in line observed shifting position gets diverted. Unless the TSA can't even trust their people to count, or their real objective is adding some weirdly alienating layer of technology between the employee and the traveler.
- jcrawfordor 11y ago
  "every nth person" selection is just too predictable. You could try to prevent people shifting in line, but in a crowded airport you will never be able to control that effectively at current staffing levels. The cost of effectively enforcing that rule would rapidly add up to more than $300k and cause serious inconvenience to travelers.
  - mikeash 11y ago
    Yes, preventing shifting in line doesn't help if your attacker can just count the length of the line before they get in it. You really want a random element. Anyone familiar with Dungeons and Dragons should be able to name the obvious solution there....
    - woodman 11y ago
      The uncertainty already inherent to the existing process is high enough in my opinion, either the software or the hand counter would equally meet the real objective here: providing a point to audit for reportable metrics. While a dungeon master presiding over a hopeful throw for critical hits would be a solution, I don't think the TSA wants to make security theater entertaining - it makes the accidental nature of the whole endeavor obvious.
    - mfukar 11y ago
      But Mike, what good would Polymorph do in this scenario?
pratheekrebala 11y ago
The $336K appears to be just the cost of one installment. The total project costed $1,444,315 Here are the rest of the transactions under the same contract: https://www.usaspending.gov/Pages/AdvancedSearch.aspx?k=HSTS0313JCIO494 https://www.usaspending.gov/Pages/AdvancedSearch.aspx?k=HSTS... TSA spends most of it's budget through a provision called "Other Transaction Authority" which is essentially a vehicle to make purchases with barely any oversight from congress. http://time.com/4134368/tsa-price-of-security/ http://time.com/4134368/tsa-price-of-security/ (Paywall)
- jdmichal 11y ago
  I'm glad you brought this up, because this comment represents a misunderstanding in how government contracting is typically done. My previous job was working for a government contractor, and we did multiple discrete contracts for several branches of the government. However, every single one of them would have shown up exactly like this, all bundled under one contract. Why? Because writing and signing a contract is expensive, for both the contractor and the government. So contracts are typically written in a way to make them very easy to extend, and existing contracts are often used as vehicles to tack on additional funding for new contracts. The original contracting agency would also usually charge a fee to the other agencies for use of their contract in this way. So, this kind of glance at individual awards under a single contract is really too simple a view. A lot of these funds could be (and from my guess, probably are) going to completely independent projects from the randomizer. TL;DR: Having multiple awards under a single contract is typically a sign of the government working around bureaucracy and attempting to save costs.
  - pratheekrebala 11y ago
    Yeah -- you are correct. TSA later reported that the actual cost of the randomizer was around $47k. The total figure I had mentioned earlier was part of a larger contract with IBM. I saw the generic "IGF::CT::IGF MOBILE APPLICATION DEVELOPMENT" note on the award and assumed it was all for the same project. It's also weird that TSA's response to OP's FOIA request cited a different and higher figure (~$340k) for the randomizer app -- I'm guessing they were grouping in other projects under the same award here too? Besides, $47k seems like a reasonable amount if that figure includes training and deployment costs.
wkoszek 11y ago
It'd be interesting to hear what do "materials" can entail. It might be that they actually delivered iPads to the airport.
brokentone 11y ago
We're talking about the app with 1 arrow on the screen that responds to a press anywhere on the same screen? 1 icon, 1 tap target, hooked up to 1 secure random algorithm. I don't know how anyone can look at this situation and not see the government is wasting a lot of money in their pursuit of not understand technology.
cantrevealname 11y ago
I suspect that one secret feature of this app is a discreet way to override the randomizer and send a passenger to the "intrusive search" lane. That is, if the screener suspects someone, he or she can swipe or click in a clandestine way to guarantee that the passenger gets the annoying search. The passenger can't complain about being targeted or profile or singled out, because, hey, it's random. (The actual rationale of the randomizer might have been to avoid accusations of profiling, while still being able to do profiling. Leaving the bad guys off balance is simply a benefit.)
blakecaldwell 11y ago
Why not just have TSA agents flip a coin, or buy a bunch of Trouble board games, and repurpose the dice rolling bubble?
- EdHominem 11y ago
  The trouble bubble is not only a perfect solution for this, but an apt metaphor for the whole process.
kevin_thibedeau 11y ago
It seems a bingo spinner with two colors of balls would do just as good of a job. https://www.google.com/search?q=bingo+spinner&tbm=isch https://www.google.com/search?q=bingo+spinner&tbm=isch
GreaterFool 11y ago
A little bit tangential: when you go to the slower lane and they swab you with those silly machines, can they actually detect anything?
- EdHominem 11y ago
  fyi, this is not a time to apply science.
joeld42 11y ago
What I want to know is does it still work if they turn on the rotation lock and hold the iPad upside down?
tn13 11y ago
Why does this surprise anyone? Forget an app. Just a normal computer that government buys ends up paying 1.5 of what you will get on Amazon.com. This is because government does not pay on time, there is a lengthy and resource consuming process in selling something to the government. The app was probably $10k, rest was the compliance cost.
makecheck 11y ago
One-sided negotiation is a cost riser. The last thing a client should be able to tell is that the payer has very deep pockets; if I know you can afford billions, I might ask a ridiculous price and expect you to not know any better or not care about paying it. An excellent use of secrecy technologies would be government contracts: partition problems into pieces that don’t necessarily reveal the final purpose or customer (e.g. “choose uniformly between A and B” is too vague to be guessed as a requirement from only a government agency). Then, anonymously ask for bids. That way, you might actually get the developer who offers a measly price for a trivial task, and only finds out later that his check is from the government.
deleted 11y ago
[deleted]
omk 11y ago
Q: How many IBM employees does it take to change the light bulb? A: Don't know. But will cost $1M+. They will pay for the bulb.
satyajeet23 11y ago
random rand = new random(); int tsaNumber = rand.nextInt();

486 points by andrewguenther 11y ago

noja 11y ago

"..that a beginner could build in a day.." Yeah... the clue is that an entire bureaucracy is required before even the first line of code was written. This thing probably took months and months of meetings with tons of people before the prototype was written.

zappo2938 11y ago
Do you know how many bike sheds they could have painted with the man hours?
pink_dinner 11y ago
Exactly. I used to work with a government contracting company. The simplest piece of software took many thousands of dollars in lawyer fees and compliancy. A beginner dev could build many simple web applications in a day. However, it will probably have security holes and spaghetti code.
- gglover 11y ago
  I don't think lawyer fees imply anything about the code quality. See: HealthCare.gov fiasco.
  - jonesb6 11y ago
    I think OP meant that bootstrapped code, written as quickly as the developer feels like writing it on a weekend, has its own flaws as well.
    - ZanyProgrammer 11y ago
      Though an app showing a randomized arrow veers closely into, well, not Hello World, but an intro example for developing a mobile app. There's not going to be a lot of code to spaghetti-ize.
      - AndyNemmity 11y ago
        In Enterprise software? You'd be surprised...
  - ori_b 11y ago
    It implies that you know who to sue if code quality isn't up to scratch. This is important to large organizations for some reason. Even though it may be cheaper to just do it 10 times over with cheap one person shops and throw out the worst 9.
    - scrollaway 11y ago
      And who was sued in the case of the Healthcare.gov debacle? I get the feeling this "knowing who to sue" saying is a bit of a myth.
      - ori_b 11y ago
        I suspect that the myth is that anyone will follow through on it. The ass covering is probably real.
- hiimnate 11y ago
  >security holes It's an arrow that points in one of two directions. I can not think of a single way this could be made insecure. Even if someone was able to influence the direction the arrow was pointing, everyone is still going through the same ineffective screening process.
  - odinduty 11y ago
    They could hack into the iPad to make it display this: http://www.thecleverest.com/countdown.swf http://www.thecleverest.com/countdown.swf High chance somebody gets shot.
    - rimantas 11y ago
      That's a lot of hacking to display Flash on iPad :)
  - Johnny555 11y ago
    One way it could be made insecure would be if it didn't use a good source of randomness, so the arrow direction could be predicted. A worst case example would be if the app just cycled through a fixed table of "random" choices like "L R L L R L R R", then the attacker could just watch the pattern and position himself in the line appropriately to make sure that his buddy Joe is the one to screen him. repositioning is easy, just dig through your bag and say to the guy behind you "I can't find my passport, you can go in front of me".
    - Scoundreller 11y ago
      That problem would be solved by making the app very unstable and require constant restarts ;)
    - gambiting 11y ago
      And all of this trouble to achieve.....what exactly?
      - 11y ago
drdeadringer 11y ago
What would happen if, some fraction of the way through the long stretch of bureaucratic meetings, someone just coded a prototype? If they sit on it and whip it out the moment the green light is given, then what? If they whip it out at the next red-tape meeting, then what?
- tdicola 11y ago
  Their manager will have a stern talking to them about how 'this isn't how we operate' and 'you broke protocol' and 'made everyone look bad'. They'd be reassigned to the most boring and menial tasks until they quit.
- grecy 11y ago
  I can tell you what happens. I worked at a very large telco. We went through months of daily meeting with 10+ very expensive execs, PMs and Business Primes from all over the business. One day the only other coder in the room leaned over to me and said "Are we still talking about <x,y,z> that you or I could code in 10 minutes?" Yes, we were. I coded it and showed my managers and so forth. I was completely ignored, and the meetings rolled on. In a place like that, there are so many people that have job titles that have nothing to do with "getting it done", which in a funny way makes them less interested in getting it done. Their job is to analyze, plan, document, process map, etc. etc. so that's what they're going to do, no matter how trivial the thing is.
  - AndyNemmity 11y ago
    Exactly! I would argue it creates MORE red tape and work to do that, than less. I've done it, I've seen it, it actually makes your life worse, not better.
    - grecy 11y ago
      When I stopped trying to make that place better (I'm banging my head against the wall) and started just going with the flow, I knew it was time to leave.
  - 11y ago
AndyNemmity 11y ago
Yep. You have the AE, and then the AE on a large project has another AE with him. Then you have a solution captain. Then you have the COE that comes in that actually knows something. Then an architect to design the thing. Then you have others who actually do the thing. Then you have a PM to schedule meetings, and bitch at the doers to do the thing. Then you have all the daily meetings to communicate to all of those people the messaging. Then you have the meetings to clarify all the wrong messaging that went in some of those meetings. Then you have the meetings for the customer facing communication. Then you have the customer communication. I'm not even close to done...

factorfractal 11y ago

They could have used a magic 8 ball with similar outcomes

SandB0x 11y ago
Or even a coin.
- fabulist 11y ago
  For that matter, they could tell odd parties to go left and even parties to go right.
  - agf 11y ago
    I doubt this was meant seriously, but it's worth noting this actually doesn't work. You need each left/right decision to be independent so that positioning yourself at a certain place in line doesn't allow you to choose left/right.
    - fabulist 11y ago
      So you're saying this app operated by TSA agents is meant to defeat attackers looking to be screened by cooperative TSA agents? I can't help but feel something is lacking in this threat model.
      - KMag 11y ago
        Defense in depth is a real thing.
        deleted 11y ago
        [deleted]
        deleted 11y ago
        [deleted]
        11y ago

seibelj 11y ago

I worked at a company that had to resell its software through a disabled, veteran-owned sole proprietorship so that the TSA could find budget to purchase it. The way it was described to me was that a portion of the budget is earmarked for these businesses, so if the primary technology purchasing budget runs out, you can still sell to the TSA if you resell through one of these businesses. Essentially, the reseller took a cut for doing nothing, and fleeced the American taxpayer for a few hundred thousand dollars. I'm sure this happens all the time.

rhizome 11y ago
To be fair, if the taxpayers are going to be victims of bureaucratic inefficiencies, I'd prefer that the beneficiaries be disabled veterans vs. that money going to Boeing or Raytheon or Palantir or whoever.
- nefitty 11y ago
  Those companies could more effectively redistribute or reinvest that wealth than an individual ever could.
  - superuser2 11y ago
    Why? The money is ultimately just going to pay some individuals - they can be upwardly mobile elites from good schools (Palantir), stodgy defense establishment types (defense contractors), or groups that traditionally get less of this kind of attention. Why not?
    - nefitty 11y ago
      Are the down votes because of the content or style of my comment? Ultimately, it's a matter of the capital those companies have invested. These companies can more efficiently allocate the extracted wealth toward research and other projects that there is demand for. I'd rather have an efficient organization running off with my tax money than an individual who might go buy a yacht with that money. A corporation has little incentive to spend money the same way an individual would.

LoSboccacc 11y ago

Couldn't find in the contract if the hardware was provisioned or not.

slagfart 11y ago

Honestly, for a custom app from one of the largest corporations (IBM), for the largest of organisations (US Govt), $336k is pretty reasonable!

AndyNemmity 11y ago
Seriously. 336k is absolutely nothing. Think about how fast that gets used up adding up the numbers of the per hour of those involved, and how many people it is (not just the developer who does it) You can argue that you don't need that many people, and certainly for a start up that's often true. But in enterprise software, you actually do need a ton of people because of the expectation. I'm in the Business and came from start ups. I'd VASTLY prefer to do things in small groups to just get things done. But even though I want that, no one else does, including the customer. They want all the bells and whistles of a ton of people. It makes no sense at all. Hell, one of my current projects has about 20 people on a phone call, and 4 that actually understand anything going on. Think about all the money wasted in all of that, and yet... It's the way business "works".

abritishguy 11y ago

If this is what it seems then that is ludicrous.

jemfinch 11y ago

If this is truly the price to develop such simple software, it's going to be very bad for the government's reputation to be releasing code like this under open source licenses.

Pxtl 11y ago
Theyve already said they won't if there's a security or privacy risk, and you can find one of those in any code.

caseysoftware 11y ago

Notice this is also T&M (time & materials) which is not just a fixed priced bid.. so that means someone "used" the actual working hours and materials for the $336k bill. (Contract negotiation, etc is not explicitly built into these, they're reflected in a higher billing rate.)

fisherjeff 11y ago
No, this is just the contract, which would be based on IBM's cost estimate. Since it's T&M, the actual total cost and hours will only be reflected on IBM's invoices.

notmyemployer 11y ago

I've done some federal contracting and the issues seem to be cultural. The government hands out what they call "prime" contracts which are then subcontracted out to multiple other firms. The primes tend to be stodgy old companies filled with lawyers and MBAs that can win the contracts, who then view the engineers as replaceable cogs. You then interact with multiple other contractors that own particular parts of the stack, for example an independent testing contractor and another for infrastructure. It wasn't uncommon to have 5 project managers for each engineer on the project. Coming from tech startups this was completely shocking, I was so used to an engineer driven culture. That developers couldn't write their own tests, or manage their own infra, or deploy multiple times a day was super frustrating. There sounds like there are some great initiatives to change this old approach, but until then I can't imagine many talented devs would put up with the bureaucratic bullshit.

jchrisa 11y ago
And yet there's good code written in the midst of it sometimes.
ViViDboarder 11y ago
I worked in the past on an IBM project with the TSA, and they were indeed the prime on a large software contract. About half the team was IBM, the other half medium to small sized subs. Honestly, most of the wasted time wasn't the engineering teams being slow, but anytime something had to be run by the government, it halted. It was utterly depressing.
- notmyemployer 11y ago
  This was my experience as well, it was mainly the arbitrary road blocks. When I'd be on a call and realize there was finally another dev on the line I'd immediately get in touch via email or chat and back channel while the swarm of project managers talked about who knows what.
cmurf 11y ago
These companies need to just be fired. Seriously. Send them back to the real world for a while. Makes me think of the Vogons from Hitchkiker's Guide to the Galaxy.

venomsnake 11y ago

It was 300000 for lobbying grease to land the contract, 30000 profit, 6k for hardware and 500$ for one intern to write it.

AndyNemmity 11y ago
Funny but ultimately wrong. The real distribution of where the money goes on projects is certainly funny from a sane persons perspective, but in a different way than this :)

TelAviv 11y ago

Not to mention the cost of all those iPads, which was probably millions of dollars. (They could have just invested in some 24-sided dice! (Search Amazon for the "D24") And I wouldn't trust the TSA with iPads anyway.... http://abcnews.go.com/Blotter/abc-news-tracks-missing-ipad-florida-home-tsa/story?id=17331937 http://abcnews.go.com/Blotter/abc-news-tracks-missing-ipad-f...

gojomo 11y ago

The PDF contract the author shares doesn't make it clear what deliverables it covers. Perhaps we can assume that if the FOIA process worked it is in fact the contract that covered that app... but it's not clear from the document itself, nor is it clear that the 'TSA Randomizer App` was the only deliverable of the contract.

kevinburke 11y ago
Yes, it was unfortunate I didn't get more data/information back from them :(

valine 11y ago

The government would be a lot more efficient if everyone who worked for them was a little more tech literate. I don't think the lack of education is entirely the US government's fault. Tech moves so insanely fast it can be a full time job just keeping up with the changes. My point being, this is not the price of building a tiny app. 336k for a randomized arrow is absolutely ridiculous. This is the price for making a change in a government run security system. A change built on technology that government officials don't fully understand.

brown9-2 11y ago
This assumes all inefficiency is accidental, or done out of a lack of knowledge.
- CyberDildonics 11y ago
  Don't attribute to malice what can be attributed to ignorance
sailfast 11y ago
They're hiring. Good people needed. Please consider it. Quite an opportunity to actually embed a solid tech culture in some places that really need it: https://www.whitehouse.gov/digital/united-states-digital-service/apply https://www.whitehouse.gov/digital/united-states-digital-ser...
rplst8 11y ago
I've actually gotten to the point that I think the speed at which software tech moves (and we're all expected to move with it) has become a negative on the industry. No one has time to hone their craft when everyone making the architecture and language choice decisions has shiny new object syndrome. It used to be that people who worked at their jobs for 20-30 years or more became "elders" in an organization and were able to use that experience to guide the younger generation and avoid the same mistakes and design pitfalls. There are some good design practices that still do carry on, but having to learn a new language every year (plus all the frameworks that go with it) is not a pleasant or rewarding experience, nor does it allow for much time to become an expert. Note, I do understand that sometimes new ways of doing things allow one to solve a problem such it would have previously been near impossible. However, most of the problems that need solving aren't these situations. I.e. if you think you have a big data problem, you probably don't, etc. I think this especially detrimental in the government sector because the folks buying the services don't always understand the technology or the maintenance requirements that come with it. Then we all foot the bill (through taxes) for a lot of OJT for said new tech.

tootie 11y ago

Once again, contracts are rarely for just development. It likely included design, development, deployment, testing, training, accessibility testing, analytics.

spilk 11y ago

No invoices against the contract? This is only part of the picture. It's a time and materials contract which means the contract award is the "ceiling", but you have to perform work billed against it to get paid any of that. Plus, the second page details several extension options that could increase the ceiling to $1,176,280. Were those options exercised?

kevinburke 11y ago
We don't know; I've amended the post to reflect this. Thanks.

lowboy 11y ago

Why is there a ?lobsters qs on the submitted url? Seems to load fine without: https://kev.inburke.com/kevin/tsa-randomizer-app-cost-336000/ https://kev.inburke.com/kevin/tsa-randomizer-app-cost-336000...

kevinburke 11y ago
I submitted it to Lobste.rs with that on the end of the URL, and someone must have submitted the URL without stripping that :)

swanson 11y ago

Let's assume a hefty markup for the long sales/BD time for a government contract like this: $336k * 0.60 = $200k Since this deals with national security, could probably justify higher rate resources staffing (security clearance required?). Ballpark $250/hr @ 8 hr/day so $200k buys you 100 days of capacity. Project breakdown: * UI/UX Design: 10 days (lots of stakeholders+approvals needed, maybe field testing with TSA agents, accessibility audit) * Development: 15 days (this is very conservative but might be some business rules that aren't being shared -- remote monitoring, analytics, auditing, static analysis, etc) * Project Management: 25 days * QA: 15 days (multiple devices, high perceived security exposure) * Third-party security audit/pen testing: 5 days * Technical writer for documentation and training materials: 5 days Total: 75 days of capacity spent. Math doesn't work out exactly and some of these numbers are pulled from thin air, but it seems like it's at least in the right ballpark. Do you folks really know people that are doing government contract work (for the TSA no less) for less than $100k?

AndyNemmity 11y ago
PM would be higher than that. There'd also be a Manager above the PM scheduled some days as well. And the security, analytics, and monitoring are probably all smaller specifics days for individual experts. Point is though, you're thinking the right way on it. Very reasonable.
- swanson 11y ago
  Good point, added a couple more line items.
joering2 11y ago
No, just like I don't believe any reasonable company could be allowed this contract for less money. Having to dealt enough with IBM on contract levels, I rather believe there has been bribes involved, but I can't prove anything.
- AndyNemmity 11y ago
  I live in this world, and have been in the upper levels of all sorts of these discussions regularly. I've never seen a direct bribe. If you want to call lavish dinners, drinking, and entertainment bribes, okay, that happens constantly. But a bribe? No... You don't need bribes. It's quid pro quo all of the time, but bribes?! No... Maybe you can get hired at a higher pay scale/job title from your work getting someone a deal. There are certainly perks.. but not bribes...

jsprogrammer 11y ago

The page (2 of 8) that had the unit prices censored (exception (b)(4))[0], claims the total award amount was $1,176,280.72. Really, what we need is IBM's invoices and TSA's pay statements. Also, you shouldn't need to file an FOIA request to get this information. Shouldn't all bids and contracts be public? [0] Exemption (b)(4): Records that contain trade secrets and commercial or financial information obtained from a person that is privileged or confidential.

AndyNemmity 11y ago
Unit prices are trade secrets. If they published that, then someone else getting a higher price could point to it, and say they only charged X for them.

kra34 11y ago

Yo app got a $1,000,000 from Andreessen Horowitz, so they kind of got a deal if you think about it that way

brandonmenc 11y ago

If you're a consultant or freelancer, and you're not charging that much to build an equivalent app, you are undercharging. This is a completely reasonable price. Honestly, it seems cheap.

mikeryan 11y ago

A few things. 1. Kevin you may want to block out your address in the attached letter. 2. I'm missing the part where this amount is tied directly to the creation of this application and this application only. From my read this is a T&M contract for mobile development there's nothing specific to this application. It could span multiple engagements and apps. Considering this contract went into place over 2 years ago this could likely be the case. 3. This is a cap not an invoice. They could have billed 60k.

sailfast 11y ago
Some details of past IBM wins here via UsASpending: https://www.usaspending.gov/Transparency/Pages/AwardSummary.aspx?AwardID=17381939 https://www.usaspending.gov/Transparency/Pages/AwardSummary.... This may be the preceding contract but very similar scope and line items if you want more details, no FOIA required.

intrasight 11y ago

What is WAY worse is that they probably purchased 10,000 ipads to run this stupid app.

jonhohle 11y ago

A coworker and I were laughing about this at the airport one day. I literally wrote the app in the flight home and almost put it up on the App Store.

jqm 11y ago

Ya well, that's 10K development, 30K profit and 290K meeting time and paying people to fill out forms. Sounds about right.

ThinkBeat 11y ago

With a reference to another recent hackernews story: https://news.ycombinator.com/item?id=11415747 https://news.ycombinator.com/item?id=11415747 Can we expect to see the source code for this application pretty soon?

bbcbasic 11y ago

But no one got fired for choosing IBM

FriedPickles 11y ago
I came here to say this.
- bbcbasic 11y ago
  Pfft! Yet another cliché slinger

kumarski 11y ago

Less worried about that and more worried about the fact that baggage scanners only scan from one angle combined with reality that they don't do material differentiation. Z-effective scores...findable.

transfire 11y ago

Hmmm... at the current rate of bureaucratic growth, I estimate in a 100 years or so, this app (or it's modern AR equivalent) will cost 20% of the entire federal budget. No worries though, 10% of all U.S. workers will be involved in its digital-paper-pushing development.

woodman 11y ago

Before we even go into line item mode we really need to ask ourselves if this is a $336k problem in the first place. They could have just asked the military how they handle this sort of problem, and the solution they would have been given is this: a $2 hand tally counter. Establish what the desired flow rate is, divert every nth person, reset the clicker, anybody in line observed shifting position gets diverted. Unless the TSA can't even trust their people to count, or their real objective is adding some weirdly alienating layer of technology between the employee and the traveler.

jcrawfordor 11y ago
"every nth person" selection is just too predictable. You could try to prevent people shifting in line, but in a crowded airport you will never be able to control that effectively at current staffing levels. The cost of effectively enforcing that rule would rapidly add up to more than $300k and cause serious inconvenience to travelers.
- mikeash 11y ago
  Yes, preventing shifting in line doesn't help if your attacker can just count the length of the line before they get in it. You really want a random element. Anyone familiar with Dungeons and Dragons should be able to name the obvious solution there....
  - woodman 11y ago
    The uncertainty already inherent to the existing process is high enough in my opinion, either the software or the hand counter would equally meet the real objective here: providing a point to audit for reportable metrics. While a dungeon master presiding over a hopeful throw for critical hits would be a solution, I don't think the TSA wants to make security theater entertaining - it makes the accidental nature of the whole endeavor obvious.
  - mfukar 11y ago
    But Mike, what good would Polymorph do in this scenario?

pratheekrebala 11y ago

The $336K appears to be just the cost of one installment. The total project costed $1,444,315 Here are the rest of the transactions under the same contract: https://www.usaspending.gov/Pages/AdvancedSearch.aspx?k=HSTS0313JCIO494 https://www.usaspending.gov/Pages/AdvancedSearch.aspx?k=HSTS... TSA spends most of it's budget through a provision called "Other Transaction Authority" which is essentially a vehicle to make purchases with barely any oversight from congress. http://time.com/4134368/tsa-price-of-security/ http://time.com/4134368/tsa-price-of-security/ (Paywall)

jdmichal 11y ago
I'm glad you brought this up, because this comment represents a misunderstanding in how government contracting is typically done. My previous job was working for a government contractor, and we did multiple discrete contracts for several branches of the government. However, every single one of them would have shown up exactly like this, all bundled under one contract. Why? Because writing and signing a contract is expensive, for both the contractor and the government. So contracts are typically written in a way to make them very easy to extend, and existing contracts are often used as vehicles to tack on additional funding for new contracts. The original contracting agency would also usually charge a fee to the other agencies for use of their contract in this way. So, this kind of glance at individual awards under a single contract is really too simple a view. A lot of these funds could be (and from my guess, probably are) going to completely independent projects from the randomizer. TL;DR: Having multiple awards under a single contract is typically a sign of the government working around bureaucracy and attempting to save costs.
- pratheekrebala 11y ago
  Yeah -- you are correct. TSA later reported that the actual cost of the randomizer was around $47k. The total figure I had mentioned earlier was part of a larger contract with IBM. I saw the generic "IGF::CT::IGF MOBILE APPLICATION DEVELOPMENT" note on the award and assumed it was all for the same project. It's also weird that TSA's response to OP's FOIA request cited a different and higher figure (~$340k) for the randomizer app -- I'm guessing they were grouping in other projects under the same award here too? Besides, $47k seems like a reasonable amount if that figure includes training and deployment costs.

wkoszek 11y ago

It'd be interesting to hear what do "materials" can entail. It might be that they actually delivered iPads to the airport.

brokentone 11y ago

We're talking about the app with 1 arrow on the screen that responds to a press anywhere on the same screen? 1 icon, 1 tap target, hooked up to 1 secure random algorithm. I don't know how anyone can look at this situation and not see the government is wasting a lot of money in their pursuit of not understand technology.

cantrevealname 11y ago

I suspect that one secret feature of this app is a discreet way to override the randomizer and send a passenger to the "intrusive search" lane. That is, if the screener suspects someone, he or she can swipe or click in a clandestine way to guarantee that the passenger gets the annoying search. The passenger can't complain about being targeted or profile or singled out, because, hey, it's random. (The actual rationale of the randomizer might have been to avoid accusations of profiling, while still being able to do profiling. Leaving the bad guys off balance is simply a benefit.)

blakecaldwell 11y ago

Why not just have TSA agents flip a coin, or buy a bunch of Trouble board games, and repurpose the dice rolling bubble?

EdHominem 11y ago
The trouble bubble is not only a perfect solution for this, but an apt metaphor for the whole process.

kevin_thibedeau 11y ago

It seems a bingo spinner with two colors of balls would do just as good of a job. https://www.google.com/search?q=bingo+spinner&tbm=isch https://www.google.com/search?q=bingo+spinner&tbm=isch

GreaterFool 11y ago

A little bit tangential: when you go to the slower lane and they swab you with those silly machines, can they actually detect anything?

EdHominem 11y ago
fyi, this is not a time to apply science.

joeld42 11y ago

What I want to know is does it still work if they turn on the rotation lock and hold the iPad upside down?

tn13 11y ago

Why does this surprise anyone? Forget an app. Just a normal computer that government buys ends up paying 1.5 of what you will get on Amazon.com. This is because government does not pay on time, there is a lengthy and resource consuming process in selling something to the government. The app was probably $10k, rest was the compliance cost.

makecheck 11y ago

One-sided negotiation is a cost riser. The last thing a client should be able to tell is that the payer has very deep pockets; if I know you can afford billions, I might ask a ridiculous price and expect you to not know any better or not care about paying it. An excellent use of secrecy technologies would be government contracts: partition problems into pieces that don’t necessarily reveal the final purpose or customer (e.g. “choose uniformly between A and B” is too vague to be guessed as a requirement from only a government agency). Then, anonymously ask for bids. That way, you might actually get the developer who offers a measly price for a trivial task, and only finds out later that his check is from the government.

deleted 11y ago

[deleted]

omk 11y ago

Q: How many IBM employees does it take to change the light bulb? A: Don't know. But will cost $1M+. They will pay for the bulb.

satyajeet23 11y ago

random rand = new random(); int tsaNumber = rand.nextInt();