7 ms·

Wow, what a cool project to work on: working out the logic flows in assembler and breaking them out so that they can be translated into a higher-level language.

Wow, what a cool project to work on: working out the logic flows in assembler and breaking them out so that they can be translated into a higher-level language. I've always thought that this type of government work should be open to bids from anyone. If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity. The fact that only very large, well-connected corporations get a crack at these types of problems is insane and a complete waste of taxpayer money.

jacquesm 9y ago
The big trick is the proposal itself. Writing that is only possible if you are already tied in at all kinds of levels. I've seen some of these tenders up close, the companies that land the deals submit phone book sized proposals to tenders that are officially open but actually closed unless you are in a very select circle already. It's not uncommon for the proposal writer to then pass on the actual work to a whole slew of subcontractors at substantially lower rates.
- ryanmarsh 9y ago
  Are you saying this is how contracts are awarded on FedBiz Ops? I know a handful of small players, myself included who have won IT contracts with the gov't and it is not at all like you stated.
  - jacquesm 9y ago
    Small contracts are an entirely different matter. Try bagging something >> $1M or defense related. I've worked for the government on small jobs here in the EU a couple of times, as long as you stay below a certain amount you can bypass a ton of requirements. But once you go above that threshold the number of players drops very rapidly and there is no escaping the formal process. If you are capable of selling large contracts to the US Federal Government without having to submit a typical bid proposal then you are in a very fortunate position, but the people that I know that have done this in the past have all worked on stuff that either required their fairly unique skills or they were working on extremely small contracts (< $250K).
    - abraae 9y ago
      The intestinal fortitude needed to prepare a proposal for these mega projects is enormous. I always end up with a gut churning feeling that no one is even going to read my work (which is interrupting any work on real projects), or that some back hander will result in a not level playing field.
      - pavel_lishin 9y ago
        Do people read them? If it's really a phone-sized book, I assume that there's some sort of algorithm, whether run by a person or a computer, that is just looking for things to check off a list.
        toomuchtodo 9y ago
        There is no algorithm or computer looking at it. It’s essentially a contract, to be looked at by lawyers and project sponsors.
        gbacon 9y ago
        Sometimes from debriefs, it is clear that they at best skimmed. Therefore the job of the proposal writer is to make the evaluator's job easy and to make the reader eager to turn the page, e.g., no walls of text just like on reddit or Hacker News.
    - woobar 9y ago
      Here is one of the two young entrepreneurs (19 and 23 years old) that secured ~$300M DoD contract. So obviously, it is not as bad. ;) In 2005, Packouz (23 years old at the time) joined Efraim Diveroli (19 years old at the time) in Diveroli's arms company AEY Inc. By the end of 2006, the company had won 149 contracts worth around $10.5 million.[1] In early 2007, AEY secured a nearly $300 million U.S. government contract to supply the Afghan Army with 100 million rounds of AK-47 ammunition, millions of rounds for SVD Dragunov sniper rifles, aviation rockets and other munitions https://en.wikipedia.org/wiki/David_Packouz https://en.wikipedia.org/wiki/David_Packouz EDIT: added quote from Wikipedia
  - robertha 9y ago
    I just registered on SAM and submitted a bid (something unrelated to this), but no idea what I was doing, I mean I put together a plan, had meaningful past performance, but overall I'm just hoping for the best. Would you happen to have any resources you followed? Or, suggestions that you believe led you to submitting a successful bid?
    - gbacon 9y ago
      Was your bid on the recent SBIR/STTR solicitation by chance?
      - robertha 9y ago
        Custom database product for EPA, seemed like a good fit. While I have no connections to EPA, I do some collaborative work with bio-engineering department at a university in Chicago, I'm a data scientist/developer/have a small team, I thought I'd try dipping my toes in something new.
        gbacon 9y ago
        Any technical people you were able to have discussions with in preparing your bid would be good places to start with follow-on conversations. An acquaintance with a setup similar to yours does well with a virtuous cycle of rolling SBIR and STTR results into his commercial products, which fuel more SBIR wins. He also does lots of legwork in the form of hand delivering white papers he’s written in office calls with customers and potential customers during site visits. People do business with people — particularly those we like, know, and trust — not companies and agencies. Find someone whose headache you can make go away. Keep the conversation moving. This is a patient person’s game. Sometimes you’re planting seeds that will bloom later.
    - efm 9y ago
      There are Procurement Technical Assistance centers in every region of the country. They are staffed by former procurement officers, and help companies learn how to sell to government and prime contractors. http://www.aptac-us.org/ http://www.aptac-us.org/
  - jseliger 9y ago
    Are you saying this is how contracts are awarded on FedBiz Ops? I do grant writing for nonprofit and public agencies, along with some research-based businesses, and people interested in getting into this sort of thing will contact us, or people like us. We're a little like lawyers: it is possible to do everything right, but it's very hard and unlikely if it's your first rodeo. To take one small example: when you submit a budget, is it a program budget or a total project budget? Err and your application may be disqualified. Or consider indirect cost rates: http://seliger.com/2016/05/16/federally-approved-indirect-cost-rates-developing-federal-grant-proposal-budgets-de-minimus-arrived http://seliger.com/2016/05/16/federally-approved-indirect-co... . Again, either of these things are small, but multiply them x1000 and suddenly you'll understand why organizations hire us!
  - JPKab 9y ago
    I'm sure you are aware of the 8a and other programs that allow you to win small contracts. If you think big projects are given on merit, then you are naive.
- TimJYoung 9y ago
  Thanks, I suspected that this was the case after I read the Cringely book on IBM (https://www.amazon.com/Decline-Fall-IBM-American-Icon-ebook/dp/B00KRHWZ22 https://www.amazon.com/Decline-Fall-IBM-American-Icon-ebook/...). It gives one the distinct impression that connections matter a lot, and that getting the contract matters more than actually completing it successfully.
  - JPKab 9y ago
    This is too true. The career path at my former defense contracting gig went from software engineer to someone who writes proposals. They clearly valued winning work over executing it. Rent seeking economics at work.
  - pjmlp 9y ago
    This applies to all big corporations, not only government related projects.
  - kjs3 9y ago
    The conclusion isn't necessarily wrong, but Cringley is the worst source here. Do a little research about how long he's been predicting Absolutely, Positively (because he's such an insider) IBM is going out of business. Still wrong. I understand now he's guaranteeing Apple is going down Real Soon Now. Don't sell your stock just yet.
- gbacon 9y ago
  Almost no one wins submitting proposals out of the blue, certainly the case with unsolicited proposals but also with responses to formal RFPs. Firing in a proposal with no agency contact, no familiarity ("customer intimacy"), no shaping, no premarketing, no office calls, no digging to understand their real pain points is even worse than a cold call because, as you noted, it is horrendously more expensive and painful to prepare a large proposal.
- innagadadavida 9y ago
  This seems like a problem of trust and the IRS has already chosen whom it trusts based on past work done etc. Seems unfair, but how would you be able to trust someone whom you’ve never done business with earlier? Could these be translated to laws to prevent this type of exclusion?
FigBug 9y ago
I would disagree. On of my first jobs was porting assembler to C for a paging switch. The assembler was well written and well documented. The task was slow, hard and painfully boring. It took months to get every function to be a perfect match.
- nickspacek 9y ago
  Different strokes. Although, given a long enough period of working through assembly it might break the strongest of wills. I can imagine some people being very excited at setting up tests to ensure compatibility and watching them slowly going from fail to pass.
- TimJYoung 9y ago
  Did you manually convert the assembler, or automate it ? With regard to manual conversions, I'm with you 100% - no thanks. :-) But, my understanding is that the bulk of the work done in this case was done by a conversion tool that was designed and developed by a group of 8 people led by the gentleman referred to in the article (Jian Wang). The conversion tool project was the work that I was referring to in my comment.
  - FigBug 9y ago
    Yes, manually. I missed that distinction, building the tool could be fun, but verifying it works would be a huge pain. Shouldn't be too hard to write test cases for tax code.
    - theptip 9y ago
      > Shouldn't be too hard to write test cases for tax code. I would disagree here -- like timezones, tax software has the disadvantage of being both extremely boring, and very fiddly (thus requiring close attention). This combination makes it very hard to maintain the concentration that's required to write exhaustive tests.
- hannibalhorn 9y ago
  Manually, sure, but it sounds like the idea is/was to use an automated approach. I imagine many of the same techniques used by modern decompilers could be applied, and there's probably lots of information in assembly that is lost in machine code (e.g., JE vs JZ on x86). One could also assume some knowledge of the coding conventions used by the organization. Even a tool that could translate 90% automatically, leaving the rest to be done manually, would work out pretty well. In some ways it reminds me of the project to programatically translate the golang compiler from C to Go. Though I imagine the codebase is quite a bit messier!
yetanotheruser 9y ago
Yeah, sounds like a cool project!
ww520 9y ago
Rather than transcode assembly to a higher-level language. It's better to nail down all the functionality the assembly program provide and re-implement them in the higher-level language. Write language independent tests against the old code and re-run the tests against the new code to make sure things are working feature for feature and bug for bug.
- makmanalp 9y ago
  > nail down all the functionality the assembly program This often proves to be much harder than expected which is /why/ people almost never touch systems like this where the creators are long gone
  - ww520 9y ago
    It's the most difficult part of the project. However, nailing down the functionality upfront separates a successful project from a failed one. The implementation is the easy part. The functionality documentation will provide great value beyond this project. I assume this assembly program does the tax calculation, so the IRS tax code can be consulted to verify the functionality. Functional tests can be written against the old code as the functionality is documented. The functional tests will guide the new implementation development, and serve as the acceptance tests of the new code. You can even structure the process such that there's a team doing just the functional specification and functional test writing, and another team takes the result fed to them and does the implementation in parallel.
    - makmanalp 9y ago
      > It's the most difficult part of the project I'll agree with that heartily. > the IRS tax code can be consulted to verify the functionality You're not wrong, but the problem is often not that. The tax code helps when verifying the system end to end. But it's not like there's one single program that "does the tax" that you can verify that way. The individual bits they're trying to replace are probably more like "fetch all these records from this one mainframe with format X and convert them this way, except if it's Feb 29th in which case fetch it from this other server and convert it another way and then pass it along, except for resident aliens which come from an entirely different place, fetch those from tape storage". This is much harder to get right without formal specs, which there almost certainly aren't. In this case what we have is a "reference implementation" (a.k.a the implementation is the specification), and you can guess how well those go. All this is not to say that they shouldn't have been doing what you're suggesting, but to say that you're making it sound easier than it is now that they're here.
jordan801 9y ago
Lobbyists usually thwart any opportunity for someone cheap and legitimate to do the job. They've been trying to get someone to replace the humvee for decades and usually it's a 400 million dollar endeavor that results in a lesser product. It's like these large companies take on the work, then instead of doing it they hire lawyers to figure out loopholes in the contract. Then they just pay lobbyists to get politicians to award them more contracts. All the while doing just enough to not get sued into oblivion. All great ideas are spoiled by great bureaucrats and even greater lobbyists.
rbanffy 9y ago
> working out the logic flows in assembler Not only assembler, but IBM mainframe assembler. That's way cooler than x86.
lazaroclapp 9y ago
> If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity. Not that I disagree with the general idea of democratizing this sort of big government contract, but, well, this is a much bigger hurdle than you are making it sound like. Audits can be good at finding unintentional flaws, but a skilled adversary can often create code that looks safe, but in fact, isn't. Consider the underhanded C contest: http://www.underhanded-c.org/ http://www.underhanded-c.org/ On a personal note: I once worked on a large research project for detecting malicious behavior on Android apps, where the idea was to produce tooling to find underhanded/undocumented behavior automatically. The project had a red team. By the final rounds, we were getting code from them where the malicious functionality was extremely hard to find via either tooling or manual inspection. Keep in mind these were simple programs, rarely over 10k lines of code, written in Java which is a remarkably transparent language to read, and that we were allowed to ban features like reflection or raise the alarm on anything that looked like intentionally obfuscated code. Additionally, we knew each of those programs had malware in them. Reporting 'clean' was often just saying 'you win, we give up, couldn't find it after staring at this 2k lines file for a week'. In theory, when working with a large contractor, you can put controls on how the software is built, not just the final code, and you can hold them accountable for backdoors discovered long after the fact. Now, not saying this always works that way, but it might still be better than accepting a system built by someone you only know from their Github handle and who might or might not live within your jurisdiction. The state of the art in software verification would need to change a great deal before that is a good idea.
- ScottBurson 9y ago
  I don't suppose those Java examples were published? I'd love to see them.
  - lazaroclapp 9y ago
    Unfortunately, I don't believe they were.
  - shagie 9y ago
    How about a nice, simple, 2 + 2 = 5? https://codegolf.stackexchange.com/a/28818 https://codegolf.stackexchange.com/a/28818
empath75 9y ago
That’s actually not true. I work for a fairly small contractor (a few hundred people) working on a fairly large government contract. It started with a handful of people when the company was a fraction of the size.
- TimJYoung 9y ago
  I'm very happy to be proven wrong. I must admit that I know very little about the process, other than what I've read, and what I've read hasn't been positive. But, I'm sure that small, successful projects aren't talked about nearly as much in the press as large, unsuccessful projects, so it could simply be bias in what I'm reading.

jacquesm 9y ago

The big trick is the proposal itself. Writing that is only possible if you are already tied in at all kinds of levels. I've seen some of these tenders up close, the companies that land the deals submit phone book sized proposals to tenders that are officially open but actually closed unless you are in a very select circle already. It's not uncommon for the proposal writer to then pass on the actual work to a whole slew of subcontractors at substantially lower rates.

ryanmarsh 9y ago
Are you saying this is how contracts are awarded on FedBiz Ops? I know a handful of small players, myself included who have won IT contracts with the gov't and it is not at all like you stated.
- jacquesm 9y ago
  Small contracts are an entirely different matter. Try bagging something >> $1M or defense related. I've worked for the government on small jobs here in the EU a couple of times, as long as you stay below a certain amount you can bypass a ton of requirements. But once you go above that threshold the number of players drops very rapidly and there is no escaping the formal process. If you are capable of selling large contracts to the US Federal Government without having to submit a typical bid proposal then you are in a very fortunate position, but the people that I know that have done this in the past have all worked on stuff that either required their fairly unique skills or they were working on extremely small contracts (< $250K).
  - abraae 9y ago
    The intestinal fortitude needed to prepare a proposal for these mega projects is enormous. I always end up with a gut churning feeling that no one is even going to read my work (which is interrupting any work on real projects), or that some back hander will result in a not level playing field.
    - pavel_lishin 9y ago
      Do people read them? If it's really a phone-sized book, I assume that there's some sort of algorithm, whether run by a person or a computer, that is just looking for things to check off a list.
      - toomuchtodo 9y ago
        There is no algorithm or computer looking at it. It’s essentially a contract, to be looked at by lawyers and project sponsors.
      - gbacon 9y ago
        Sometimes from debriefs, it is clear that they at best skimmed. Therefore the job of the proposal writer is to make the evaluator's job easy and to make the reader eager to turn the page, e.g., no walls of text just like on reddit or Hacker News.
  - woobar 9y ago
    Here is one of the two young entrepreneurs (19 and 23 years old) that secured ~$300M DoD contract. So obviously, it is not as bad. ;) In 2005, Packouz (23 years old at the time) joined Efraim Diveroli (19 years old at the time) in Diveroli's arms company AEY Inc. By the end of 2006, the company had won 149 contracts worth around $10.5 million.[1] In early 2007, AEY secured a nearly $300 million U.S. government contract to supply the Afghan Army with 100 million rounds of AK-47 ammunition, millions of rounds for SVD Dragunov sniper rifles, aviation rockets and other munitions https://en.wikipedia.org/wiki/David_Packouz https://en.wikipedia.org/wiki/David_Packouz EDIT: added quote from Wikipedia
- robertha 9y ago
  I just registered on SAM and submitted a bid (something unrelated to this), but no idea what I was doing, I mean I put together a plan, had meaningful past performance, but overall I'm just hoping for the best. Would you happen to have any resources you followed? Or, suggestions that you believe led you to submitting a successful bid?
  - gbacon 9y ago
    Was your bid on the recent SBIR/STTR solicitation by chance?
    - robertha 9y ago
      Custom database product for EPA, seemed like a good fit. While I have no connections to EPA, I do some collaborative work with bio-engineering department at a university in Chicago, I'm a data scientist/developer/have a small team, I thought I'd try dipping my toes in something new.
      - gbacon 9y ago
        Any technical people you were able to have discussions with in preparing your bid would be good places to start with follow-on conversations. An acquaintance with a setup similar to yours does well with a virtuous cycle of rolling SBIR and STTR results into his commercial products, which fuel more SBIR wins. He also does lots of legwork in the form of hand delivering white papers he’s written in office calls with customers and potential customers during site visits. People do business with people — particularly those we like, know, and trust — not companies and agencies. Find someone whose headache you can make go away. Keep the conversation moving. This is a patient person’s game. Sometimes you’re planting seeds that will bloom later.
  - efm 9y ago
    There are Procurement Technical Assistance centers in every region of the country. They are staffed by former procurement officers, and help companies learn how to sell to government and prime contractors. http://www.aptac-us.org/ http://www.aptac-us.org/
- jseliger 9y ago
  Are you saying this is how contracts are awarded on FedBiz Ops? I do grant writing for nonprofit and public agencies, along with some research-based businesses, and people interested in getting into this sort of thing will contact us, or people like us. We're a little like lawyers: it is possible to do everything right, but it's very hard and unlikely if it's your first rodeo. To take one small example: when you submit a budget, is it a program budget or a total project budget? Err and your application may be disqualified. Or consider indirect cost rates: http://seliger.com/2016/05/16/federally-approved-indirect-cost-rates-developing-federal-grant-proposal-budgets-de-minimus-arrived http://seliger.com/2016/05/16/federally-approved-indirect-co... . Again, either of these things are small, but multiply them x1000 and suddenly you'll understand why organizations hire us!
- JPKab 9y ago
  I'm sure you are aware of the 8a and other programs that allow you to win small contracts. If you think big projects are given on merit, then you are naive.
TimJYoung 9y ago
Thanks, I suspected that this was the case after I read the Cringely book on IBM (https://www.amazon.com/Decline-Fall-IBM-American-Icon-ebook/dp/B00KRHWZ22 https://www.amazon.com/Decline-Fall-IBM-American-Icon-ebook/...). It gives one the distinct impression that connections matter a lot, and that getting the contract matters more than actually completing it successfully.
- JPKab 9y ago
  This is too true. The career path at my former defense contracting gig went from software engineer to someone who writes proposals. They clearly valued winning work over executing it. Rent seeking economics at work.
- pjmlp 9y ago
  This applies to all big corporations, not only government related projects.
- kjs3 9y ago
  The conclusion isn't necessarily wrong, but Cringley is the worst source here. Do a little research about how long he's been predicting Absolutely, Positively (because he's such an insider) IBM is going out of business. Still wrong. I understand now he's guaranteeing Apple is going down Real Soon Now. Don't sell your stock just yet.
gbacon 9y ago
Almost no one wins submitting proposals out of the blue, certainly the case with unsolicited proposals but also with responses to formal RFPs. Firing in a proposal with no agency contact, no familiarity ("customer intimacy"), no shaping, no premarketing, no office calls, no digging to understand their real pain points is even worse than a cold call because, as you noted, it is horrendously more expensive and painful to prepare a large proposal.
innagadadavida 9y ago
This seems like a problem of trust and the IRS has already chosen whom it trusts based on past work done etc. Seems unfair, but how would you be able to trust someone whom you’ve never done business with earlier? Could these be translated to laws to prevent this type of exclusion?

FigBug 9y ago

I would disagree. On of my first jobs was porting assembler to C for a paging switch. The assembler was well written and well documented. The task was slow, hard and painfully boring. It took months to get every function to be a perfect match.

nickspacek 9y ago
Different strokes. Although, given a long enough period of working through assembly it might break the strongest of wills. I can imagine some people being very excited at setting up tests to ensure compatibility and watching them slowly going from fail to pass.
TimJYoung 9y ago
Did you manually convert the assembler, or automate it ? With regard to manual conversions, I'm with you 100% - no thanks. :-) But, my understanding is that the bulk of the work done in this case was done by a conversion tool that was designed and developed by a group of 8 people led by the gentleman referred to in the article (Jian Wang). The conversion tool project was the work that I was referring to in my comment.
- FigBug 9y ago
  Yes, manually. I missed that distinction, building the tool could be fun, but verifying it works would be a huge pain. Shouldn't be too hard to write test cases for tax code.
  - theptip 9y ago
    > Shouldn't be too hard to write test cases for tax code. I would disagree here -- like timezones, tax software has the disadvantage of being both extremely boring, and very fiddly (thus requiring close attention). This combination makes it very hard to maintain the concentration that's required to write exhaustive tests.
hannibalhorn 9y ago
Manually, sure, but it sounds like the idea is/was to use an automated approach. I imagine many of the same techniques used by modern decompilers could be applied, and there's probably lots of information in assembly that is lost in machine code (e.g., JE vs JZ on x86). One could also assume some knowledge of the coding conventions used by the organization. Even a tool that could translate 90% automatically, leaving the rest to be done manually, would work out pretty well. In some ways it reminds me of the project to programatically translate the golang compiler from C to Go. Though I imagine the codebase is quite a bit messier!

yetanotheruser 9y ago

Yeah, sounds like a cool project!

ww520 9y ago

Rather than transcode assembly to a higher-level language. It's better to nail down all the functionality the assembly program provide and re-implement them in the higher-level language. Write language independent tests against the old code and re-run the tests against the new code to make sure things are working feature for feature and bug for bug.

makmanalp 9y ago
> nail down all the functionality the assembly program This often proves to be much harder than expected which is /why/ people almost never touch systems like this where the creators are long gone
- ww520 9y ago
  It's the most difficult part of the project. However, nailing down the functionality upfront separates a successful project from a failed one. The implementation is the easy part. The functionality documentation will provide great value beyond this project. I assume this assembly program does the tax calculation, so the IRS tax code can be consulted to verify the functionality. Functional tests can be written against the old code as the functionality is documented. The functional tests will guide the new implementation development, and serve as the acceptance tests of the new code. You can even structure the process such that there's a team doing just the functional specification and functional test writing, and another team takes the result fed to them and does the implementation in parallel.
  - makmanalp 9y ago
    > It's the most difficult part of the project I'll agree with that heartily. > the IRS tax code can be consulted to verify the functionality You're not wrong, but the problem is often not that. The tax code helps when verifying the system end to end. But it's not like there's one single program that "does the tax" that you can verify that way. The individual bits they're trying to replace are probably more like "fetch all these records from this one mainframe with format X and convert them this way, except if it's Feb 29th in which case fetch it from this other server and convert it another way and then pass it along, except for resident aliens which come from an entirely different place, fetch those from tape storage". This is much harder to get right without formal specs, which there almost certainly aren't. In this case what we have is a "reference implementation" (a.k.a the implementation is the specification), and you can guess how well those go. All this is not to say that they shouldn't have been doing what you're suggesting, but to say that you're making it sound easier than it is now that they're here.

jordan801 9y ago

Lobbyists usually thwart any opportunity for someone cheap and legitimate to do the job. They've been trying to get someone to replace the humvee for decades and usually it's a 400 million dollar endeavor that results in a lesser product. It's like these large companies take on the work, then instead of doing it they hire lawyers to figure out loopholes in the contract. Then they just pay lobbyists to get politicians to award them more contracts. All the while doing just enough to not get sued into oblivion. All great ideas are spoiled by great bureaucrats and even greater lobbyists.

rbanffy 9y ago

> working out the logic flows in assembler Not only assembler, but IBM mainframe assembler. That's way cooler than x86.

lazaroclapp 9y ago

> If the government is worried about trojans, malware, etc., then they can easily hire an auditor to audit the code and vouch for its authenticity. Not that I disagree with the general idea of democratizing this sort of big government contract, but, well, this is a much bigger hurdle than you are making it sound like. Audits can be good at finding unintentional flaws, but a skilled adversary can often create code that looks safe, but in fact, isn't. Consider the underhanded C contest: http://www.underhanded-c.org/ http://www.underhanded-c.org/ On a personal note: I once worked on a large research project for detecting malicious behavior on Android apps, where the idea was to produce tooling to find underhanded/undocumented behavior automatically. The project had a red team. By the final rounds, we were getting code from them where the malicious functionality was extremely hard to find via either tooling or manual inspection. Keep in mind these were simple programs, rarely over 10k lines of code, written in Java which is a remarkably transparent language to read, and that we were allowed to ban features like reflection or raise the alarm on anything that looked like intentionally obfuscated code. Additionally, we knew each of those programs had malware in them. Reporting 'clean' was often just saying 'you win, we give up, couldn't find it after staring at this 2k lines file for a week'. In theory, when working with a large contractor, you can put controls on how the software is built, not just the final code, and you can hold them accountable for backdoors discovered long after the fact. Now, not saying this always works that way, but it might still be better than accepting a system built by someone you only know from their Github handle and who might or might not live within your jurisdiction. The state of the art in software verification would need to change a great deal before that is a good idea.

ScottBurson 9y ago
I don't suppose those Java examples were published? I'd love to see them.
- lazaroclapp 9y ago
  Unfortunately, I don't believe they were.
- shagie 9y ago
  How about a nice, simple, 2 + 2 = 5? https://codegolf.stackexchange.com/a/28818 https://codegolf.stackexchange.com/a/28818

empath75 9y ago

That’s actually not true. I work for a fairly small contractor (a few hundred people) working on a fairly large government contract. It started with a handful of people when the company was a fraction of the size.

TimJYoung 9y ago
I'm very happy to be proven wrong. I must admit that I know very little about the process, other than what I've read, and what I've read hasn't been positive. But, I'm sure that small, successful projects aren't talked about nearly as much in the press as large, unsuccessful projects, so it could simply be bias in what I'm reading.