20 ms·

John Carmack on expert witnesses and 'non literal' copying

505 points by samlittlewood 10y ago

kchoudhu 10y ago
Well, who was the expert witness?
potatoman2 10y ago
Loser's lament.
vanattab 10y ago
Isn't the testimony already a matter of public record?
- delinka 10y ago
  He mentions that it's not. "... his expert testimony in trial is under seal, rather than in the public record."
trendia 10y ago
This struck out at me: > There are objective measures of code similarity that can be quoted, like the edit distance between abstract syntax trees ... If this became the primary legal metric, then programmers who stole code would change the code so that functions achieved the same output with practically no AST similarity. That is, they could maximize functional similarity while minimzing code similarity. This would be a large task, but one that might be easier and preferred to writing code from scratch, since you drastically reduce the trial and error process of writing code in a field where best practices aren't currently known. But, while it's possible to game any 'objective' metric like AST distance, maybe that's preferable to the subjective and incentivized claims of an expert witness, who will be biased in favor of the party paying them. And while it's easy to get lost in the paragraphs of expert witness testimony, a collection of objective metrics is easy to compare to previous cases.
- rocky1138 10y ago
  Someone should write a script which takes any code and mucks the AST.
  - gravypod 10y ago
    You should be able to just compile it with -O3 and then decompile it and rename all your variables back. You'll get a very wonky source output but it will be extremely different.
    - xxs 10y ago
      why would would you keep debug info like variable names while compiling w/ -O3 in the 1st place?
      - gravypod 10y ago
        You woudln't. This would be for an intial obfuscation pass. You'd not do this for release but for sufficiently scrambling the code base.
- gravypod 10y ago
  Funnily enough I actually implemented this [0] two semesters ago while working as a Teacher's Assistant for the into CS class. It worked fairly well. I basically took the levenshtein algorithm and mapped it directly to ASTs and used that to find the % of similar elements in the code. It worked a few times and I caught 1 or 2 groups of cheaters. ....sadly in that class you were allowed to "work together" you just had to include a note saying you did so I've got no "kill count" anymore and I'm not planning on TAing any time soon (at least not for python). Also that code was hacked together in a weekend and that anti-cheet was done in 2-3 hr. It's not production ready by a longshot. [0] - https://github.com/gravypod/GradeO/blob/master/libs/cheating_manager.py https://github.com/gravypod/GradeO/blob/master/libs/cheating...
  - timtadh 10y ago
squizzel 10y ago
I often wonder if John was involved with the story telling aspect of his games, not just the code.
- whatever_dude 10y ago
  I think it has a recurring theme that any time he got involved, things didn't go well so he decided to let go of it. That's not to say he didn't have influence; many times the games' stories were derived FROM the tech, not the other way around. Doom was originally supposed to be much more story- and character-driven, but that took a backseat given the focus on the development side was on the gameplay mechanics. Also, this quote: > Story in a game is like a story in a porn movie. It's expected to be there, but it's not that important From "Masters of Doom".
  - dosgonlogs 10y ago
    He was the Dungeon Master in their Dungeons & Dragons games. John Romero loved Carmack's campaign world, but John Romero destroyed it, Opening up a gateway to hell, and then immediately regretted it (but they got the idea for Doom from it), since it was a long running campaign in a world completely designed by John Carmack. I think Heretic/Hexen and Quake also borrowed heavily from their D&D games. Masters of Doom is an awesome book.
    - whatever_dude 10y ago
      Good point. I forgot about that.
- samlittlewood 10y ago
  From 'Masters of Doom' - he used to make and DM D&D campaigns.
- Bamafan 10y ago
  John Romero was largely responsible for the stories and designs of the old school iD games from back in the day. Carmack brought the tech brilliance to make it happen.
jstanley 10y ago
Link doesn't work for me, I just get a popup telling me to sign up to Facebook, and when I close it the page is blank? EDIT: Incognito mode fixed it. http://pastebin.com/KxAyvqyM http://pastebin.com/KxAyvqyM
addisonj 10y ago
I don't have the evidence so I can't make a judgement about whether Zenimax or Carmack is in the wrong here, but this does point out something strange in our court system and that is where a "jury of peers" which translates to normal people with little to no knowledge of the underlying subject matter. As our society becomes more specialized it seems a bit absurd to have people make judgements mostly based on how well each side can make incredibly complex things comprehensible and convincing without any real understanding of the underlying principles in the field. It seems likely that the best expert witness in this case is not the most correct or credentialed, but the most charismatic. It would be interesting to see how some of these big software trials (Google v Oracle) would have come out if the jury were made up of people who were both impartial and familiar with the industry.
- lmm 10y ago
  How is the court supposed to judge familiarity? What if jury members think they're technical experts, but aren't? There are always going to be complex technical matters that are relevant to lawsuits - this was the case before software even existed - and the way the system is supposed to work is that rather than the jury bringing their own set of preconceptions on the technical aspects, the expert witnesses answer the technical questions in the court (where they are subject to cross-examination etc.) and the jury makes the factual judgements that have always been their job.
- smacktoward 10y ago
  The more familiar you are with the industry, the less likely you are to be impartial.
- pavel_lishin 10y ago
  > this does point out something strange in our court system and that is where a "jury of peers" which translates to normal people with little to no knowledge of the underlying subject matter. Nothing in the constitution says anything about peers, and the fact that most juries are comprised of people who don't know much about the subject matter comes from lawyers wanting those people on the bench, since they're less likely to have pre-conceived notions - possibly notions conflicting with their clients' goals. If you're trying to convince someone of something, it helps if they start with a blank slate.
pyb 10y ago
"The expert witness circuit is surely tempting for many academics, since a distinguished expert can get paid $600+ an hour to prepare a weighty report that supports a lawyer’s case. I don’t have any issue with that, but testifying in court as an expert should be as much a part of your permanent public record as the journal papers you publish. In many cases, the consequences are significant. There should be a danger to your reputation if you are imprudent." So basically he's kind of threatening the expert witness now?
- perfmode 10y ago
  John makes a good point. Here, transparency appears to be in truth's best interest.
  - pyb 10y ago
    From the downvotes, sounds like I'm the only one who is shocked by this.
    - intherdfield 10y ago
      Could you explain why you think this is a threat aimed at the expert witness in this trial? Any time someone publishes a work, they are putting their reputation at risk. If the work is bad, the whole world will know they did it. Carmack is saying he thinks this should be the case for reports and analyses given by expert witnesses.
      - 794CD01 10y ago
        You could say the same thing about anything people do in secret. Yet many people still want privacy.
        intherdfield 10y ago
        > You could say the same thing about anything people do in secret. I don't think that makes sense. There is no attempt to generalize this to things people do in secret. There is no claim that if something could risk your reputation then it follows that it should be made public. Carmack's assertion is only that this particular work (for which the witness does voluntarily and is paid for) should be made public so that these witnesses use the same level of rigor they would for their other published work. And letting the public review it would have benefits too. (I am not agreeing or disagreeing with this.) Ultimately, if we take Carmack's statements as true, it sounds to me like Facebook/Occulus' defense did not do an adequate job of instilling doubt in the report. He wrote that the defense did a technical tear down. I think if I was a juror, I would need to see this report completely destroyed. I'd need to see the same methodology applied to works where we know there was no copying and have it find false positives.
ChuckMcM 10y ago
Interesting rant on expert witnesses. Mostly interesting because I've done some expert witness testimony and found it fascinating. I completely resonate with John's issue with the expert's testimony and his understanding of it. John mentioned that their own expert testified differently but he didn't say whether or not their lawyers attempted to impeach their expert with cross examination. In the two cases I participated in, both sides agreed ahead of time on how to "handle" experts (in terms of cross examination). In one it was documents only (no testimony) and in one I was deposed by the opposing counsel after documents in which the lawyer clearly had some notes (presumably from their expert(s)) and was trying to get me to recant or change some of my points (I didn't need to). In a jury trial I can imagine that having the jury understand your testimony is probably the most challenging. And, like John, I consider the notion of 'non-literal copying' to be pretty ridiculous. In the limit it means "you read this code, understood how it worked, and wrote new code that could do the same function." You can stretch that to cover anything you have ever seen. Which is sad.
- madgar 10y ago
  > In the limit it means "you read this code, understood how it worked, and wrote new code that could do the same function." You can stretch that to cover anything you have ever seen. Which is sad. I thought the industry adopted "clean room" techniques decades ago to avoid this kind of appearance of impropriety. It's more expensive, sure, but isn't eliminating legal risk something that typically gets paid for?
  - johnbellone 10y ago
    This particular point worries me. Are we now not going to be able to make a clean room re-implementation of concept? Do we need to worry about our previous employer suing us because we are successful?
    - madgar 10y ago
      They didn't seem to do clean room here. Seems like Carmack was wrote code for both companies which is the exact opposite of clean room. (Please correct me if I misunderstood the facts of the case)
sfifs 10y ago
The problem for John Camack was I think he was on both sides of the table writing code - so he wrote the code at Zenimax & re-wrote the code at Oculus. From a layman's perspective, he very well could have been "copying". Normally when you design clean room implementations, you use different people and make sure that the people who write the code never see the prior written code and the person giving the spec doesn't see the newly written code. They were just in-sufficiently cautious legally, possibly un-surprising given Carmack's prior history in founding Id software, and got hit with a lawsuit.
- TheCoelacanth 10y ago
  That seems like it is saying that Zenimax owns not only the code Carmack wrote while he was there, but also the knowledge and experience he gained while working there.
  - leggomylibro 10y ago
    I think it's more that Zenimax thinks it owns the unique ideas that Carmack, a top talent in his field, created while he was working there. They're not angry that he wrote the same 4x4 matrix inverse function for both them and Oculus. Or more specifically, if Zenimax's claims are correct, while he was working on company time and equipment.
    - jandrese 10y ago
      What is the alternative? Do we force everybody to undergo mind wipes whenever they leave the company so they don't take any skills they picked up on the job with them? How many people can forget everything they know every time they get a new job?
      - wmf 10y ago
        The alternative is probably non-compete contracts (which are enforceable if paid) so that people don't immediately go from one company to its direct competitor.
6stringmerc 10y ago
An interesting statement on things that may be relevant in the grand scheme of the trial, but, unless I'm really missing something here, the verdict was about busting an NDA (the Code being downstream of the NDA violation). Hence the $500M vs the $4B in damages. If it was just about code, then I'm sure Carmack's points would hold up just as well as they read in hindsight. But - and it's a big but - keep in mind his post is in a vacuum of sorts by not relating it back to the mechanics of how the case was actually ruled. Again, I might be missing something in his post, but I don't think so. As for this part: >The notion of non-literal copying is probably delicious to many lawyers, since a sufficient application of abstraction and filtering can show that just about everything is related. There are certainly some cases where it is true, such as when you translate a book into another language, but copyright explicitly does not apply to concepts or algorithms, so you can’t abstract very far from literal copying before comparing. As with many legal questions, there isn’t a bright clear line where you need to stop. Non-literal copying is why I like the "Gaye Family vs. Blurred Lines" verdict and why a wide swath of business folks and industry creatives hate it. Also, it's nearly a 1:1 if you want to talk about Code like Carmack does or Music like Musicians do. It's a totally reasonable thought experiment for one side to claim "Well, a jury of non-musicians ruled the wrong way and called it derivative" to which I'd counter "Well, a jury of expert musicians would probably rule the same way because they understand the nuances even better than the average lay person." Again, these are hypothetical arguments with real-world consequences, but I don't think these cases can be easily ruled upon. There's always going to be some hard feelings at the end of it.
- Bamafan 10y ago
  I believe the "non-literal copying" thing simply refers to the insight gained from the R&D time he spent on Occulus while still an employee of Zenimax. 90% of HN members know that this deep understanding is FAR, FAR, FAR more valuable than blindly copying over code. It was this deep understanding that made Occulus valuable and it was seemingly funded on Zenimax's time and dime.
  - wang_li 10y ago
    Perhaps it's not an issue, but it's not like Carmack learned stuff as an employee. He has been the principle architect of the graphics engine for every product he's worked on for the last 20 years. How do you take that out of your head when you change companies but stay in the same field?
Arathorn 10y ago
Having gone through a similar trial, this is painfully familiar. The reality seems to be that the legal process optimises for whoever argues the most manipulatively (using every dirty trick in the book) rather than actually optimising for truth. Perhaps this is inevitable: rather than hoping to successfully convey a logical scientific argument to the judge/jury/arbitrator about a deeply technical area, instead the lawyers find themselves painting a more subjective simplified story that the audience may relate to... at which point you have to fight like with like. It's almost a post-truth kind of situation; the actual reality is irrelevant; it's instead how well the theatrical posturing is executed or how compelling the simplified version of the story is. That said, I have no idea whether Carmack is in the right or wrong here. In my instance the system worked in my favour, but I suspect it had a lot more to do with having lawyers who were good at debating than the almost incidental fact that objectively and demonstrably I had done nothing wrong...
- abraae 10y ago
  Absolutely concur. In an arbitration case, our attorney told us beforehand that (pardon the language) "the only bigger whores than lawyers are expert witnesses". The outcomes seem to depend mainly on, as you say, which side can spin the most compelling simplified story that resonates with the arbitrator/jury. As another poster says, perhaps use of analogies should be banned (Harry Potter in this case for goodness sake).
deleted 10y ago
[deleted]
wnevets 10y ago
Expert witnesses is kinda crazy in this country. One of the most famous cases had to be the father wrongfully executed because of it[0]. Law enforcement are also trained to talk about their years of experience and how they're an expert when giving testimony regardless of facts and juries love it. [0] https://en.wikipedia.org/wiki/Cameron_Todd_Willingham https://en.wikipedia.org/wiki/Cameron_Todd_Willingham
Digory 10y ago
I suppose many weird things happen in $500m cases, but it'd be really unusual to have trial testimony sealed against the agreement of the parties. Pretrial reports and depositions, sure. Media was apparently there reporting during the expert's trial testimony, and I'd be surprised if they cannot buy a transcript from the court reporter. (http://uploadvr.com/court-oculus-zenimax-last-day/ http://uploadvr.com/court-oculus-zenimax-last-day/) A quick look at the docket shows the expert witness at issue is likely to be David Dobkin of Princeton. That name appears on the docket and in media reports. Let me know if that violates HN norms, but expert witness trial testimony is part of the public, permanent record in almost every case. If a witness is excluded for using unreliable methods, his or her value as an expert is ... diminished.
- itsdrewmiller 10y ago
  Yeah, this was the part that seemed most important and potentially most unreliable to me. I would like to see where they officially requested that this testimony be made public and were denied.
- gregw2 10y ago
  Interesting. David Dobkin is the Dean of Faculty at Princeton University who used to be a Computer Science professor there for a very long time before being Dean; his specialty was 3D geometry (not a 3D shader guy, kinda a bit more math-y than applied.) I don't know if he ever presided over code plagarism cases with clever Princeton CS undergrads but since he was department chair for a fair chunk of time, information on that may well provide interesting insight into his methodology and previous experience. (Reviewing papers for various computer graphics entities might also contain additional plagarism experience but that's unlikely to be code-comparison-type plagarism.) Given his CV I would be very surprised whether he has considered abstract syntax trees much since he got his PhD 43 years ago, but I am not convinced by Carmack that is automatically exculpatory. I would consider him a reasonable pick as an expert in 3D geometry and algorithms (although he hasn't been active in the field in 10+ years he did it for a /long/ time which is likely sufficient) and he may have dealt with plagarism of regular text during his career a moderate amount, but he does NOT seem to be an expert in code-plagarism or code similarity. It's interesting that the former is good enough and they didn't need an expert in the latter. In any case, given the positions he's been at, whatever he's getting paid, even $600+/hour, I doubt he /needs/ the money. (If he is in fact the expert witness.)
minimaxir 10y ago
I recently served on a jury in an assault and battery case. The defense offered a psychological defense and called a psychologist as an expert witness. The witness offered a few theories which were credible but not evidentiary. The prosecution, however, did not attempt to contradict the claims, and instead argued that the witness was too expensive and did not have much contact with the defendant. It was a mistrial due to jury deadlock, as reasonable doubt was present.
tetraodonpuffer 10y ago
This is a thorny issue because say you could make the argument that if I spend 5 years writing and rewriting code from 0.1 to 0.8 at company A to understand a domain, then I move to company B and write 0.9 and 1.0 in the same domain and make a pile of $$$, then company A should somehow be compensated even if the code itself is not exactly the same (which it wouldn't be since you learned from mistakes made). On the other hand if you take, for example, an engineer who learns to build bridges and builds a lot of bridges in one company, and then moves to a different company still building bridges, one would not think that just because they build a great bridge at company B, company A should be compensated. Where do you draw the line between 'improving your craft / becoming a better software developer' to 'taking a company's IP'?
- perlgeek 10y ago
  The issue of potential IP transfer is why companies sometimes include non-compete clauses in their employment contracts. I don't know if they can be enforced in the relevant jurisdictions though.
- rhino369 10y ago
  The way the law usually protects is via trade secret law. So the deciding factor is if you are later using information that was a trade secret. Trade secrets have to have information that isn't known to the public. For the bridge example, just because they taught you how to build a great bridge, doesn't mean it was a trade secret. But if they had some sort of special sauce on how to build a bridge, it could be. It's more complicated than this. They actually have to keep the information secret, usually through NDAs, controlling code, not letting people show it off at conferences, etc. This sort of "mind" trade secret is hard to deal with. There isn't a clear distinction between talent/skill and information owned by your former employer.
sandworm101 10y ago
Too many people read too much into "peers". These are your legal peers, not social or educational equivalents. These are fellow citizens. So long as they arent royals, elected officials or cops they are peers. (Lawyers can technically serve on juries but every court i know doesnt want them to do so, nor law students.) We live now in a society with fewer official class systems, or we at least now better separate legal classes, and so easily forget the original intention of such words.
unityByFreedom 10y ago
Should he be talking about this? Don't they still have a chance to appeal?
timtadh 10y ago
I have do (academic) work [1, 2] on finding semantic code duplication. Two points that I have learned about code duplication: 1. There is are a lot, A LOT, of code regions that share similar constructions when analyzed in terms of dependencies. Dependencies only consider data flow and control dependencies. Where a statement X is control dependent on another statement Y (usually a if-condition or loop-condition) if Y decides whether X executes. In my studies I have found modestly sized Java programs (~ 75 KLOC) have > 500 million patterns representing duplication in their dependence graphs. 2. Not all dependence structures which are "duplicate" would be considered duplicated by a human programmer [2]. It takes discernment by someone familiar with the code base to decide whether or not regions are actually duplicated. I would argue you can draw similarities using automated metrics between disparate code bases. Those similarities are not evidence of copying. To decide whether similar regions are actually copied you would need to do further and subjective analysis. Without directly evidence of copying it would be very difficult to make a solid claim one way or the other. But, given the vast amount of similar code regions that exist (and assuming most code is not copied) I believe it should be given the benefit of the doubt. Note: I have not studied density of duplicated code between different projects. The above is merely an conjecture based on my experience. [1] http://hackthology.com/rethinking-dependence-clones.html http://hackthology.com/rethinking-dependence-clones.html [2] http://hackthology.com/sampling-code-clones-from-program-dependence-graphs-with-graple.html http://hackthology.com/sampling-code-clones-from-program-dep...
DannyBee 10y ago
"There are objective measures of code similarity that can be quoted, like the edit distance between abstract syntax trees, but here the expert hand identified the abstract steps that the code fragments were performing, made slides that nobody in the courtroom could actually read, filled with colored boxes outlining the purportedly analogous code in each case. In some cases, the abstractions he came up with were longer than the actual code they were supposed to be abstracting." This is because of the abstraction filtration comparison test that courts use: https://en.wikipedia.org/wiki/Abstraction-Filtration-Comparison_test https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari... I think it's garbage too, but such is life. I don't think you can blame the expert for producing something that goes along with what the current law is.
gregw2 10y ago
It's kinda hard to square John's post-trial comments "I never tried to hide or wipe any evidence, and all of my data is accounted for, contrary to some stories being spread." with ZeniMax's post-trial comments at http://www.gameinformer.com/b/news/archive/2017/02/01/zenimax-awarded-500-million-in-oculus-lawsuit.aspx http://www.gameinformer.com/b/news/archive/2017/02/01/zenima... that " (vi) Carmack intentionally destroyed data on his computer after he got notice of this litigation and right after he researched on Google how to wipe a hard drive—and data on other Oculus computers and USB storage devices were similarly deleted (as determined by a court-appointed, independent expert in computer forensics); ... (viii) Carmack filed an affidavit which the court's expert said was false in denying the destruction of evidence; " What is a reasonable explanation of this discrepancy? Varying notions of what consitutes "evidence"? Or differences in facts?
- mememachine 10y ago
  It looks like either someones lying or wrong
- zardo 10y ago
  I wouldn't trust the testimony of a paid expert witness. They always have strong convictions in favor of the side paying the bill, even if the evidence available clearly can't support such absolute conclusions. Even if they aren't just liars, the lawyers can just keep trying different 'experts' untill they get one who tells the story they want to hear. It's just cover for making an argument from authority.
  - Buttons840 10y ago
    "As an expert, I can definitely tell you that the people paying me are correct."
  - arca_vorago 10y ago
    I did an advanced forensics course a few years back that was geared towards preparing me for court cases for the company I was working for. I quickly realized how easily "expert" witnesses could and most likely were being abused in the system just as you say. Expert shop till you get what you want. That's not even mentioning the epic level of incompetence I saw among others. I'm talking about "forensic experts" who didn't know any filesystem other than NTFS, and even then if the tool didn't do it for them they didn't see it... At least I know if I ever end up defending myself I have plenty of inside knowledge on how to undermine the prosecutors arguments.
hoorayimhelping 10y ago
>* After he had said he was “Absolutely certain there was non-literal copying” in several cases, I just wanted to shout “You lie!”. By the end, after seven cases of “absolutely certain”, I was wondering if gangsters had kidnapped his grandchildren and were holding them for ransom.* This seems to be a very common reaction when reasonable people are put through a lawsuit.
- xxs 10y ago
  At least we can get few bits if information knowing the expert witness was old enough to have grandchildren.
emmett 10y ago
My father was an expert witness in electronic discovery cases for many years in the 90s and early 00s. I remember so many stories growing up about similar situations. The main lesson was that it was actually way easier to be an expert witness when your side was totally in the wrong, because you could just make stuff up. Explaining the truth about how computers work is hard, compared to making up simpler explanations that don't happen to actually reflect reality.
LeicaLatte 10y ago
Carmack is a hero for me but he seems to come across as very naive here. I employ other people now and employees who argue this sort of stuff isn't copying are toxic to the culture and ethics of work. When you are paid a salary every month, I actually don't care if you are productive. But the least I expect you to be is loyal. And I am no monster for expecting that.
- Strom 10y ago
  Loyalty in a free market is a pipe dream. Extremely few employers will stay loyal to their employees when times get rough, say when an employee gets very sick and can't work for several years. Thus no rational person should have any long term loyality to an employer if the market providers better alternatives.
  - LeicaLatte 10y ago
    This is a big reason why a US company might prefer immigrant workers. Local workers tend to treat their jobs more as transactions and less like a relationship with their employer. Immigrant workers come with built in motivation to stick to their employers and stay loyal.
- logfromblammo 10y ago
  You have unrealistic expectations for 2017, i think. The loyalty you receive will be proportional to the loyalty you demonstrate. When I am paid a salary every month, my expectation is that I am being paid for services rendered, and that what I am paid is a fraction of the value I produced. There is no loyalty demonstrated there; it's just business. As the company I work for now had in the past kicked me to the curb with only 3 days of notice and no severance whatsoever, I don't feel particularly loyal to them. But I'm still doing their work [again], and they are paying me salary for it [again]. I'm just a worker bee to them, and they're just a mealticket to me. It works out fine as long as I sleep with one eye open. The instant I can secure a better deal for me, I will pack up my personal possessions and go. I won't take a few weeks to wrap things up in a neat little package. I'll just turn in my badges and walk out. I will stop caring about the company that used to pay my salary on the day I get my last paycheck from them. I know they never cared about me; they only ever cared about my work product. In practice, in the real world, US employees won't be loyal, because US companies have in aggregate treated them like fungible resources, rather than people. The instant they are no longer profitable to keep, they are fired. So it isn't the employees that are polluting the workplace culture. The employers will eventually be hoist on their own collective petard, but it isn't as though the rest of us will come away clean from that disaster to bask in our schadenfreude, either. It's nice that you think that temporarily supporting unproductive employees until things turn back around is good business practice, but you're sort of swimming against the tide. I started my career with a small measure of loyalty, but a series of disloyal acts has conditioned me to instead be strictly mercenary. You're no monster, but you shouldn't expect that other employers are anywhere near your level of cooperation with their employees.
bisby 10y ago
That's a great analogy that I'm surprised I haven't heard before. If I write a book called Gary Potter, about a boy who finds out he's a wizard and goes on (specific) adventures with his friends Jon and Germione... I'd be in trouble. But a book about someone wishing for a better life > finding out they secretly had great powers > struggle to gain control/mastery of the powers > must use the powers to save the world from evil... that's just a story framework and applies to Harry Potter, Star Wars, Hercules(the Disney version at least). There's variations in why they want the better life (abuse, boredom, etc), how they find out, and what the exact evil is. but if you abstract far enough, they're all basically the same story. Using exact code and changing variables like "main_character_name" is clearly infringement, but using a similar abstracted flow shouldn't be. ESPECIALLY in code. If your end goal is to put pixels on a screen. you run algorithms to determine pixels > add those pixels to a buffer > send the buffer to the screen. There's not many other ways to do it. There's only so many ways to implement some things. And if a way is good, independently coming up with the solution isn't copying. In fact, if I came up with a different solution, it would be sub-optimal. For some problems, the abstracted flow of the code is defined for you. "We're doing VR, we want X, Y, Z to happen"... There's not a whole lot of options on how to do that. If Oculus only figured out the flow-charts based on copying zenimax's prototype, then they perhaps are copying. And thats the argument zenimax is shooting for. "this is a non-trivial, non-obvious solution they only got by copying us". But the next question is: at what level of abstraction am i just changing variables at a large scale? If I wrote a book about a young boy living with abusive relatives because his parents died and he finds out he's a wizard and moves away to a magical wizard school, does that count as copyright infringement? There's a point where it is, and a point where it isn't. I can never find the source, but I once saw a 256x256 picture of yoshi, that immediately below it had a 128x128 version of the same picture (stretched to 256x256). Below that was 64x64, etc until the 1x1 was just a green pixel. Along the side it said "at what point does it stop being copyright infringement? If you started at the top (the high res), you could get pretty far down and still go "oh yeah, this is still the same image". but if you started at the bottom, you could get much higher and go "this is some weird abstract art of random pixels"
almonj 10y ago
All laws relating to Intellectual Property need to be abolished.
grellas 10y ago
I learned long ago (at the beginning of my legal career) that a determined litigant can essentially cook up expert testimony on demand as needed to further some tendentious goal or other. It is dressed up to be "scientific," "learned," etc. but, when objectively analyzed, it is in reality utter garbage. But, and this is a big "but," it is the sort of garbage that cannot be refuted conclusively owing either to uncertainty in the science involved or uncertainty in the facts and assumptions on which it is based. By this, I don't mean there are experts who have integrity and who will not allow their good names and reputations to be cynically used for such purposes. But, for every expert of such integrity, it is (sadly) pretty easy to find others who can be bought. Don't know what happened in this case but I instinctively can sympathize with the author here that this is what might easily have happened.
DonHopkins 10y ago
Dave Beazley (Python expert and author of SWIG) was an expert witness and analyzed a huge body of code. This is his fascinating talk about his adventure: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 So, what happens when you lock a Python programmer in a secret vault containing 1.5 TBytes of C++ source code and no internet connection? Find out as I describe how I used Python as a secret weapon of "discovery" in an epic legal battle. Slides can be found at: https://speakerdeck.com/pycon2014 https://speakerdeck.com/pycon2014 and https://github.com/PyCon/2014-slides https://github.com/PyCon/2014-slides
qwertyuiop924 10y ago
To be clear, I (like many of you) have poured countless hours into John Carmack's games, and stand in awe of his work. I also have considerably less capability than most HN users in programming, etc. So I may have a large pro-Carmack bias going into this... Having said that, the concept of "non-literal copying" is, IMHO, nonsense. And dangerous. So I'll agree with Carmack on that much. And while I don't know (I don't think it's been revealed, although I haven't been following the case closely), it seems that Carmack had some hand in writing the original code, so of course the code will have similarities. Also, the analogy of "Harry Potter with the names changed" is, from what I can tell, a pretty bad one for what actually happened. In short, I think this is pretty much nonsense. OTOH, Zenimax does have so evidence of wrongdoing on Carmack's part, so it's not like their case has no basis, and I am inclined to be biased toward Carmack. And there's always the possibly that I have no idea what I'm talking about and I'm just stark raving mad.
wiz21c 10y ago
The goal of the expert is not to give a "truth", it's to convince the jury. That's a very different job.
nabla9 10y ago
Battle between experts trying to convince people from the street is very fragile system. Arguments start from the scratch every time. The idea of making expert witness testimonies as part of ones academic record is great. Scientific expert testimony literature should be cumulative in the same way as legal precedents are. Code copying is case that comes up again and again. There should be convergence towards scientifically justified ways for experts to determine copying.
hanginjudge 10y ago
I think the last sentence is the meat of the post. The biggest credential of some expert witnesses is the number of trials for which they have been an expert witness. Put your testimony out there for the public and let other experts critique it. Stake your reputation and your career on it.
cdevs 10y ago
I'm still surprised Facebook still has Facebook.com/somescript.php style links unless it's not as simple as it looks in the url. I know there's about 4 layers of loadbalance/cache and layers of that in data centers but any idea about the framework or code design?
- rachelbythebay 10y ago
  It would be kind of bad to break something named "permalink", wouldn't it?
msmith10101 10y ago
John Carmack has a history of betraying employers (e.g. softdisk). Hard to take him seriously when the evidence is he said she said and the law decided against him.
acqq 10y ago
The relevant court decision seems not to be about the topics about which Carmack writes: https://arstechnica.com/gaming/2017/02/oculus-execs-liable-for-500-million-in-zenimax-vr-trial/ https://arstechnica.com/gaming/2017/02/oculus-execs-liable-f... "While the jury found that Oculus had not misappropriated trade secrets from ZeniMax—a major part of the case—it determined that Oculus co-founder Palmer Luckey had violated a non-disclosure agreement with the company when working on early prototypes of the Rift VR headset" "Oculus is liable for $300 million in the verdict ($50 million for trademark infringement, $50 million for copyright infringement, and $200 million for breaking the NDA), while Luckey owes $50 million and former Oculus CEO Brendan Iribe owes $150 million (both for false designation). Oculus CTO John Carmack, who previously worked for ZeniMax and was accused of stealing code and destroying evidence, is not personally liable for any damages. ZeniMax had sought damages of $6 billion in the case."
didibus 10y ago
Does this mean if I write code that does X at company A, and then move to company B and am asked to write code that does X, I would be copyright infringing my own code I wrote at company A unless I purposely went out of my way to write it completly differently? Because chances are I still remember how I did it, or even if I don't, chances are I'll come up with pretty much the same way, as it's inherent to my code style and thought process. I guess it's something I'll be taking great care in doing if this ever happens to me.

505 points by samlittlewood 10y ago

kchoudhu 10y ago

Well, who was the expert witness?

potatoman2 10y ago

Loser's lament.

vanattab 10y ago

Isn't the testimony already a matter of public record?

delinka 10y ago
He mentions that it's not. "... his expert testimony in trial is under seal, rather than in the public record."

trendia 10y ago

This struck out at me: > There are objective measures of code similarity that can be quoted, like the edit distance between abstract syntax trees ... If this became the primary legal metric, then programmers who stole code would change the code so that functions achieved the same output with practically no AST similarity. That is, they could maximize functional similarity while minimzing code similarity. This would be a large task, but one that might be easier and preferred to writing code from scratch, since you drastically reduce the trial and error process of writing code in a field where best practices aren't currently known. But, while it's possible to game any 'objective' metric like AST distance, maybe that's preferable to the subjective and incentivized claims of an expert witness, who will be biased in favor of the party paying them. And while it's easy to get lost in the paragraphs of expert witness testimony, a collection of objective metrics is easy to compare to previous cases.

rocky1138 10y ago
Someone should write a script which takes any code and mucks the AST.
- gravypod 10y ago
  You should be able to just compile it with -O3 and then decompile it and rename all your variables back. You'll get a very wonky source output but it will be extremely different.
  - xxs 10y ago
    why would would you keep debug info like variable names while compiling w/ -O3 in the 1st place?
    - gravypod 10y ago
      You woudln't. This would be for an intial obfuscation pass. You'd not do this for release but for sufficiently scrambling the code base.
gravypod 10y ago
Funnily enough I actually implemented this [0] two semesters ago while working as a Teacher's Assistant for the into CS class. It worked fairly well. I basically took the levenshtein algorithm and mapped it directly to ASTs and used that to find the % of similar elements in the code. It worked a few times and I caught 1 or 2 groups of cheaters. ....sadly in that class you were allowed to "work together" you just had to include a note saying you did so I've got no "kill count" anymore and I'm not planning on TAing any time soon (at least not for python). Also that code was hacked together in a weekend and that anti-cheet was done in 2-3 hr. It's not production ready by a longshot. [0] - https://github.com/gravypod/GradeO/blob/master/libs/cheating_manager.py https://github.com/gravypod/GradeO/blob/master/libs/cheating...
- timtadh 10y ago

squizzel 10y ago

I often wonder if John was involved with the story telling aspect of his games, not just the code.

whatever_dude 10y ago
I think it has a recurring theme that any time he got involved, things didn't go well so he decided to let go of it. That's not to say he didn't have influence; many times the games' stories were derived FROM the tech, not the other way around. Doom was originally supposed to be much more story- and character-driven, but that took a backseat given the focus on the development side was on the gameplay mechanics. Also, this quote: > Story in a game is like a story in a porn movie. It's expected to be there, but it's not that important From "Masters of Doom".
- dosgonlogs 10y ago
  He was the Dungeon Master in their Dungeons & Dragons games. John Romero loved Carmack's campaign world, but John Romero destroyed it, Opening up a gateway to hell, and then immediately regretted it (but they got the idea for Doom from it), since it was a long running campaign in a world completely designed by John Carmack. I think Heretic/Hexen and Quake also borrowed heavily from their D&D games. Masters of Doom is an awesome book.
  - whatever_dude 10y ago
    Good point. I forgot about that.
samlittlewood 10y ago
From 'Masters of Doom' - he used to make and DM D&D campaigns.
Bamafan 10y ago
John Romero was largely responsible for the stories and designs of the old school iD games from back in the day. Carmack brought the tech brilliance to make it happen.

jstanley 10y ago

Link doesn't work for me, I just get a popup telling me to sign up to Facebook, and when I close it the page is blank? EDIT: Incognito mode fixed it. http://pastebin.com/KxAyvqyM http://pastebin.com/KxAyvqyM

addisonj 10y ago

I don't have the evidence so I can't make a judgement about whether Zenimax or Carmack is in the wrong here, but this does point out something strange in our court system and that is where a "jury of peers" which translates to normal people with little to no knowledge of the underlying subject matter. As our society becomes more specialized it seems a bit absurd to have people make judgements mostly based on how well each side can make incredibly complex things comprehensible and convincing without any real understanding of the underlying principles in the field. It seems likely that the best expert witness in this case is not the most correct or credentialed, but the most charismatic. It would be interesting to see how some of these big software trials (Google v Oracle) would have come out if the jury were made up of people who were both impartial and familiar with the industry.

lmm 10y ago
How is the court supposed to judge familiarity? What if jury members think they're technical experts, but aren't? There are always going to be complex technical matters that are relevant to lawsuits - this was the case before software even existed - and the way the system is supposed to work is that rather than the jury bringing their own set of preconceptions on the technical aspects, the expert witnesses answer the technical questions in the court (where they are subject to cross-examination etc.) and the jury makes the factual judgements that have always been their job.
smacktoward 10y ago
The more familiar you are with the industry, the less likely you are to be impartial.
pavel_lishin 10y ago
> this does point out something strange in our court system and that is where a "jury of peers" which translates to normal people with little to no knowledge of the underlying subject matter. Nothing in the constitution says anything about peers, and the fact that most juries are comprised of people who don't know much about the subject matter comes from lawyers wanting those people on the bench, since they're less likely to have pre-conceived notions - possibly notions conflicting with their clients' goals. If you're trying to convince someone of something, it helps if they start with a blank slate.

pyb 10y ago

"The expert witness circuit is surely tempting for many academics, since a distinguished expert can get paid $600+ an hour to prepare a weighty report that supports a lawyer’s case. I don’t have any issue with that, but testifying in court as an expert should be as much a part of your permanent public record as the journal papers you publish. In many cases, the consequences are significant. There should be a danger to your reputation if you are imprudent." So basically he's kind of threatening the expert witness now?

perfmode 10y ago
John makes a good point. Here, transparency appears to be in truth's best interest.
- pyb 10y ago
  From the downvotes, sounds like I'm the only one who is shocked by this.
  - intherdfield 10y ago
    Could you explain why you think this is a threat aimed at the expert witness in this trial? Any time someone publishes a work, they are putting their reputation at risk. If the work is bad, the whole world will know they did it. Carmack is saying he thinks this should be the case for reports and analyses given by expert witnesses.
    - 794CD01 10y ago
      You could say the same thing about anything people do in secret. Yet many people still want privacy.
      - intherdfield 10y ago
        > You could say the same thing about anything people do in secret. I don't think that makes sense. There is no attempt to generalize this to things people do in secret. There is no claim that if something could risk your reputation then it follows that it should be made public. Carmack's assertion is only that this particular work (for which the witness does voluntarily and is paid for) should be made public so that these witnesses use the same level of rigor they would for their other published work. And letting the public review it would have benefits too. (I am not agreeing or disagreeing with this.) Ultimately, if we take Carmack's statements as true, it sounds to me like Facebook/Occulus' defense did not do an adequate job of instilling doubt in the report. He wrote that the defense did a technical tear down. I think if I was a juror, I would need to see this report completely destroyed. I'd need to see the same methodology applied to works where we know there was no copying and have it find false positives.

ChuckMcM 10y ago

Interesting rant on expert witnesses. Mostly interesting because I've done some expert witness testimony and found it fascinating. I completely resonate with John's issue with the expert's testimony and his understanding of it. John mentioned that their own expert testified differently but he didn't say whether or not their lawyers attempted to impeach their expert with cross examination. In the two cases I participated in, both sides agreed ahead of time on how to "handle" experts (in terms of cross examination). In one it was documents only (no testimony) and in one I was deposed by the opposing counsel after documents in which the lawyer clearly had some notes (presumably from their expert(s)) and was trying to get me to recant or change some of my points (I didn't need to). In a jury trial I can imagine that having the jury understand your testimony is probably the most challenging. And, like John, I consider the notion of 'non-literal copying' to be pretty ridiculous. In the limit it means "you read this code, understood how it worked, and wrote new code that could do the same function." You can stretch that to cover anything you have ever seen. Which is sad.

madgar 10y ago
> In the limit it means "you read this code, understood how it worked, and wrote new code that could do the same function." You can stretch that to cover anything you have ever seen. Which is sad. I thought the industry adopted "clean room" techniques decades ago to avoid this kind of appearance of impropriety. It's more expensive, sure, but isn't eliminating legal risk something that typically gets paid for?
- johnbellone 10y ago
  This particular point worries me. Are we now not going to be able to make a clean room re-implementation of concept? Do we need to worry about our previous employer suing us because we are successful?
  - madgar 10y ago
    They didn't seem to do clean room here. Seems like Carmack was wrote code for both companies which is the exact opposite of clean room. (Please correct me if I misunderstood the facts of the case)

sfifs 10y ago

The problem for John Camack was I think he was on both sides of the table writing code - so he wrote the code at Zenimax & re-wrote the code at Oculus. From a layman's perspective, he very well could have been "copying". Normally when you design clean room implementations, you use different people and make sure that the people who write the code never see the prior written code and the person giving the spec doesn't see the newly written code. They were just in-sufficiently cautious legally, possibly un-surprising given Carmack's prior history in founding Id software, and got hit with a lawsuit.

TheCoelacanth 10y ago
That seems like it is saying that Zenimax owns not only the code Carmack wrote while he was there, but also the knowledge and experience he gained while working there.
- leggomylibro 10y ago
  I think it's more that Zenimax thinks it owns the unique ideas that Carmack, a top talent in his field, created while he was working there. They're not angry that he wrote the same 4x4 matrix inverse function for both them and Oculus. Or more specifically, if Zenimax's claims are correct, while he was working on company time and equipment.
  - jandrese 10y ago
    What is the alternative? Do we force everybody to undergo mind wipes whenever they leave the company so they don't take any skills they picked up on the job with them? How many people can forget everything they know every time they get a new job?
    - wmf 10y ago
      The alternative is probably non-compete contracts (which are enforceable if paid) so that people don't immediately go from one company to its direct competitor.

6stringmerc 10y ago

An interesting statement on things that may be relevant in the grand scheme of the trial, but, unless I'm really missing something here, the verdict was about busting an NDA (the Code being downstream of the NDA violation). Hence the $500M vs the $4B in damages. If it was just about code, then I'm sure Carmack's points would hold up just as well as they read in hindsight. But - and it's a big but - keep in mind his post is in a vacuum of sorts by not relating it back to the mechanics of how the case was actually ruled. Again, I might be missing something in his post, but I don't think so. As for this part: >The notion of non-literal copying is probably delicious to many lawyers, since a sufficient application of abstraction and filtering can show that just about everything is related. There are certainly some cases where it is true, such as when you translate a book into another language, but copyright explicitly does not apply to concepts or algorithms, so you can’t abstract very far from literal copying before comparing. As with many legal questions, there isn’t a bright clear line where you need to stop. Non-literal copying is why I like the "Gaye Family vs. Blurred Lines" verdict and why a wide swath of business folks and industry creatives hate it. Also, it's nearly a 1:1 if you want to talk about Code like Carmack does or Music like Musicians do. It's a totally reasonable thought experiment for one side to claim "Well, a jury of non-musicians ruled the wrong way and called it derivative" to which I'd counter "Well, a jury of expert musicians would probably rule the same way because they understand the nuances even better than the average lay person." Again, these are hypothetical arguments with real-world consequences, but I don't think these cases can be easily ruled upon. There's always going to be some hard feelings at the end of it.

Bamafan 10y ago
I believe the "non-literal copying" thing simply refers to the insight gained from the R&D time he spent on Occulus while still an employee of Zenimax. 90% of HN members know that this deep understanding is FAR, FAR, FAR more valuable than blindly copying over code. It was this deep understanding that made Occulus valuable and it was seemingly funded on Zenimax's time and dime.
- wang_li 10y ago
  Perhaps it's not an issue, but it's not like Carmack learned stuff as an employee. He has been the principle architect of the graphics engine for every product he's worked on for the last 20 years. How do you take that out of your head when you change companies but stay in the same field?

Arathorn 10y ago

Having gone through a similar trial, this is painfully familiar. The reality seems to be that the legal process optimises for whoever argues the most manipulatively (using every dirty trick in the book) rather than actually optimising for truth. Perhaps this is inevitable: rather than hoping to successfully convey a logical scientific argument to the judge/jury/arbitrator about a deeply technical area, instead the lawyers find themselves painting a more subjective simplified story that the audience may relate to... at which point you have to fight like with like. It's almost a post-truth kind of situation; the actual reality is irrelevant; it's instead how well the theatrical posturing is executed or how compelling the simplified version of the story is. That said, I have no idea whether Carmack is in the right or wrong here. In my instance the system worked in my favour, but I suspect it had a lot more to do with having lawyers who were good at debating than the almost incidental fact that objectively and demonstrably I had done nothing wrong...

abraae 10y ago
Absolutely concur. In an arbitration case, our attorney told us beforehand that (pardon the language) "the only bigger whores than lawyers are expert witnesses". The outcomes seem to depend mainly on, as you say, which side can spin the most compelling simplified story that resonates with the arbitrator/jury. As another poster says, perhaps use of analogies should be banned (Harry Potter in this case for goodness sake).

deleted 10y ago

[deleted]

wnevets 10y ago

Expert witnesses is kinda crazy in this country. One of the most famous cases had to be the father wrongfully executed because of it[0]. Law enforcement are also trained to talk about their years of experience and how they're an expert when giving testimony regardless of facts and juries love it. [0] https://en.wikipedia.org/wiki/Cameron_Todd_Willingham https://en.wikipedia.org/wiki/Cameron_Todd_Willingham

Digory 10y ago

I suppose many weird things happen in $500m cases, but it'd be really unusual to have trial testimony sealed against the agreement of the parties. Pretrial reports and depositions, sure. Media was apparently there reporting during the expert's trial testimony, and I'd be surprised if they cannot buy a transcript from the court reporter. (http://uploadvr.com/court-oculus-zenimax-last-day/ http://uploadvr.com/court-oculus-zenimax-last-day/) A quick look at the docket shows the expert witness at issue is likely to be David Dobkin of Princeton. That name appears on the docket and in media reports. Let me know if that violates HN norms, but expert witness trial testimony is part of the public, permanent record in almost every case. If a witness is excluded for using unreliable methods, his or her value as an expert is ... diminished.

itsdrewmiller 10y ago
Yeah, this was the part that seemed most important and potentially most unreliable to me. I would like to see where they officially requested that this testimony be made public and were denied.
gregw2 10y ago
Interesting. David Dobkin is the Dean of Faculty at Princeton University who used to be a Computer Science professor there for a very long time before being Dean; his specialty was 3D geometry (not a 3D shader guy, kinda a bit more math-y than applied.) I don't know if he ever presided over code plagarism cases with clever Princeton CS undergrads but since he was department chair for a fair chunk of time, information on that may well provide interesting insight into his methodology and previous experience. (Reviewing papers for various computer graphics entities might also contain additional plagarism experience but that's unlikely to be code-comparison-type plagarism.) Given his CV I would be very surprised whether he has considered abstract syntax trees much since he got his PhD 43 years ago, but I am not convinced by Carmack that is automatically exculpatory. I would consider him a reasonable pick as an expert in 3D geometry and algorithms (although he hasn't been active in the field in 10+ years he did it for a /long/ time which is likely sufficient) and he may have dealt with plagarism of regular text during his career a moderate amount, but he does NOT seem to be an expert in code-plagarism or code similarity. It's interesting that the former is good enough and they didn't need an expert in the latter. In any case, given the positions he's been at, whatever he's getting paid, even $600+/hour, I doubt he /needs/ the money. (If he is in fact the expert witness.)

minimaxir 10y ago

I recently served on a jury in an assault and battery case. The defense offered a psychological defense and called a psychologist as an expert witness. The witness offered a few theories which were credible but not evidentiary. The prosecution, however, did not attempt to contradict the claims, and instead argued that the witness was too expensive and did not have much contact with the defendant. It was a mistrial due to jury deadlock, as reasonable doubt was present.

tetraodonpuffer 10y ago

This is a thorny issue because say you could make the argument that if I spend 5 years writing and rewriting code from 0.1 to 0.8 at company A to understand a domain, then I move to company B and write 0.9 and 1.0 in the same domain and make a pile of $$$, then company A should somehow be compensated even if the code itself is not exactly the same (which it wouldn't be since you learned from mistakes made). On the other hand if you take, for example, an engineer who learns to build bridges and builds a lot of bridges in one company, and then moves to a different company still building bridges, one would not think that just because they build a great bridge at company B, company A should be compensated. Where do you draw the line between 'improving your craft / becoming a better software developer' to 'taking a company's IP'?

perlgeek 10y ago
The issue of potential IP transfer is why companies sometimes include non-compete clauses in their employment contracts. I don't know if they can be enforced in the relevant jurisdictions though.
rhino369 10y ago
The way the law usually protects is via trade secret law. So the deciding factor is if you are later using information that was a trade secret. Trade secrets have to have information that isn't known to the public. For the bridge example, just because they taught you how to build a great bridge, doesn't mean it was a trade secret. But if they had some sort of special sauce on how to build a bridge, it could be. It's more complicated than this. They actually have to keep the information secret, usually through NDAs, controlling code, not letting people show it off at conferences, etc. This sort of "mind" trade secret is hard to deal with. There isn't a clear distinction between talent/skill and information owned by your former employer.

sandworm101 10y ago

Too many people read too much into "peers". These are your legal peers, not social or educational equivalents. These are fellow citizens. So long as they arent royals, elected officials or cops they are peers. (Lawyers can technically serve on juries but every court i know doesnt want them to do so, nor law students.) We live now in a society with fewer official class systems, or we at least now better separate legal classes, and so easily forget the original intention of such words.

unityByFreedom 10y ago

Should he be talking about this? Don't they still have a chance to appeal?

timtadh 10y ago

I have do (academic) work [1, 2] on finding semantic code duplication. Two points that I have learned about code duplication: 1. There is are a lot, A LOT, of code regions that share similar constructions when analyzed in terms of dependencies. Dependencies only consider data flow and control dependencies. Where a statement X is control dependent on another statement Y (usually a if-condition or loop-condition) if Y decides whether X executes. In my studies I have found modestly sized Java programs (~ 75 KLOC) have > 500 million patterns representing duplication in their dependence graphs. 2. Not all dependence structures which are "duplicate" would be considered duplicated by a human programmer [2]. It takes discernment by someone familiar with the code base to decide whether or not regions are actually duplicated. I would argue you can draw similarities using automated metrics between disparate code bases. Those similarities are not evidence of copying. To decide whether similar regions are actually copied you would need to do further and subjective analysis. Without directly evidence of copying it would be very difficult to make a solid claim one way or the other. But, given the vast amount of similar code regions that exist (and assuming most code is not copied) I believe it should be given the benefit of the doubt. Note: I have not studied density of duplicated code between different projects. The above is merely an conjecture based on my experience. [1] http://hackthology.com/rethinking-dependence-clones.html http://hackthology.com/rethinking-dependence-clones.html [2] http://hackthology.com/sampling-code-clones-from-program-dependence-graphs-with-graple.html http://hackthology.com/sampling-code-clones-from-program-dep...

DannyBee 10y ago

"There are objective measures of code similarity that can be quoted, like the edit distance between abstract syntax trees, but here the expert hand identified the abstract steps that the code fragments were performing, made slides that nobody in the courtroom could actually read, filled with colored boxes outlining the purportedly analogous code in each case. In some cases, the abstractions he came up with were longer than the actual code they were supposed to be abstracting." This is because of the abstraction filtration comparison test that courts use: https://en.wikipedia.org/wiki/Abstraction-Filtration-Comparison_test https://en.wikipedia.org/wiki/Abstraction-Filtration-Compari... I think it's garbage too, but such is life. I don't think you can blame the expert for producing something that goes along with what the current law is.

gregw2 10y ago

It's kinda hard to square John's post-trial comments "I never tried to hide or wipe any evidence, and all of my data is accounted for, contrary to some stories being spread." with ZeniMax's post-trial comments at http://www.gameinformer.com/b/news/archive/2017/02/01/zenimax-awarded-500-million-in-oculus-lawsuit.aspx http://www.gameinformer.com/b/news/archive/2017/02/01/zenima... that " (vi) Carmack intentionally destroyed data on his computer after he got notice of this litigation and right after he researched on Google how to wipe a hard drive—and data on other Oculus computers and USB storage devices were similarly deleted (as determined by a court-appointed, independent expert in computer forensics); ... (viii) Carmack filed an affidavit which the court's expert said was false in denying the destruction of evidence; " What is a reasonable explanation of this discrepancy? Varying notions of what consitutes "evidence"? Or differences in facts?

mememachine 10y ago
It looks like either someones lying or wrong
zardo 10y ago
I wouldn't trust the testimony of a paid expert witness. They always have strong convictions in favor of the side paying the bill, even if the evidence available clearly can't support such absolute conclusions. Even if they aren't just liars, the lawyers can just keep trying different 'experts' untill they get one who tells the story they want to hear. It's just cover for making an argument from authority.
- Buttons840 10y ago
  "As an expert, I can definitely tell you that the people paying me are correct."
- arca_vorago 10y ago
  I did an advanced forensics course a few years back that was geared towards preparing me for court cases for the company I was working for. I quickly realized how easily "expert" witnesses could and most likely were being abused in the system just as you say. Expert shop till you get what you want. That's not even mentioning the epic level of incompetence I saw among others. I'm talking about "forensic experts" who didn't know any filesystem other than NTFS, and even then if the tool didn't do it for them they didn't see it... At least I know if I ever end up defending myself I have plenty of inside knowledge on how to undermine the prosecutors arguments.

hoorayimhelping 10y ago

>* After he had said he was “Absolutely certain there was non-literal copying” in several cases, I just wanted to shout “You lie!”. By the end, after seven cases of “absolutely certain”, I was wondering if gangsters had kidnapped his grandchildren and were holding them for ransom.* This seems to be a very common reaction when reasonable people are put through a lawsuit.

xxs 10y ago
At least we can get few bits if information knowing the expert witness was old enough to have grandchildren.

emmett 10y ago

My father was an expert witness in electronic discovery cases for many years in the 90s and early 00s. I remember so many stories growing up about similar situations. The main lesson was that it was actually way easier to be an expert witness when your side was totally in the wrong, because you could just make stuff up. Explaining the truth about how computers work is hard, compared to making up simpler explanations that don't happen to actually reflect reality.

LeicaLatte 10y ago

Carmack is a hero for me but he seems to come across as very naive here. I employ other people now and employees who argue this sort of stuff isn't copying are toxic to the culture and ethics of work. When you are paid a salary every month, I actually don't care if you are productive. But the least I expect you to be is loyal. And I am no monster for expecting that.

Strom 10y ago
Loyalty in a free market is a pipe dream. Extremely few employers will stay loyal to their employees when times get rough, say when an employee gets very sick and can't work for several years. Thus no rational person should have any long term loyality to an employer if the market providers better alternatives.
- LeicaLatte 10y ago
  This is a big reason why a US company might prefer immigrant workers. Local workers tend to treat their jobs more as transactions and less like a relationship with their employer. Immigrant workers come with built in motivation to stick to their employers and stay loyal.
logfromblammo 10y ago
You have unrealistic expectations for 2017, i think. The loyalty you receive will be proportional to the loyalty you demonstrate. When I am paid a salary every month, my expectation is that I am being paid for services rendered, and that what I am paid is a fraction of the value I produced. There is no loyalty demonstrated there; it's just business. As the company I work for now had in the past kicked me to the curb with only 3 days of notice and no severance whatsoever, I don't feel particularly loyal to them. But I'm still doing their work [again], and they are paying me salary for it [again]. I'm just a worker bee to them, and they're just a mealticket to me. It works out fine as long as I sleep with one eye open. The instant I can secure a better deal for me, I will pack up my personal possessions and go. I won't take a few weeks to wrap things up in a neat little package. I'll just turn in my badges and walk out. I will stop caring about the company that used to pay my salary on the day I get my last paycheck from them. I know they never cared about me; they only ever cared about my work product. In practice, in the real world, US employees won't be loyal, because US companies have in aggregate treated them like fungible resources, rather than people. The instant they are no longer profitable to keep, they are fired. So it isn't the employees that are polluting the workplace culture. The employers will eventually be hoist on their own collective petard, but it isn't as though the rest of us will come away clean from that disaster to bask in our schadenfreude, either. It's nice that you think that temporarily supporting unproductive employees until things turn back around is good business practice, but you're sort of swimming against the tide. I started my career with a small measure of loyalty, but a series of disloyal acts has conditioned me to instead be strictly mercenary. You're no monster, but you shouldn't expect that other employers are anywhere near your level of cooperation with their employees.

bisby 10y ago

That's a great analogy that I'm surprised I haven't heard before. If I write a book called Gary Potter, about a boy who finds out he's a wizard and goes on (specific) adventures with his friends Jon and Germione... I'd be in trouble. But a book about someone wishing for a better life > finding out they secretly had great powers > struggle to gain control/mastery of the powers > must use the powers to save the world from evil... that's just a story framework and applies to Harry Potter, Star Wars, Hercules(the Disney version at least). There's variations in why they want the better life (abuse, boredom, etc), how they find out, and what the exact evil is. but if you abstract far enough, they're all basically the same story. Using exact code and changing variables like "main_character_name" is clearly infringement, but using a similar abstracted flow shouldn't be. ESPECIALLY in code. If your end goal is to put pixels on a screen. you run algorithms to determine pixels > add those pixels to a buffer > send the buffer to the screen. There's not many other ways to do it. There's only so many ways to implement some things. And if a way is good, independently coming up with the solution isn't copying. In fact, if I came up with a different solution, it would be sub-optimal. For some problems, the abstracted flow of the code is defined for you. "We're doing VR, we want X, Y, Z to happen"... There's not a whole lot of options on how to do that. If Oculus only figured out the flow-charts based on copying zenimax's prototype, then they perhaps are copying. And thats the argument zenimax is shooting for. "this is a non-trivial, non-obvious solution they only got by copying us". But the next question is: at what level of abstraction am i just changing variables at a large scale? If I wrote a book about a young boy living with abusive relatives because his parents died and he finds out he's a wizard and moves away to a magical wizard school, does that count as copyright infringement? There's a point where it is, and a point where it isn't. I can never find the source, but I once saw a 256x256 picture of yoshi, that immediately below it had a 128x128 version of the same picture (stretched to 256x256). Below that was 64x64, etc until the 1x1 was just a green pixel. Along the side it said "at what point does it stop being copyright infringement? If you started at the top (the high res), you could get pretty far down and still go "oh yeah, this is still the same image". but if you started at the bottom, you could get much higher and go "this is some weird abstract art of random pixels"

almonj 10y ago

All laws relating to Intellectual Property need to be abolished.

grellas 10y ago

I learned long ago (at the beginning of my legal career) that a determined litigant can essentially cook up expert testimony on demand as needed to further some tendentious goal or other. It is dressed up to be "scientific," "learned," etc. but, when objectively analyzed, it is in reality utter garbage. But, and this is a big "but," it is the sort of garbage that cannot be refuted conclusively owing either to uncertainty in the science involved or uncertainty in the facts and assumptions on which it is based. By this, I don't mean there are experts who have integrity and who will not allow their good names and reputations to be cynically used for such purposes. But, for every expert of such integrity, it is (sadly) pretty easy to find others who can be bought. Don't know what happened in this case but I instinctively can sympathize with the author here that this is what might easily have happened.

DonHopkins 10y ago

Dave Beazley (Python expert and author of SWIG) was an expert witness and analyzed a huge body of code. This is his fascinating talk about his adventure: https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 https://www.youtube.com/watch?v=RZ4Sn-Y7AP8 So, what happens when you lock a Python programmer in a secret vault containing 1.5 TBytes of C++ source code and no internet connection? Find out as I describe how I used Python as a secret weapon of "discovery" in an epic legal battle. Slides can be found at: https://speakerdeck.com/pycon2014 https://speakerdeck.com/pycon2014 and https://github.com/PyCon/2014-slides https://github.com/PyCon/2014-slides

qwertyuiop924 10y ago

To be clear, I (like many of you) have poured countless hours into John Carmack's games, and stand in awe of his work. I also have considerably less capability than most HN users in programming, etc. So I may have a large pro-Carmack bias going into this... Having said that, the concept of "non-literal copying" is, IMHO, nonsense. And dangerous. So I'll agree with Carmack on that much. And while I don't know (I don't think it's been revealed, although I haven't been following the case closely), it seems that Carmack had some hand in writing the original code, so of course the code will have similarities. Also, the analogy of "Harry Potter with the names changed" is, from what I can tell, a pretty bad one for what actually happened. In short, I think this is pretty much nonsense. OTOH, Zenimax does have so evidence of wrongdoing on Carmack's part, so it's not like their case has no basis, and I am inclined to be biased toward Carmack. And there's always the possibly that I have no idea what I'm talking about and I'm just stark raving mad.

wiz21c 10y ago

The goal of the expert is not to give a "truth", it's to convince the jury. That's a very different job.

nabla9 10y ago

Battle between experts trying to convince people from the street is very fragile system. Arguments start from the scratch every time. The idea of making expert witness testimonies as part of ones academic record is great. Scientific expert testimony literature should be cumulative in the same way as legal precedents are. Code copying is case that comes up again and again. There should be convergence towards scientifically justified ways for experts to determine copying.

hanginjudge 10y ago

I think the last sentence is the meat of the post. The biggest credential of some expert witnesses is the number of trials for which they have been an expert witness. Put your testimony out there for the public and let other experts critique it. Stake your reputation and your career on it.

cdevs 10y ago

I'm still surprised Facebook still has Facebook.com/somescript.php style links unless it's not as simple as it looks in the url. I know there's about 4 layers of loadbalance/cache and layers of that in data centers but any idea about the framework or code design?

rachelbythebay 10y ago
It would be kind of bad to break something named "permalink", wouldn't it?

msmith10101 10y ago

John Carmack has a history of betraying employers (e.g. softdisk). Hard to take him seriously when the evidence is he said she said and the law decided against him.

acqq 10y ago

The relevant court decision seems not to be about the topics about which Carmack writes: https://arstechnica.com/gaming/2017/02/oculus-execs-liable-for-500-million-in-zenimax-vr-trial/ https://arstechnica.com/gaming/2017/02/oculus-execs-liable-f... "While the jury found that Oculus had not misappropriated trade secrets from ZeniMax—a major part of the case—it determined that Oculus co-founder Palmer Luckey had violated a non-disclosure agreement with the company when working on early prototypes of the Rift VR headset" "Oculus is liable for $300 million in the verdict ($50 million for trademark infringement, $50 million for copyright infringement, and $200 million for breaking the NDA), while Luckey owes $50 million and former Oculus CEO Brendan Iribe owes $150 million (both for false designation). Oculus CTO John Carmack, who previously worked for ZeniMax and was accused of stealing code and destroying evidence, is not personally liable for any damages. ZeniMax had sought damages of $6 billion in the case."

didibus 10y ago

Does this mean if I write code that does X at company A, and then move to company B and am asked to write code that does X, I would be copyright infringing my own code I wrote at company A unless I purposely went out of my way to write it completly differently? Because chances are I still remember how I did it, or even if I don't, chances are I'll come up with pretty much the same way, as it's inherent to my code style and thought process. I guess it's something I'll be taking great care in doing if this ever happens to me.