8 ms·
German tank problem
- sbirch 13y agoMy favorite explanation of this (posed instead as the Locomotive problem) is in Allen Downey's "Think Bayes," pp.22 It's online too, and worth reading! http://www.greenteapress.com/thinkbayes/ http://www.greenteapress.com/thinkbayes/
- jmount 13y agoBook seems nice, but its discussion on the German Tank problem sets up code to calculate posteriors from priors that are more detailed than the Bayesian argument in the Wikipedia article. That is useful, but you don't get the problem intuition that the answer is the maximum ID you saw inflated by a factor depending on the expect ID-gap. There are some assumptions in the Wikipedia Bayesian analysis, but they are less determining than the ones in the book.
- therealmarv 13y agoInteresting. Especially because there is no German translation or Wikipedia entry for this article.
- wereHamster 13y agoThe germans didn't have to estimate the numbers, they just looked into their books!
- deleted 13y ago[deleted]
- frik 13y agoUnsurprisingly, ...German Wikipedia has (sadly) some badly behaved admins that delete pages because the are not "relevant". These admins that delete pages like at random are called "Löschnazi" by the community [1]. My observation is that one has to resort to the English version, because either the page is missing or it is biased to Germany (albeit German language is also the native language of Austria, Switzerland, etc.). German Wikipedia has of course also a lot of good efforts like the WikiData and Geolocation sub-project, etc. Hopefully they can kick the badly behaved admins soon... the reader to author ratio is already alarming. [1] http://de.wikipedia.org/wiki/L%C3%B6schnazi http://de.wikipedia.org/wiki/L%C3%B6schnazi of course the page got deleted as well so there is a backup: http://de.pluspedia.org/wiki/L%C3%B6schnazi http://de.pluspedia.org/wiki/L%C3%B6schnazi
- Havoc 13y agoI don't think it has any significance in that context to be honest. I've got a German background & have seen this posted a couple of times. Every time I think...of all the events/things in WW2 this is what you pick as interesting?
- te 13y agoThe frequentist and Bayesian analyses give different answers to the central question. Which one is more correct?
- thearn4 13y agoI would guess that it mostly depends on the quality of the prior.
- deleted 13y ago[deleted]
- _delirium 13y agoThere isn't really 'a' correct frequentist or Bayesian answer to the problem; it's more two different ways of thinking about the problem, which could well get you the same numerical results (though they might not). The frequentist way of thinking about it is to ask what you mean by "more correct", i.e. what properties do you want an optimal estimator to have? Another way of putting this is: if you were to set up a simulation where the real answer is known and data is sampled, and then you judge estimators by how close they get (according to some penalty function scoring closeness) when you run this simulation 10,000 times, which estimator would score the best? The estimator with the minimum variance of all unbiased estimators (the MVUE) will do optimally under some definitions of optimal; the MLE is another one that is optimal for other definitions. Note that they're both frequentist and give different answers. The Bayesian analysis of the situation is that it basically comes down to your choice of prior: the observed information is not in itself sufficient to produce a single "best" estimate, but rather you combine it with your prior distribution to produce an estimate. The Bayesian could end up producing exactly the same estimate as the estimators this article labels "frequentist". The Bayesian argument would be that what these estimators are really doing under the language of MVUE/MLE/etc. is implicitly choosing priors, whereas the Bayesian would explicitly choose one. The Bayesian would also probably not really like the simulation-experiment idea (which is a pretty directly frequentist thought experiment).
- bazzargh 13y ago
- schoen 13y agoHuh, I once visited a military base where people on the trip wanted to be photographed with a tank. The soldiers said it was OK, as long as somebody obscured the tank's serial number by standing in front of it. I wonder if their training in this respect was inspired by this history! (But if so, why not print the serial numbers inside the tank, not outside? Or maybe encrypt or HMAC them?)
- Robin_Message 13y agoSo, I HMAC the serial numbers of the tank. Now you have a series of very large numbers, evenly distributed. Someone else can do the proof, but I'm pretty sure the total number of tanks is inversely proportional to the smallest difference between two observed HMACs (or between the highest observed HMAC and the highest possible HMAC). EDIT: Actually, scratch that. That only works if you see all the tanks...
- herge 13y agoIf the serial numbers are randomly distributed in some finite span, and you have the serial number of three tanks, say: a: 000001 b: 000101 c: 100000 What is the probability that c is not the highest serial number and that there are 1000 total tanks. How many tank serial numbers would we need before those probabilities get > 90%, > 95%?
- Spooky23 13y agoI had a similar experience and asked. The answer was that when things like tanks get moved, it's by rail or ship. When figuring out what goes where, having to climb up on a railcar to figure out whose tank it was gets old.
- Malician 13y agoThe wiki article mentions estimations being done from parts from broken tanks, in which case internal numbers wouldn't necessarily help.
- vonmoltke 13y ago
- sparkman55 13y agoThere is some practical relevance to software development here. One shouldn't expose sequential IDs (a.k.a. serial numbers) to the public for anything non-public. I see this Hacker News post has a numerical ID in the URL, for example; I can estimate the size of Hacker News given enough of these numbers... More directly, I can modify that numerical ID to crawl Hacker News. Many sites do this; it's generally better to generate a (random or hashed or generated from a natural key) 'slug' to use as the key instead. For example, Amazon generates a unique, non-sequential, 10-digit alphanumeric string for each item in their catalog.
- pavel_lishin 13y agoThe flipside is that you can give off the impression of having a large user base/product catalog/etc if you number things sequentially... but start at a large non-round number.
- fecak 13y agoSome businesses do this with invoice numbers to give the false impression of being more successful (more invoices should equal more sales and customers) than it actually is.
- michaeleloftis 13y agoI did this myself with personal checks. I asked my bank to give me personal checks starting at some moderately sized number so that I wouldn't be giving anyone check #0000001.
- sparkman55 13y agoIt seems like one could use the same technique to estimate the initial (lowest-observable) serial number... From the article: If starting with an initial gap between 0 and the lowest sample (sample minimum), the average gap between samples is (m - k)/k; the -k being because the samples themselves are not counted in computing the gap between samples. Perhaps someone with a better grasp on the math can confirm that this makes 'obfuscating size by starting with a higher serial number' an ineffective mechanism?
- stillsut 13y agoThis is why the whole secret agent "#3" thing in movies like Bourne Legacy, James Bond etc are so ridiculous. That's a worse code name then just using the person real name as it gives hints of the total participation in the secret organization.
- Tohhou 13y agoThe name's Agent 127356 617521 468927 932557 316478 323374 395995 326630 491034 217268 274611 237524 337188 146754 679652 340738 635877 617517 992248 343367 515750 470302 132876 177599 156605 28370 149544 889133 470520 994752 52998 306348 827980 134251 494718 157786 643512 976648 676871 335476 314504 786821 432468 815692 537830 465962 245564 22239 948088 588642 356978 27525 7635 565720 138592 82302 437935 431429 66539 283187 428296 276837 52407 584999 385187 461461 616784 947454 981732 580233 239585 601256 943780 385422 669503 579611 619964 902619 999399 317012 976906 634968 515478 979532 360526 554976 333481 560942 196337 397222 19738 518392 842556 570366 142058 557450 818663 997306 239940 429107 Shaken not stirred.
- jxf 13y agoIt's astounding how accurate they were using only statistical methods: > Analysis of wheels from two tanks (48 wheels each, 96 wheels total) yielded an estimate of 270 produced in February 1944, substantially more than had previously been suspected. > German records after the war showed production for the month of February 1944 was 276.
- ta53535 13y agoIt wasn't exclusively statistical: The analysis of tank wheels yielded an estimate for the number of wheel molds that were in use. A discussion with British road wheel makers then estimated the number of wheels that could be produced from this many molds...
- Someone 13y agoYou can't do statistics without data. You aren't complaining that they used wheel data, either, did you?
- auctiontheory 13y agoI first read about this work a few years ago, but had I encountered it before college, I think I might have majored in statistics. Such powerful results - feel like magic.
- concernedctzn 13y agoThis sounds like an excerpt from the Cryptonomicon, which I happen to be reading right now.
- elwell 13y agoThe Germans should have sanded off the serial numbers.
- T-hawk 13y agoThis actually hit my previous company in a software context. We would number our hotfixes sequentially. Many would be items demanded by a single client, so would get deployed as hotfixes only to that customer's site, and just rolled into the main trunk for the next quarterly release for everyone else. Clients would always be notified about hotfixes going onto their live sites. One savvy client noticed the hotfix numbering sequence. Naturally, that ensued quite a number of extremely awkward discussions as they would regularly ask why our software needed so many hotfixes (tens per week) and why they weren't entitled to all of them right away. Solution: a new policy to randomly generate hotfix numbers. Which of course led to the next problem, that now the sequence was not obvious from the names, so dependent hotfixes would sometimes get deployed in the wrong order. Why can't anything be easy...
- Crito 13y agoWhat you need is random but monotonically increasing sequential version numbers. Add a random number to your last version number, and that is your new version number. ;)
- sspiff 13y agoThat's still vulnerable to statistics though. I would just use timestamps.
- PythonicAlpha 13y agoVery good idea. Timestamps do not reveal how many hotfixes are made and are chronological by default. To be sure, there could of course be a central instance providing the stamps and ensuring that no two fixes get the same ... (when several fixes could be made in parallel)
- vacri 13y agoJust name the hotfixes by day (140222). It monotically increases, and if you do multiple hotfixes in one day, suffix a/b/c etc. Generally you're unlikely to get up to b or c, and there's no clue to how many previous versions there's been.
- germanTankPanzy 13y agoI think the most important information is the table: Month Statistical estimate Intelligence estimate German records June 1940 169 1,000 122 June 1941 244 1,550 271 August 1942 327 1,550 342 Intelligence estimates... so off the mark.
- twic 13y agoBut these estimates were derived directly from a graph in the Germans' pitch deck!
- ariwilson 13y agoI encountered a slightly different problem trying to find the size of the union of a bunch of sets. We ended up just storing the smallest k int64 hashes of each item in each set, and computing 2^64 / ((largest hash - smallest hash) / (k - 1) as an estimate of the size of the union.
- robryk 13y agoThis is a very old (and nice) problem in streaming algorithms. The solution currently used in most places is HyperLogLog[1], which basically uses the distribution of log(minimum value of hash) for a set of hashes. [1] http://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf http://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf
- bane 13y agoSo here's an idea. Conventional intelligence was off by quite a bit, spurring the allies to overproduce tanks (which was possible due to the absurd American industrial capacity), which then allowed the allies to cleanly overwhelm the order of magnitude fewer tanks they actually came in kinetic contact with.
- dhoulb 13y agoIn case anyone is interested in the numbers, USA produced in the region of 50,000 Sherman tanks during the Second World War. 5-10x more than Germany. Plus the British tanks. It didn't take all that long to reach Berlin!
- nl 13y agoActually, the US produced 102,000 tanks and self propelled guns, while Germany produced 67,000. Russia produced 105,000 which was the real difference. http://en.m.wikipedia.org/wiki/Military_production_during_World_War_II http://en.m.wikipedia.org/wiki/Military_production_during_Wo...
- IgorPartola 13y agoDon't remember where I read it at least 12 years ago, but someone talked about an April Fools prank where they released three pigs in their high school, with numbers 1, 2, and 4 written on them. Allegedly the administrators spent weeks looking for number 3.
- idProQuo 13y agoThat's the version from 20 years ago. Those people have grown up and become school administrators, so it doesn't work as well anymore. The new plan is to release pigs numbered 1,2, 5,6,7, 10 and -2.
- lotharbot 13y agoThe "weeks looking for number 3" part sounds apocryphal to me. Snopes [0] has one real example and one television example of this prank. In the real example, the students were caught on camera, and there was no long search for the remaining livestock. [0] http://www.snopes.com/college/pranks/livestock.asp http://www.snopes.com/college/pranks/livestock.asp
- IgorPartola 13y agoThanks for actually researching this. As I remember, I read this bash.org, so I was pretty skeptical of the story just because of the source. It's one of those stories I'd like to believe was true just because I like it.
- RA_Fisher 13y agoI remember my theoretical stats teacher showing us this problem. It's used all the time in ecology. His example used it to estimate the number of alligators in Louisiana swamps. They tag the alligators, release, and then using the tags they re-capture over subsequent years, they can get an estimate of how many alligators exist in the wild!
- taejo 13y agoThe German tank problem applies when all the alligators come pre-tagged with serial numbers :) Capture-recapture is a slightly different problem: https://en.wikipedia.org/wiki/Mark_and_recapture https://en.wikipedia.org/wiki/Mark_and_recapture
- Pinatubo 13y agoTank #7, you are now known as tank #22347. Tank #1 and tank #22347, report to the commander for the orders related to your suicide mission ...