7 ms·

> Notice that this method is a lot simpler than the other two, but because you have to use average price, you are exposed to floating point imprecision which is

> Notice that this method is a lot simpler than the other two, but because you have to use average price, you are exposed to floating point imprecision which is a lack of accuracy that computers have when computing decimals. Using average price and floating point imprecision are two separate, unrelated issues. You don't have to, and probably shouldn't use floating point to calculate the average price, and it's perfectly possible to use decimal arithmetic or fractions rather than floating point. The real issue here is that 2/3 does not have a decimal representation, not floating point imprecision. No matter what you do, if you need to express it as a decimal of finite length it will need to be rounded, and that's where the imprecision is. In practice, you can generally get away with using floats because the floating point error ends up being less than the rounding imprecision.

hansvm 5y ago
Given that the average is just a means to an end (total P&L), why would they not simply rescale by total shares to make that technical difficulty go away?
- tacostakohashi 5y ago
  It's not a technical difficulty, it's an arithmetic difficulty.
  - hansvm 5y ago
    Sorry, I probably wasn't clear enough --- the only reason there exists an arithmetical difficulty in the first place is because we're materializing an intermediate value we don't really care about with any kind of precision and using it to compute a value which needs perfect precision. The proposed solution is to never mess with division in the first place when computing P&L; operate on "total" values: cost basis, total profit, .... Edit: and as a super minor point of contention, I don't think it's wrong to call it a technical difficulty. You can represent the involved quantities exactly with a decimal ratio type, and the existence of a technical solution indicates that the problem is at least somewhat technical in nature.
    - tacostakohashi 5y ago
      Let's say you buy three shares at prices of $5, $5 and $10. You paid a total of $20 for the three shares. Now you sell 2 of them for $20 each ($40 in total), and have 1 share remaining. You want to calculate realized P&L using the average cost method. The average cost per share is $20/3, so the P&L is $40-(20/3).
      - hansvm 5y ago
        Thank you for the example; that nuance didn't click initially.
is0tope 5y ago
You are absolutely correct, and in general no-one should use floating point for any monetary values. I was trying to keep it as simple as possible without going into the nuances of float vs decimal. I might rephrase it to just "precision" in general, and skip the floating point part. Thank you for pointing this out.
- jrm4 5y ago
  This makes intuitive sense, but I still can't quite wrap my head around this idea of "no floating point for monetary values?," or actually, the opposite. I'm trying to get what floating point is really good or necessary for, then. Would it be something like "huge or tiny scale, generally theoretical, scientific things?"
  - tacostakohashi 5y ago
    Floating point is ideal for scientific applications, where the numbers are measurements/observations/approximations that have a margin of error built into them anyway, which is likely to be much larger than any additional imprecision introduced by floating point use.
  - is0tope 5y ago
    Floating point is good enough when you need to display something, and also for certain scientific calculations where high precision is necessary. You only start to see errors after a while, and it is likely good enough for the calculations being done. For something like money, you need to be very exact since 0.123541234123 cents is not a real value. Every trade you might "lose some cents" eg 0.00000001 or something like that. Over billions of transactions that starts to build up, and you start to either create or lose money arbitrarily. In finance, if you are keeping track of money you want to use integer values where at all possible.
    - foolinaround 5y ago
      in which scenarios is it not good enough?

hansvm 5y ago

Given that the average is just a means to an end (total P&L), why would they not simply rescale by total shares to make that technical difficulty go away?

tacostakohashi 5y ago
It's not a technical difficulty, it's an arithmetic difficulty.
- hansvm 5y ago
  Sorry, I probably wasn't clear enough --- the only reason there exists an arithmetical difficulty in the first place is because we're materializing an intermediate value we don't really care about with any kind of precision and using it to compute a value which needs perfect precision. The proposed solution is to never mess with division in the first place when computing P&L; operate on "total" values: cost basis, total profit, .... Edit: and as a super minor point of contention, I don't think it's wrong to call it a technical difficulty. You can represent the involved quantities exactly with a decimal ratio type, and the existence of a technical solution indicates that the problem is at least somewhat technical in nature.
  - tacostakohashi 5y ago
    Let's say you buy three shares at prices of $5, $5 and $10. You paid a total of $20 for the three shares. Now you sell 2 of them for $20 each ($40 in total), and have 1 share remaining. You want to calculate realized P&L using the average cost method. The average cost per share is $20/3, so the P&L is $40-(20/3).
    - hansvm 5y ago
      Thank you for the example; that nuance didn't click initially.

is0tope 5y ago

You are absolutely correct, and in general no-one should use floating point for any monetary values. I was trying to keep it as simple as possible without going into the nuances of float vs decimal. I might rephrase it to just "precision" in general, and skip the floating point part. Thank you for pointing this out.

jrm4 5y ago
This makes intuitive sense, but I still can't quite wrap my head around this idea of "no floating point for monetary values?," or actually, the opposite. I'm trying to get what floating point is really good or necessary for, then. Would it be something like "huge or tiny scale, generally theoretical, scientific things?"
- tacostakohashi 5y ago
  Floating point is ideal for scientific applications, where the numbers are measurements/observations/approximations that have a margin of error built into them anyway, which is likely to be much larger than any additional imprecision introduced by floating point use.
- is0tope 5y ago
  Floating point is good enough when you need to display something, and also for certain scientific calculations where high precision is necessary. You only start to see errors after a while, and it is likely good enough for the calculations being done. For something like money, you need to be very exact since 0.123541234123 cents is not a real value. Every trade you might "lose some cents" eg 0.00000001 or something like that. Over billions of transactions that starts to build up, and you start to either create or lose money arbitrarily. In finance, if you are keeping track of money you want to use integer values where at all possible.
  - foolinaround 5y ago
    in which scenarios is it not good enough?