3 ms·

Work-related, I built a backend of a database engine on top of a transactional distributed key-value store. When building a secondary index, suppose you traver

Work-related, I built a backend of a database engine on top of a transactional distributed key-value store. When building a secondary index, suppose you traverse the primary index in a series of transactions, reading a batch of key/values and updating the secondary index. With the scheme that was used to build the index, this has the chance of conflicting with an incoming write, which will also try to update the secondary index. You'll have some kind of backoff/retry strategy. What is the optimal batch size? Treating it as a clean calculus problem with static, homogeneous workloads, you could model the probability of getting a conflict, the performance of a given batch size, set the derivative to zero and find the maximum. (In reality instead of modeling the probability, you can measure it.) In another instance, I got a summer job working at a company that made an optics-related product, and that involved a bunch of numerical integration, Fourier transforms, some optimization problems in stuff like calibrating equipment. So naturally, there was calculus. Fun-related, recently, I noticed that a long-term call option I was holding seemed surprisingly expensive compared to just buying the stock on margin, given current and expected future interest rates, and comparing against the option's theta and delta values. Then I realized I was assuming the stock only goes up, ignoring short-term vs. long term taxes, etc. To get a general sense of how an alternative strategy of just buying the stock and adjusting position sizes at a few price points might work out, in a back-of-the-envelope calculation, I considered this problem: on average, how many times will a Wiener process W(t) completely cross the small interval [a,a+h] in the time interval 0<t<T (for some T)? What's its asymptotic behavior with respect to h, T, and a? (This problem of hedging options with stock, surely, is studied much more in depth, but I wanted to understand the general picture.) Edit: I'll add, that's also taking for granted a lot of stuff with infinite series in comp sci that is also part of calculus. Like, if you want to pick a random number from 1 to 5, you know you can pick from 1-8 and retry if it's too large, and with calculus you know it will average a finite number of tries (8/5, specifically) because the infinite series passes the ratio test.

SamReidHughes 5y ago
Here is a new example. I was thinking about strategy as a pro golfer playing in a 72-hole tournament. Generally, you want to minimize expected value, get the lowest average score on each hole. It is obvious that near the end of the tournament, if you're behind, you want to play riskier golf, and if you're in the lead, you want to play less risky golf. That maximizes your chance of winning. This means instead of optimizing for expected value, you sacrifice expected value in exchange for higher or lower variance, respectively. What I realized a couple of days ago is that even earlier in the tournament, you don't want to optimize expected value -- you always want to give some of that for higher variance. Why? Because in the variance/EV trade-off curve, the slope of the curve is zero at the maximum value, which means to add h units of variance, you give up o(h), realistically, O(h^2), expected value. So there is some value of h where the trade-off is worth it. (Concrete example: Golf tournaments could be modeled as a coin-flipping contest, where you get +1 point for flipping heads, -1 point for flipping tails, with 72 coin tosses. Suppose you could pay a 0.01 point cost to double the point value, i.e. you get +1.99 or -2.01 as a result of the coin toss. Then you should, because doing so on the first coin tosses will increase your chance of winning.)

SamReidHughes 5y ago

Here is a new example. I was thinking about strategy as a pro golfer playing in a 72-hole tournament. Generally, you want to minimize expected value, get the lowest average score on each hole. It is obvious that near the end of the tournament, if you're behind, you want to play riskier golf, and if you're in the lead, you want to play less risky golf. That maximizes your chance of winning. This means instead of optimizing for expected value, you sacrifice expected value in exchange for higher or lower variance, respectively. What I realized a couple of days ago is that even earlier in the tournament, you don't want to optimize expected value -- you always want to give some of that for higher variance. Why? Because in the variance/EV trade-off curve, the slope of the curve is zero at the maximum value, which means to add h units of variance, you give up o(h), realistically, O(h^2), expected value. So there is some value of h where the trade-off is worth it. (Concrete example: Golf tournaments could be modeled as a coin-flipping contest, where you get +1 point for flipping heads, -1 point for flipping tails, with 72 coin tosses. Suppose you could pay a 0.01 point cost to double the point value, i.e. you get +1.99 or -2.01 as a result of the coin toss. Then you should, because doing so on the first coin tosses will increase your chance of winning.)