3 ms·

A lot to like here, a couple of things to comment on: > "If you tell people that their systems could be 1.5x faster or 5% more effective, that's rarely going t

A lot to like here, a couple of things to comment on: > "If you tell people that their systems could be 1.5x faster or 5% more effective, that's rarely going to beat inertia. In my view, you need to find problems where there's non-zero hope that you'll make things, say, 20x faster or 30% more effective, at least after years of work." This works for up-and-coming fields, but once something is stable and works at large scale, it's all about the small improvements. Making petrol engines 1% more fuel-efficient would be massive. Increasing the conversion rate of online ads by 1% could make you very, very rich indeed. Good advice for AI probably; bad advice in other fields. > "Invest in projects, not papers" The best way I think you can go about this is allocate some fraction alpha of your time to projects, and (1-alpha) to things that produce short-term papers. Alpha should never be zero if you want a career, but it will start out small as you begin your PhD and gradually grow, if you can make it in academia. At some point you'll reach a compounding return where the projects themselves are spawning papers - one way to do this is to get to the point where you can hire your own PhD students, but there are several others. As long as your 2-years-into-a-PhD review as some unis have them is about how many papers you've published (somehow weighted by journal/conference rank) and how many others are in the pipeline, you need to focus on papers until the point when your institution will let you do something more useful. Think of it as paper writing bootcamp so that once you do get more time for projects, you'll have practiced how to write up your results. > "Make your release usable, useful ..." This is excellent advice, also for anything else related to code.

light_hue_1 2y ago
> Making petrol engines 1% more fuel-efficient would be massive. Increasing the conversion rate of online ads by 1% could make you very, very rich indeed. Good advice for AI probably; bad advice in other fields. It's good advice in every field. Most improvements have a cost. If you make engines 1% more efficient but 5% more expensive no one will care. Heck electric engines are wildly more efficient (80-95%) compared to combustion engines (20-35%), not to mention far simpler, and we can hardly get people to switch. Even immense improvements can be hard to roll out. Don't work on marginal gains. Any minor problem or inconvenience will wipe them out and all your time and effort will be worth nothing. Let people who are paid by companies do that. If you're going to do a PhD focus on something that could be big. Whenever students suggest a project I always start by asking what the upside is: what if we massively succeed?
aleph_minus_one 2y ago
> Increasing the conversion rate of online ads by 1% could make you very, very rich indeed. I claim there is actually a rather easy way to do this (but it won't make you rich!). The basic idea is rather simple: for each potential (ad/product, user) pair, ideally store two kinds of information: - the estimated probability that the user will click on the ad and/or make a buying decision - the confidence that you have in the correctness of this estimate Then only show ads where, based on this data, the conversion rate will be high with a high probability. Result: by only showing ads to those few people who very likely want to buy your product, you create an insanely high conversion rate, but you loose money because the ad will be shown to rather few people (possibly these are even people who don't actually have to get "convinced" to buy your product). In other words: we just created an artificial example of Goodhart's law ("When a measure becomes a target, it ceases to be a good measure") in action. What this lesson tells us is - the "conversion rate" of an online ad is just a proxy for some entirely different business goal that you have - the conversion rate can rather easily be manipulated

light_hue_1 2y ago

> Making petrol engines 1% more fuel-efficient would be massive. Increasing the conversion rate of online ads by 1% could make you very, very rich indeed. Good advice for AI probably; bad advice in other fields. It's good advice in every field. Most improvements have a cost. If you make engines 1% more efficient but 5% more expensive no one will care. Heck electric engines are wildly more efficient (80-95%) compared to combustion engines (20-35%), not to mention far simpler, and we can hardly get people to switch. Even immense improvements can be hard to roll out. Don't work on marginal gains. Any minor problem or inconvenience will wipe them out and all your time and effort will be worth nothing. Let people who are paid by companies do that. If you're going to do a PhD focus on something that could be big. Whenever students suggest a project I always start by asking what the upside is: what if we massively succeed?

aleph_minus_one 2y ago

> Increasing the conversion rate of online ads by 1% could make you very, very rich indeed. I claim there is actually a rather easy way to do this (but it won't make you rich!). The basic idea is rather simple: for each potential (ad/product, user) pair, ideally store two kinds of information: - the estimated probability that the user will click on the ad and/or make a buying decision - the confidence that you have in the correctness of this estimate Then only show ads where, based on this data, the conversion rate will be high with a high probability. Result: by only showing ads to those few people who very likely want to buy your product, you create an insanely high conversion rate, but you loose money because the ad will be shown to rather few people (possibly these are even people who don't actually have to get "convinced" to buy your product). In other words: we just created an artificial example of Goodhart's law ("When a measure becomes a target, it ceases to be a good measure") in action. What this lesson tells us is - the "conversion rate" of an online ad is just a proxy for some entirely different business goal that you have - the conversion rate can rather easily be manipulated