I've done head-to-head tests vs. Claude 3.5 Sonnet on a variety of coding tasks and basic reasoning questions over the last week. It's not worth $200. Claude

I've done head-to-head tests vs. Claude 3.5 Sonnet on a variety of coding tasks and basic reasoning questions over the last week. It's not worth $200. Claude is better on most tasks. For example, I pasted my HN profile and asked it to extract my email address in JSON format ({"email": ""}). I would expect any HN reader to be able to do this in seconds using an online ROT13 calculator. Certainly a "phd-level reasoning model" should be able to do this correctly. Claude outputs the correct answer in seconds. o1 Pro thought for two minutes and eventually output an email address that was invalid. I'm a little surprised that there isn't more discussion about this on HN, as it seems highly relevant to the recent pivot from training-compute to inference-compute.