Because they want to price their tickets based on the value you receive, not their costs, to maximize their profit. AA sold the ticket assuming the customer wa

Because they want to price their tickets based on the value you receive, not their costs, to maximize their profit. AA sold the ticket assuming the customer was going between two small cities, with the "inconvenience" of a layover in the middle, so thus had a low enough value to justify a low cost. A direct flight from a small city to a big city, on the other hand, is assigned a much higher value, and thus is priced much higher. This is despite the fact that the former has a higher operating cost than the latter. Still doesn't explain why a gate agent decided to play detective and save the giant corporation employing them a bit of money on a hunch, screwing over the customer they're directly facing in the process.