This is to say that we’re facing a coordination problem around executives being honest around the AI gains they’ve witnessed – if they co-operate, they keep their jobs. If they defect, they will possibly be fired by their embarrassed peers (who have now been implicitly called liars, cowards, or incompetents) and then replaced with someone that will toe the line anyway. If they could all admit the truth at once there might be some hope, but there is no way to coordinate that event.

I think it’s interesting to notice that this is not AI-specific. He kinda alludes to that in a different context but I think it’s important for people in the industry (and any industry really) to understand. This is driven by the incentives built into the system. Specifically the competition for profit. Whenever there’s a thing (technology, process, method) that promises significant profit, everyone and their mother jumps on it. For the simple reason that if they don’t, they’re getting replaced by someone who would. For the simple reason that if they don’t jump in, capital is going to leave their company and enter another one that promises to jump in, and therefore has higher expected profit. This can look as simple as large investors dumping your shares for the shares of a different company, sending your stock price in steep decline. Everyone competes for capital which requires competition for growing profits. Capital goes where the expected profits are higher. Without capital, firms can’t hire people and buy what they need to conduct business and create new products. This incentive makes it very difficult to avoid a situation like the one above. We’ve seen this in corporations with the move to the cloud, with outsourcing development to CHWTIA and so on.