Being "Smart" does not exempt an individual from being an idiot in some aspects/making idiotic comments and decisions. To fuck up is to be human, after all. Could have been chemical damage like you said, could have always been like that and just never had a chance to show it like others here have said. The reason's honestly irrelevant. The fact remains that Musk is kinda like that one person who seems pretty cool from the few interactions you've had with them, but when you sit down and have an actual conversation with them, they start saying things that make you raise a brow. And it is very, very hard for people to actually change in any meaningful way, so best bet is: he was always like that, but well, nothing lasts forever. Especially the masks we all use to hide ourselves from others. That's what people are saying.
That he's a rich brat who bought out most of the things he now owns rather than build them from the ground up is a known fact too, contrary to how what most people think and what (I'd assume) he'd like you to believe. People are also saying this.
People aren't saying he isn't smart, they're saying he's an idiot, protrays himself as smarter than he actually is, and is arguably, a fraud. All those aren't mutually exclusive.
Tbf, the OP pretty much made his stance clear out the gate: he refuses to believe "Musk has always been like that" answers. If he's not even willing to entertain the idea at all, and the vast majority of the replies are going to (correctly, IMO. Musk is no different than Jobs, or Edison) say that he was, yeah, he was never gonna engage...