The way I understand "Smalltalk" is not whether the subject matter is "serious enough" but rather whether either party actually has any interest in it, or if it is a polite nicety to avoid awkward silence.
Discussing the weather in a car ride with a coworker is smalltalk, contemplating with a friend how one might conquer the world using ant-controlling super powers is not.
This is not a counter-argument, it's a thought terminating cliche.