You can turn off the swiping gesture. I've done it on my AppleTV, and it is WAY better.
Yes, swiping works well on a phone...because there are no buttons. I don't want to swipe on my TV remote, my car radio, my laptop screen, and fridge, my door handle...
They're both saying, "I love you".