Sticks and stones may "break" my bones, but words will never hurt me.
A popular saying can be: Go your own way and let others talk.
It is also Dante's famous saying:)