"Life is only the first time" means that your ex-boyfriend likes the first time you meet, because the feeling of the first time at that time is the most beautiful. I feel very helpless, and I always can't see the first sight in the later development.

"My seriousness lost to your black humor." I think that's why you broke up. He is too serious, but you don't seem to be

He shouldn't hate you, but he regrets that this relationship didn't last.