Feelings for a Married Man?

Not too long ago, I ended an emotional affair I was having with an older, married man. It never grew into a real, "physical" relationship & I ended it for good recently. He's about 20+ years older than I am, I was flooded with memories of briefly being attached to this man. I miss him, but I know it's not right for me to feel this way. So why do I still feel this way? I'm tempted to contact him again but I know it's wrong and it's not a healthy relationship? Please help and please don't make me feel bad enough.