You did the right thing by listening to your gut. I feel like very few people could pull that sort of relationship off without someone growing unhappy. I’m sure it still hurts though 😞