My husband broke our date for the third time in six months. I'm so hurt. What should I do?

Generally, our date means, well, er, you know. My husband is good to me but I'm starting to wonder why he's avoiding me. We can only be intimate on "dates" because he needs the help of that magic blue pill due to other meds he takes. We date once a week and I'm starting to even think he's got a playmate. We talked about it but he says sometimes he feels rushed or tired. Its hard to feel special. I'm not overweight or bad looking so should I be concerned?