If you don't have empathy, it's because you're not CHOOSING empathy. You're choosing something else. That's your decision to make and only you are responsible for it.
It's not an emotion. It's a decision.
You don't have to empathize or be considerate or care, but you do have to own that decision. You can't lack empathy and avoid responsibility for that choice.