The focus of this article is on the notion that, whether in forgiveness or non-forgiveness, negative moral emotions towards a wrongdoer may remain extant and continue to generate memories of wrongdoing. This constitutes a kind of objection to the wrongdoing, even a sort of protest against it insofar as the wrongdoing seems to undermine the victim’s sense of worth. I argue that certain non-retributive attitudes or emotions play under-appreciated roles in policing the boundaries of victims’ self-respect and I attempt to explain what makes such attitudes or emotions – those that reflect the deepest concerns of victims of wrongdoing – centrally important. I defend the need for diverse responses that better fit the nuanced reactions amongst those who seek to come to terms with their hurt, especially those choosing not to forgive. Whatever stance is adopted, protecting one’s self-respect remains the core value that victims of wrongdoing seek to defend.