I believe that allowing someone to remain a part of your life after they've betrayed or hurt you is simply giving them permission and opportunity to do it again.
"Dear Maths Students,Please don't try to find us, we owe your teacher money.Sincerely,X and Y."
