Anyways, as soon as this left his lips, I told him "we are done". I hung up the phone. As I see is there are a couple of scenarios:
1. He was trying to get the upper hand by trying make me fear that I would be sued, so putting that to a stop was important. A profuse apology and no more talk of suing me would be a minimum for me to talk to him again.
2. He actually is going to have an attorney contact me. If this is the case, communicating further is also not a good idea.
In this case, I don't think we'll be having any major lawsuit over $100, but I would assume you would agree that anytime this comes up in conversation it is something that needs to be shut down immediately or else all communication should end.