Accountant aggressively refusing refund?

I signed Terms Of Business with firm of 1 man band accountants and paid initial deposit of circa £2150. I never met this accountant face to face only transacted over the phone, email and once video call. After signing terms of business the only job he has done was to complete simple 1 page HMRC form, that’s all.

After 10 days, I have decided that I no longer wish to proceed with his services and sent him an email asking him that I wish to cancel exercising my cooling off period under consumer rights and distance selling. He initially called me with very aggressive call and then threatening to use the services of legal firm. I offered him £400 which is well in excess of time he has spent he asked for 50% then later withdrew his offer and asking me if I consider myself business or consumer.

Please note, his Terms Of business does not provide details of how to cancel. I have had instances, when I decided to withdraw the services of conveyancers half way through purchase in which case they either did not charge me at all or charge only for the work done, but this person appears to think I am committed to contract no matter what my circumstances are.

For the avoidance of doubt we are not business, but merely run our portfolio as individuals and Terms Of Business was signed on personal names and payment was made using personal account. We do not have any official business. As I understand, there ought some solicitors may read this and I would appreciate any steer or template I could use?

Many thanks

Simon