He said while he never lost his love of watching cricket "when it came to my part in the game I just hated it".
"I would rather be anywhere else in the world because I was convinced I was going to fail. I don't think anyone knew that I was struggling at all, not my best mates, not even my partner.
"I remember times when I would be sitting at home, she'd be at work, and I would literally be sitting on the couch, not bawling my eyes out but I'd be crying," he added.
"It was weird and really painful, it was hard to explain, but yeah I just thought I was letting so many people down."
Mental health has been thrust into the spotlight of Australian cricket in recent times with Glenn Maxwell, Nic Maddinson and young prospect Will Pucovski all taking time out to deal with personal struggles.
Paine said he turned the corner after confiding in a sports psychologist at Cricket Tasmania.
"It was the first time I actually told anyone what was going on, but I remember walking out of that room and instantly feeling better, that I had let someone in and that was the first step to dealing with, admitting I needed help," he said.
Paine finally won his place back in the Test team during the 2017-18 Ashes series, but the inner turmoil returned.
"It went from an amazing feeling ... and then I thought that's not good," he said.
"I'm going to have to bat in front of people and there are going to be millions of people watching. And for three or four days after I thought I don't want to do this.
"Again, spoke to some people and got that stuff off my chest and I thought bugger it, I'll just make the most of it ... I'm going to enjoy it."