Denise Richards and Charlie Sheen, who share two daughters, were married from 2002 until their very public divorce in 2006

Denise Richards and Charlie Sheen. Photo: Rich Fury/Getty; David M. Benett/Dave Benett/Getty

Denise Richards is getting honest about her turbulent past with Charlie Sheen.

In the April 1 episode of her new reality series, Denise Richards & Her Wild Things, she reunited with the actor, whom she was married to from 2002 until 2006, to get lunch with their 19-year-old daughter Lola.

As they enjoyed their family time, Denise, 54, admitted that it was anything but an easy road to get there.

“Charlie and I were married for four-and-a-half years before I filed,” she shared. “He had been sober for about four years and very committed to his sobriety, and I never thought someone that committed could fall off and fall back into that because I, at the time, hadn’t really been around it.”

Denise Richards and Charlie Sheen’s Relationship Timeline

Denise Richards and Charlie Sheen. Gregg DeGuire/WireImage

“Right after I got pregnant with Lola, it started to turn fast,” she recalled of Charlie, with whom she had already welcomed her first daughter, Sami, the year prior. “One day, he went to work — there was an awful evening the night before — and I packed my s—, packed Sami’s stuff, called my housekeeper to come, and we went to the Beverly Hills Hotel, and I filed for divorce in between dropping them off at the hotel and a table reading.”

When Lola admitted that she was “scared of being famous” like her parents, Charlie, 59, said that his best piece of advice was not to read or engage with anything written about her.

Hearing this brought back memories for Denise, who reminisced on the “disgusting” things Charlie, who has been sober since 2017, used to say about her. “I always had your back at the end of the day,” she told him.

“I appreciate that,” he replied. “If you didn’t, then we wouldn’t be able to have moments like this.”

In her confessional, Denise explained: “I never had to say anything bad about Charlie — he spiraled.”

Denise Richards Recalls Waiting for ‘S—‘ to ‘Hit the Fan’ After She Filed for Divorce from Charlie Sheen While Pregnant

Denise Richards and Charlie Sheen. Gregg DeGuire/WireImage

She then recalled taking temporary custody of the twin boys Charlie and his ex-wife Brooke Mueller share in 2013 when they were both unable to care for them due to substance abuse issues.

“I didn’t publicly say I did,” Denise emphasized. “I was in the tabloids all the time, and it got to a point where I had to literally put myself, basically, in a bubble. I told my publicist, ‘Don’t send me any article.’ Because it was getting to me. I was getting to the place of being rock bottom. I became a shell of myself.”

Growing emotional, Denise noted that her top priority was always to make sure her children had a good life.

“I would tell Charlie, ‘I don’t care if you hate me,’” she remembered. “‘You can think what you want to think about me. When we are at a birthday party, or a horse show, or a school function, can we please just fake it and be so kind and loving to each other in a respectful way so that the kids feel loved?’”

“At the end of the day, what really matters? It’s my daughters and my family, and that’s all I care about,” she added.

These days, there’s no bad blood between the exes as Denise recently told PEOPLE: “I believe in destiny. I wouldn’t have Sami and Lola if it weren’t for him. Despite everything we went through, we can have a sense of humor about all the s— we went through.”

“It was such a difficult hard time,” she added. “He knows I will always be there for him and I’m there for his sons. Now we look back and we’re like, ‘Oh God…’ But at the end of the day, we’re here for each other.”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

Denise Richards & Her Wild Things airs Tuesdays at 9 p.m. ET on Bravo.