It’s one of the cardinal rules of retirement planning: pay off the mortgage before quitting work. Giving up your income while still supporting a big debt can mean chewing away at your retirement savings way too fast, and can leave you in a tight spot if something goes wrong.

But paying off a mortgage years early is easier said than done, and the Center for Retirement Research at Boston College says way too many pre-retirees are too far behind schedule, largely because of borrowing before the housing bust and financial crisis.

On the other hand, some experts say carrying low-interest debt into retirement is not always such a bad thing, especially if it means leaving money in investments that perform well.

“In 2013, almost 40 percent of all households ages 55 and over had not paid off their mortgages, up from 32 percent in 2001,” the Center reports, citing a study using data from the Federal Reserve’s Survey of Consumer Finances in 2013. “These borrowers were also carrying a lot more housing debt by 2013.”

“I’ve been advising clients for over 20 years and on just an anecdotal level, I can tell you that more clients are retiring with mortgage balances than in years past,” says Margaret R. McDowell, founder of Arbor Wealth Management in Miramar Beach, Florida.

A.W. Pickel III, president of the Midwest division of AmCap Mortgage in Overland Park, Kansas, says many baby boomers traded up as their families grew, then took second mortgages to help fund college costs.

In the years before 2008, homeowners were encouraged to take out big loans when home values appeared to be soaring, the center says. They bought expensive homes or tapped home value through cash-out refinancing or home equity loans, it says.

When home prices collapsed, millions were left “underwater” – owing more than their homes were worth – and were unable to get out from under because they could not sell for enough to pay off their loan. McDowell believes many homeowners also concluded their home was not the rock-solid asset they’d thought, so they felt it unwise to pour more money into it by paying down the mortgage early.

So many just hung in there. By taking on too much debt, and monthly payments so large they could not afford extra payments to bring it down, they left themselves with too much debt too late in the game.

The center says “that 51.6 percent of working-age households were at risk of having a lower standard of living in retirement,” largely because of mortgage debt.

“In recent years, U.S. house prices have started to really improve, to the benefit of homeowners and retirees,” the center says. “But it’s difficult to predict whether the other factor that has reduced retirement preparedness – more older households with big housing debts – was a boom-time phenomenon or represents the new normal.”

But is the situation really as dire as it seems? David Reiss, a professor at Brooklyn Law School in New York City, thinks it may not be.

“According to the National Association of Realtors, the median sales price of an existing home increased from $197,100 in 2013 to $232,200 in October of 2016,” he says. “That is a roughly 15 percent price increase and about $40,000 of additional equity for the owner of the median home.”

Many homeowners who were underwater may not be any longer.

Also, he adds, it’s not necessary to be absolutely debt free at retirement so long as income is large enough to cover expenses and leave a cushion.

“Often, paying off a mortgage gets a retiree where he or she needs to be in terms of that balance, but it is not always necessary,” he says.

The key, he says, is to not be underwater. Once the remaining debt is smaller than the home value, the homeowner is better able to sell. One option is downsizing, selling the current home, then using cash from the sale or a new, smaller mortgage to buy a cheaper home. A less expensive home will also likely have lower property taxes and maintenance costs.