Social Security is actually designed to pay you the same total lifetime benefit regardless of when you sign up, assuming you live an average lifespan. But if you don't expect to have a long retirement ahead of you, then it's generally better to file early.
Say you're entitled to $1,500 a month in Social Security at a FRA of 67. If you delay your filing until age 70, you'll boost your monthly benefit to $1,860, but you'll also collect 36 fewer payments. Plus, it'll take you until age 82 1/2 to break even. If your health isn't great and you think you'll pass away before 82 1/2, you should file earlier.
Your goal should be to have money available in your IRA or 401(k) plan for withdrawals throughout your entire retirement. But if delaying Social Security causes you to spend down your savings too quickly, then you could end up in a tough financial spot later in life.
Leaving some money in your IRA or 401(k) is important, because that money can, and should, stay invested so it grows into a larger sum. But your money can't grow if there's none left.