The IRS considers marital status and provisional income to determine whether Social Security benefits are taxable.