Your options depend on the type of penalty that would apply.
If the penalty is a surrender charge or another penalty that would apply to an annuity or insurance product, you may want to consider leaving the assets in the plan until the time when the penalty will no longer apply. These penalties are not usually IRS-assessed penalties; rather, they are assessed by the company offering the product.
If the penalty is the 10% early-distribution penalty assessed by the IRS when an individual makes withdrawals before reaching age 59.5, then you may avoid that penalty by keeping the assets in the plan. If your plan is a 403(b) account, you may transfer the balance to a 403(b) account at another financial institution or roll over the amount to a Traditional IRA. These options are not taxable, and you will not be subject to the 10% penalty.
To learn more about this topic, read Taking Penalty-Free Withdrawals From Your IRA.
This question was answered by Denise Appleby