In this case, the "cost" being referred to is the measurable cost of obtaining capital. With debt, this is the interest expense a company pays on its debt. With equity, the cost of capital refers to the claim on earnings which must be afforded to shareholders for their ownership stake in the business.
For example, if you run a small business and need $40,000 of financing, you can either take out a $40,000 bank loan at a 10% interest rate or you can sell a 25% stake in your business to your neighbor for $40,000.
Suppose your business earns $20,000 profits during the next year. If you took the bank loan, your interest expense (cost of debt financing) would be $4,000, leaving you with $16,000 in profit.
Conversely, had you used equity financing, you would have zero debt (and thus no interest expense), but would keep only 75% of your profit (the other 25% being owned by your neighbor). Thus, your personal profit would only be $15,000 (75% x $20,000).
From this example, you can see how it is less expensive for you, as the original shareholder of your company, to issue debt as opposed to equity. Taxes make the situation even better if you had debt, since interest expense is deducted from earnings before income taxes are levied, thus acting as a tax shield (although we have ignored taxes in this example for the sake of simplicity).
Of course, the advantage of the fixedinterest nature of debt can also be a disadvantage, as it presents a fixed expense, thus increasing a company's risk. Going back to our example, suppose your company only earned $5,000 during the next year. With debt financing, you would still have the same $4,000 of interest to pay, so you would be left with only $1,000 of profit ($5,000  $4,000). With equity, you again have no interest expense, but only keep 75% of your profits, thus leaving you with $3,750 of profits (75% x $5,000).
So, as you can see, provided a company is expected to perform well, debt financing can usually be obtained at a lower effective cost. However, if a company fails to generate enough cash, the fixedcost nature of debt can prove too burdensome. This basic idea represents the risk associated with debt financing.
Companies are never 100% certain what their earnings will amount to in the future (although they can make reasonable estimates), and the more uncertain their future earnings, the more risk presented. Thus, companies in very stable industries with consistent cash flows generally make heavier use of debt than companies in risky industries or companies who are very small and just beginning operations. New businesses with high uncertainty may have a difficult time obtaining debt financing, and thus finance their operations largely through equity.
(For more on the costs of capital, see Investors Need A Good WACC.)
RELATED FAQS

How do analysts and investors interpret Economic Value Added?
Understand how economic value added (EVA) is interpreted by analysts and investors and how to calculate this important metric ... 
How do I use the CAPM (capital asset pricing model) to determine the cost of equity?
Learn about the elements of the capital asset pricing model, and discover how to use this formula to calculate a business' ... 
How do I calculate the cost of equity using Excel?
Learn how to calculate the cost of equity in Microsoft Excel using the capital asset pricing model, or CAPM, including brief ... 
How do you calculate the proper weights of different costs of capital?
Understand how to calculate the weights of the difference costs of capital and how this calculation is used to determine ...

Working Capital
This ratio indicates whether a company has enough short term ... 
Net Present Value  NPV
The difference between the present value of cash inflows and ... 
Cost Of Capital
The required return necessary to make a capital budgeting project, ... 
Net Investment
The amount spent by a company or an economy on capital assets, ... 
Equivalent Annual Annuity Approach  EAA
One of two methods used in capital budgeting to compare mutually ... 
Replacement Chain Method
A capital budgeting decision model that is used to compare two ...