Originally posted by JoeF
If you operate as a sole proprietor, it usually doesn't cost anything. You just write invoices, and pay taxes on that. You usually have to pay quartely estimated taxes. At tax time, you get 1099s from your customers, and you can of course deduct any business expenses from your business income. You need to file Schedule C (Profit or loss from business) with your tax return.
It is advisable to have a different bank account and creditcard for the business to separate business and personal expenses and income.
All the complicated stuff only applies if you actually have employees and/or expensive inventory, or if you incorporate the business. With incorporating, you need to have bylaws. This is usually done through a lawyer, and can cost you a couple $1000. You also would need a tax id, and possible a business permit from your city in this case.
Oh, and while it may be ok to have more expenses than income originally, IRS eventually wants to see some profits (and taxes paid), otherwise they can end up declaring the business as a hobby, and expenses for hobbies are obviously not deductable.
You should also get a tax accountant (look in the phone book under CPA). They know all this stuff.