flying turk
You said that you and your wife filed for an extension for tax return for the year of 2003, AND did not file any return for 2004, right? Then, there are two questions raised:
First, why a tax return was not filed by you guys for the year of 2004? Is it because you guys did not make REQUIRED income in 2004? If so, then you don't need to worry at all. Because, USCIS cannot expect you to have filed a tax return for the period when you were not even required to file it as per IRS laws. USCIS is concerned only those periods wherein tax returns were REQUIRED. But if you were REQUIRED to file tax return for the year of 2004 but somehow you failed to file it then your citizenship application CANNOT be approved until you file a tax return (you can file it even late), AND you will have to pay all the taxes and penalty that you might owe to IRS. If you cannot pay all the taxes that you owe to IRS then you must have to have something in writing about some kind of payment arrangement with IRS on taxes that you might owe to IRS. You would need to take that Payment Agreement with you at your interview with USCIS. So long, you would have all the paper work from IRS despite of owing taxes to them, you will be all right. Just because you did not/don't have money to pay owed taxes, that doesn't mean that you are in violation of any law of United States, so long you filed tax returns and made a payment arrangement with IRS on taxes that you owe.
Second, what is the current status on the tax return for the year of 2003? The answer to this will be the same. You need to file a tax return for a year wherein you were REQUIRED to file it. Then, you need to try to pay the taxes BY THE TIME OF CZ INTERIVEW, and not necessarily BEFORE FILING N-400. If you cannot pay all the taxes by the time of interview, then you must need to have all the correspondences from IRS on all these tax returns, especially Payment Arrangement with IRS on those taxes. Otherwise, don't even think in your wildest dreams that your citizenship application could have a chance to be approved.
And, it is very important for you to know that you CAN file citizenship application now even if you are in the mess of all these tax returns. Just include an honest explanation on this tax issues along with your N-400. You would have a couple of months to solve your tax problems with IRS as you would not be called upon for citizenship interview at least a couple of months after filing it. Meanwhile, you can try to resolve it. I hope you to understand also that even if you might owe taxes to IRS at the time of your interview on citizenship application, you would still be okay so long you would have some kind of proof about Payment Arrangement with IRS on owed taxes. Good Luck.