The N400 form only prints residency and family related requirements on it, so from what you disclosed, I see no reason why you couldn't apply any time under those conditions. The only pertinent N400 question may be the one about previous declaration of US citizenship, but you can add a note that you did that legally, with a copy of your old US passport, I guess, so it can't ban you. A lawyer may even have some USCIS form alternative to N400 that titles something like "re-application for citzenship", although I've never seen one.