Nope, just as long as they don't miss the interview (sure they would know by now if it's schedualed). You can still travel at any time during the process (before the interview, after the interview and before the oath etc).
The only time you probably don't want to do short trips out of the country is right after you get your Citizenship and you're waiting for your passport. At that point you no longer will have your greencard and using the Citizen Cert for proof doesn't always work in place of a passport...
