You can apply immediately after you get your naturalization certificate - though you probably want to make a copy of the certificate first.
If you apply at a post office or other passport acceptance facility, you can pay extra for "expedited processing". You will get your passport in a week or two if there are no glitches (for example, your pictures don't meet the specs). Make sure to follow the instructions carefully.
If you live near a regional passport facility (there are about a dozen, for example, in Texas, there is one in Houston), and you have plane tickets, you can get a very expedited passport in a day or three (I have no experience with this)
To apply, go to
http://travel.state.gov, fill in the on line form and print it out. Don't sign it (you sign it in the presence of a witness at the post office (or where ever) after taking an oath).
The US allows (but doesn't really recognize) dual citizenship. I have no idea if Bangladesh does. The "dual citizenship faq" (find using any search engine) is a good reference about dual citizenship.
As a US citizen, I believe you are supposed to travel on a US passport. Some countries have "citizenship cards" (for example, Canada). Canadian/US duals traveling home can use the US passport on the US side and the citizenship certificate on the canadian side.
I am not a lawyer, this is not legal advice or opinion, treat everything I say as if I was making it up.