INA 215(b):
"(b) Except as otherwise provided by the President and subject to such limitations and exceptions as the President may authorize and prescribe, it shall be unlawful for any citizen of the United States to depart from or enter, or attempt to depart from or enter, the United States unless he bears a valid United States passport. "
http://www.richw.org/dualcit/law.html#Passport lists a couple of exceptions, one is indeed for adjacent countries in the Western Hemisphere, like Canada and Mexico.