When you enter the US for the first time, you will receive another stamp in your passport called I-551 which can be used for travel without the plastic card. The stamp will normally be valid for 6 months.
However, you should be aware that if you simply turn around and leave the next day, and then stay outside the US for a number of months, when you apply for citizenship they may say that your 5-year clock did not start when you arrived on that initial 24-hour stay ... rather it started at a later date, when you returned to the US and began to actually live like a US resident. That could delay your citizenship eligibility for a few months.