You need to understand that bringing your family later is not an easy, quick or cheap process. In general it is better to bring them with as the cheapest fastest option. If you bring them later, the costs of the application to do so are much higher than DV fees, there can be quite a time delay, and you have to prove you are earning enough already in the US to sponsor them.
Many people work lower-paying jobs at first just to help pay the bills until they get something they'd rather do. If you are not too fussy at first it shouldn't be difficult to find work. Remember if your kids are old enough for school, public school is free.
If you are sure you don't want them to come with then don't do medicals or interview for them, but be very clear that the idea you have in your mind that you can just easily bring them after a particular time may not be what happens at all.