Hello, docboston. Yes, you can finish your fellowship even if the H1 get approved much earlier. I actually had my H1 approved in January yet I'm not finishing until June 30. My H1b start date is on July, as well.
Upon approval of your waiver by the USCIS, the letter will state that you must start work within 90-days of approval. I was so worried about that until I learned from this forum that this has happened to many people before and it's no big deal. Furthermore, I called the coordinator of the J1-waiver program in the state where I got my waiver; she told me that it is not a problem, this "happens all the time" and I should not worry about it.