Well seems like opinion is mixed on what you should do. Here is another good link for you -
http://www.visajourney.com/faq/k1k2visa-enter.html#5.12
From this FAQ, which has two answers, one not recommending and one saying there is no problem
-
5.12)...Is it okay for us to go to the US Virgin Islands or Puerto Rico on our honeymoon without Advance Parole?
A...Maybe not a wise thing to do. Even though these islands are US territories or possessions, you may have to deal with Immigration and Customs to get back to the United States. You might have trouble getting back home if you encounter an insane USCIS (INS) inspector, plus, there are too many opportunities to "technically" leave US jurisdiction (by visiting the adjacent islands, for example). Best to take the honeymoon somewhere in the 50 States, where there is plenty to see and do.
A..(newsgroup)..Let me make it as clear as it was explained to me and which the regulation bears out: So long as the K-1'er does not depart ("depart" is also defined in the reg.) the US after making a legal entry s/he can travel to any other location which includes the USVI, PR, Guam, Hawaii, and Alaska.
The inspector I spoke to gave clear examples. (My alien fiancée) can enter the US and we can fly directly from the mainland to the USVI. We can go wherever we want on the islands so long as we don't "touch" ground or water not under US jurisdiction. When we leave she shows her K-1 and I-94 to immigration (which shows she legally entered the US) and off we fly directly to the mainland. However, if while we are there (USVI) we decided to take a trip to the British islands then she would be denied re-entry. I know I used "directly" in this example but we could fly "outbound" mainland-PR-USVI and "inbound" USVI-PR-mainland without fear because in this case she never "departs" the US.
Inspectors (both mainland and USVI) told me that this is not an unusual occurrence. They see K-1'ers all the time.
note that this context if for a K1 visa holder, so use you wiser judgement in interpreting.
hope this helps. If you do land up going, let us know how it went.
all the best.