Malel, please search this forum,....as if I read of something before in you case...it seems somebody just traveled out with his/her NP and returned with a blank RTD.
In my understanding, the IO wouldn't challenge you in that case because as long as your NP is still valid you can use it as the travel identity, and if you wanna re-enter US you'll need RTD/AP which you've already had. So I assume the function of RTD to Asylee/refugees is the same as AP to LPR, it is just a reentry permit. I think when you applied for RTD, you have already claimed that you will travel out and DHS issued you the RTD meaning they approved your re-entry. Of course for refugees or asylees RTD can work as a temporary passport since some people lost their nationality, but since you hold the valid NP, and you got a "re-entry permit", why not let you in?
Anyway, you can also write to DHS to ask. Their answer should be the best one to take;-p
Good Luck!