You have to be 'Full-time' at the HPSA site. Some states specify that as 40hrs, often you can get away with 32-34hrs at the site plus 8hrs administrative time.
If you have other HPSA sites you go to, you can let state health know and they would typically not object to it. If you go to non-hpsa sites, you have to make sure that you document every hour you spend at your HPSA site.