As mentioned here, you are legally NOT allowed to go to Schengen countries that would normally require a visa from you. The exception to the visa rule for RTD holders or LPRs is a bilateral one that several countries make. In practice, they do not check within Schengen, and if they do, they might not know/bother to give you trouble. However, there is, of course, the chance that you COULD be detained for the lack of a visa, depending on many factors.
It seems that most of those on this forum who have been around Schengen without a visa after entering via a country with an exception have experienced no problems. I think there was only one person who faced some issues, but was let go in the end (I am not sure - does somebody remember?).
So, in the end, it is at your risk to do so. I personally would not, unless I was staying near a border and wanted to try to cross it for a quick trip or something. But, I am generally paranoid about being stuck somewhere with a travel document.
BTW, regarding visa requirements being based on citizenship and not travel document, this varies. The blanket rule is it is based on citizenship. But, on top of that, many countries make exceptions based on residency and travel history.