I've wondered that too. Maybe there's a point where variations in density below the satellite would interfere enough with the orbit to make it unstable? I.e. it would be possible if the moon were smooth and of uniform density, but it isn't?
ETA: It seems there are mountains 7km tall on the moon, so 30km may just be a margin of error / ability to correct kind of thing then.
I haven't done the math, so this is speculation, but:
Variations in the moon's gravitational field due to mountains, density variations, etc, will indeed cause some orbital variation. I wouldn't call itunstable, really - it's not like the orbit is critically well-balanced and would rapidly decay if something went wrong, although that always seemed to be an issue on Star Trek. But the orbit would vary some, and those variations might show up as a lower periapsis. If the periapsis was 23 km lower, you'd have to worry about whether it lined up with the wrong mountain and go splat. Since the orbit radius is measured from the center of the moon, 23 km isn't necessarily a huge change.
But (again, haven't done the math) irregularities in the moon's gravitational field are probably a small problem compared to the rest of the solar system. The Earth's gravitational field is still quite significant out there, and the sun's is even bigger (though I'm pretty sure the sun's tidal effects are smaller). The constantly varying earth-sun-moon geometry will perturb the orbit, and probably a lot more than the moon's gravitional anomalies. So you probably want 20+ km of altitude margin at beginning-of-life (BOL) just so you can be confident that you'll still be above the mountains at periapsis at the end of your life.
Certainly, spacecraft in orbit around the earth see their apses drift if no stationkeeping is done.
Also, and probably most important*, as one of the other posters noted, flying low to image Apollo wasn't the main mission
*most important to the mission, not most important to this post.