Here's the removed finder and the piece of felt that fell out of it. To glue it back in place, remove the ground glass by screwing loose the two springs that hold it in place.

Now you can clean the ground glass with whatever you want, it's real glass. Use water with dish washing agent, acetone, ethanol, isopropyl alcohol, whatever you have at hand whatever is needed to remove the dirt.