Yes. I have tried a levalier mic from Radio Shack and a little stereo mic that came with a cheap Sony tape recorder. There is hiss with all of them. I get the least noise with the levalier mic and the recorder set to "low sensitivity". But it's still there.
I have noticed that if I switch to "hi sensitivity" there will be noise as soon as I plug the mic in, before it's turned on. This happens with all the mics.