Fireplaces, Heating Stoves, Flues and Chimneys - Cleaning mortar residue off old fireplace

I have an old brick fireplace (circa 1958) where there is a lot of old mortar residue stuck to the brick surface. I don't really know if this was done intentionally to achieve some kind of effect, or if it was sloppy workmanship. Regardless, I'd like to get the old residue off the face of the bricks, and show off my beautiful red bricks! In most cases, its just a thick stubborn grey film with a small amount of grit. Is there a trick to getting this off, or special chemicals? I tried a scotch brite pad and water on one brick just for the heck of it, and it was quite tedious, though it was working. Any tips would be greatly appreciated.

Also, there is black soot stuck to some of the bricks too. What's the best way to get rid of that also? Thanks for all the help!

James