Water Heaters - Pilot Poops Out

I noticed the hot water wasn't hot today and looked at the heater; the pilot light had gone out. I started it, turned the control knob to "On" and the burner kicked in. Later I looked at the pilot light and it had gone out again. Repeated the procedure and the same thing happened. The pilot does not go out immediately when the burner turns off, but waits 'til I'm not looking.

Thermocouple? or is the control valve shot? The heater is 20 years old...

Try the thermocoupler -- they're cheap. If the problem persists, replace the entire WH. At 20 years old, your's is at the end of its useful life (you'll probably be replacing it in 5 years or less anyway), and the gas valve is too expensive of an item to put into it.