IIRC, there are a couple of nuts that hold it onto the trans tunnel.  If you lift up the rear of the cover (pull slightly towards the rear of the car and lift up) it will tip forward (or come completely off depending on your cover's condition) and you should be able to see a couple of nuts or bolts holding the handle to the car.  If you have a Mustang II or other non-stock center console, you will most likely need to remove it to access the hardware to tighten.

My guess is that over the years, one or both of them have come loose.  Worst case scenario would be if the mounts had broken loose from the sheetmetal or the handle itself is cracked, but since you say it is fine when the pressure of the brake cable is added, this leads me to believe it is just loose.

Good luck!
--Doug