Well, it can be done.
Seat removed, saber saw held upside down under the shelf (it was near deafening and a little white knuckle!), and a couple of homemade 1/2" plywood fillers/adapters.
But if I had to do it again, I wouldn't have cut up the shelf. I would have made a couple of adapters out of 1/4" plywood to go underneath to fit that speaker up with that D-shaped hole. It can be gamed to lose only 2.7 square inch of opening.