Failure due to over compression

If somehow I have a rubber which is under a state of over compression and I apply more pressure to it. How can I determine the failure mode? Shear, tensile, energy consumption? Which? And when do I know I am in a state of over compression?

I dont quite follow your question. I would perform experimental tests to determine the appropriate failure condition for your specific rubber. Often a strain or stress based failure model is sufficient as long as you dont have fatigue failure.

-Jorgen