I would have said that it important to air it and keep it on a raised dish with as many drainage holes as possible. As you've tried all that without success, perhaps you could just get a soap dish where it's easy to get the slops out and use them for washing your hands every second or third wash to give the soap a chance to harden up between times. Also prevents waste as you are using what you would otherwise just rinse away.
My parents used to buy soap in larger batches and keep it in the cupboard for ages to harden up. I'm not sure if it helped, once the soap was exposed to water, but it might be an idea to try.
I put the paper the soap comes wrapped in in my knicker drawer until the smell has faded.