Can you not just make the ball the same speed as the player? It would work then I think. So when it touches the ball, the ball moves the other way very fast so it bounces.
Would be really great if you edit this so I can see how it looks like with almost perfect bouncing :)

