According to Newton’s Third Law if your hand exerts a force on the ball, the ball exerts a force on your hand. Thus there must be an impulse on your hand. Does the momentum of your hand change? Of course your hand is just a part of your body, and if you are standing firmly on the ground, your body is unlikely to move. Let’s pick a simpler situation. Suppose you are sitting in a desk chair with wheels on a smooth floor and you catch a heavy ball thrown by someone. In this case you and the chair will roll backward. In other words, the momentum of you and the chair will increase as the momentum of the ball decreases.