Conventional methods for evaluating residual stress in mechanical components, such as X-ray diffraction and the hole-drilling method, are time-consuming. To improve efficiency, a non-destructive technique—the ball impact method—was proposed to rapidly assess the surface residual stress of components. This study investigated the response of a WC (tungsten carbide) ball impacting an Al2024 substrate under uniaxial residual stress using the finite element method. The effects of residual stress amplitude, impact velocity, and ball size on the coefficient of restitution (COR) were analyzed. The current results indicated that the COR decreases with an increase in tensile residual stress but increases with compressive residual stress. Furthermore, the impact velocity of 1 m/s is more suitable than those at 0.5, 3, and 5 m/s, as it results in a more apparent variation in the COR when the residual stress changes. However, ball size did not show significant influence on the COR. Additionally, a theoretical equation based on the Winkler model was developed, demonstrating high accuracy in predicting the tensile residual stress when the impact velocity higher than 1 m/s. Future studies will explore other influencing parameters, such as the biaxial residual stress field and substrate thickness. Experimental work will also be conducted to validate the reliability of this method.