Couldn't anyone defeat Voldemort if they destroyed all of his Horcruxes?

My question is fairly straight forward.

I do recognize the prophecy which states,

"The one with the power to vanquish the Dark Lord approaches ... born to those who have thrice defied him, born as the seventh month dies ... and the Dark Lord will mark him as his equal, but he will have power the Dark Lord knows not ... and either must die at the hand of the other for neither can live while the other survives ... the one with the power to vanquish the Dark Lord will be born as the seventh month dies ..."

Is this the only fact to counter-attack my question? Is Harry the only one to vanquish the Dark Lord? And that they must die at the hand of the other? Voldemort would be mortal again when his Horcruxes have been destroyed, so a mortal man could be killed, with difficulty of course, as Voldemort is an extremely powerful wizard.

I may be easily overlooking the answer, however, why couldn't anyone kill Voldemort when his horcruxes have been destroyed.