Where magic actually lies? In wizards or wands?

I always thought about this whenever I saw Harry Potter movies or read books, that where exactly magic lies? Is it in wizards or wands? As mentioned in Harry Potter and the Philosopher's Stone, Harry is able to disappear the glass. Then in Harry Potter and the Prisoner of Azkaban he blows away his aunt without even touching the wand. So if magic is possible without wands why do they need it in the first place. But Ollivander tells Harry that wand chooses the wizards, so does wand actually have magic? Kindly help.