Did all of the Elves leave Middle Earth after the defeat of Sauron?

In both the books and movies of The Lord of the Rings the elves go to the West. However it says nothing of the Mirkwood Elves heading to go West, nor does it say anything about them having any plans to.

Do they leave across the sea at The Grey Havens or do they remain? And if they do stay in Middle Earth, what happened to them? Did other elves stay as well?

I did not read The Silmarillion.