Yellow Jacket Wasp nest. If the wasps are black and white, you have bald-faced hornets, which are also a bit bigger than yellow jackets. Wasp nests grow to a population of several thousand.
Paper wasps are the only wasp in this area that reuses their paper nests from year to year. All the other wasps have an annual cycle. Paper wasps most commonly build their small, open faced nests under the eaves of buildings. Nests can be the size of a coin up to the size of a small saucer, with a population of up to a couple dozen wasps, but no more.
In-ground Yellow Jacket wasp nest. If you see wasps coming from a hole in the ground, this is most likely what you have.
Ground nests may also be bumble bee nests, so look carefully at the activity around the entrance. Bumble bees have more round bodies, and have hair, so they look fuzzy unlike the shiny bodies of wasps. Bumble bees usually make nests in old, pre-existing cavities such as mole holes, but they also can be in attics, or any loose rubbish heaps.
Bumblebee nest