The Atmosphere

At every elevation, the weight of all the air molecules higher than that location must be supported by the upward force exerted by all the air molecule collisions from underneath. (Otherwise, Newton's laws say that the molecules above it would fall downwards toward the ground.)

Since the total number of molecules above you in the atmosphere decreases with elevation, so does the air pressure. But, the drop in air pressure with altitude goes further than this. The Earth's gravity also has its part to play. Because of the pull of gravity, the number of molecules in a unit of volume (or number density) decreases with altitude as well. A basic simulation of an atmosphere shows this effect. As a result, the air pressure decreases faster with elevation than what you would expect if the air density stayed the same.