The fact that the dataset includes people who all live in the same zip code might get in the way of fairness.

Correct
It’s your responsibility as a data analyst to make sure your analysis is fair. Although many zip codes do reflect diverse populations, a better choice would be to include data about people who live in multiple zip codes.