“Vatican” gets used loosely to mean a country, a church, and a government all at once — which is confusing, because they aren't quite the same thing.

Three related things, often confused

Vatican City (the state), the Holy See (the Church's government) and the Catholic Church (the worldwide religious institution) overlap almost completely in practice, but are legally and historically distinct.

Why visitors of any faith come here

Most visitors come for the art and the history, not the religion — and that's a perfectly normal reason. Understanding the religious context just makes the rest make more sense.