The tax collector could be an agent, not of the people, but of the very rich people.

You mean that the tax collector would change the tax code at the moment the ownership is changed? Why otherwise would very rich people pay tax from a plot that does not produce anything? And when the land becomes more valuable, the tax increases even more. You are not talking about land tax at all. You are talking about "poll tax."

The tax collector is an agent of the very rich people now. They tax labour for the benefit of banks and property owners.