Something to keep in mind here: you can only be attacked by people with the same army strength as you. Of course, they could still have a much bigger army than you (like the orcs; they have huge armies, but they need them to make up for their weak troops).
As for this being a pay to win game... I'm sure some people spend plenty of money on this game, and they can get big armies and tons of resources out of it. But you can only be attacked by people with the same strength as you, so you would never have to worry about them.
And, like knight said, it's just as easy to get influence from actually playing the game as it is paying money. In the end, I would never
buy influence in this game, because it's really just a waste of money.
A good idea might be to start a new city. That way, you take things slower; and if you want, you could not build walls, so no one could attack you. But then you would miss out on all the fun of pvp.