Yeah...that might be taking it a bit too far. And as you guessed seasons and rain right now mostly effect wild fires, there are a few more things we wanted to add down the road.
1) Night - ranged units accuracy decreased by 100% - won`t be able to hit targets that well. And reload time is decreased as well.
2) Rain - unit movement speed decreased 30%, arrow damage decreased 30%.
3) Winter... this is a hard one
not really sure what we can do...