Are you sure that you didn't produce all of them? Sometimes game is buggy and produces more and more units, even though you didn't choose that. Try relogging, and be sure that your stables isn't producing any more units.
Also, when you are viewing your army/town screen, same units will stack one on each other.
Community Manager - feel free to PM me about any game related issue, bug, question...