In Civilization IV the differences are more drastic, as each nation has a leader(s) with certain character that defines (or being defined by) 2 “traits”. In previous versions of the game, the differences included unique units for each civilization and different starting technologies. Of course, you can always choose “random” and let the fate decide for you, but even so you have to be aware of the fact that each civilization has its advantages, inviting certain strategy of play.
More have been added in the expansions, but we currently prefer to deal with the above ones.