From The Heroes of Might and Magic III wiki

Units fighting on their native Terrain gain +1 Attack, +1 Defense and +1 Speed. Also, if a hero moves over a terrain and only has units with that terrain as their native, all Movement Penalties are removed.

[edit] List of Towns and their Native Terrains

• Castle - Grass
• Rampart - Grass
• Tower - Snow
• Inferno - Lava
• Dungeon - Subterranean
• Necropolis - Dirt
• Fortress - Swamp
• Stronghold - Rough
• Conflux - Grass
• Neutral - Dirt

Towns are always on their Native Terrain so in a siege defenders always have a Terrain bonus.