Hi I think that it is not possible. There is no any attribute in addresses that make one of them as primary (unlike telephones where you can set one of them as primary and there is field Telephone Number that you can use). Thats why SD can not use Adresses as separate field.
A way round is to have a custom text field to hold the address and then youc an put it in a quick view. If you organizattions only hav eone address, you can use this in place of the standard Address field.