If you make any changes to any of your My CRM contacts, the changes are reflected in Outlook the next time you synchronize.
When you remove a contact from Contacts in CRM or if the contact is deleted from CRM and therefore automatically removed from Contacts, it is deleted from Outlook the next time you synchronize.