.. _data-sync-types:


Data sync types
---------------

.. _18.1.2|DOC-225:
.. _20.1.1|VOSS-620:
.. _19.3.4-PB1|EKB-7436:
.. _21.1|EKB-7436:



Overview 
..........


Automate provides the following data sync types:

* :ref:`data-sync-type-pull-from-device`

  This data sync type is available to all device types:

  * Pull all data from the device
  * Pull only the schema from the device (used for LDAP)
  * Pull data from the Change Notification Feature (CNF) local data collection

* :ref:`data-sync-type-push-to-device`: Available only to Cisco UCM devices. Push data in the cache to the device
* :ref:`data-sync-type-cnf-sync`: Available only to Cisco UCM devices
* :ref:`data-sync-type-purge-local-resources`: Available only to Cisco UCM devices



.. note::
   A quick import option is available to fetch only summary data that is contained
   in a list operation response and not the data for all instances/fields. See 
   *Data Sync Overview* in the Core Guide for details.


For all sync types, Automate typically builds up the lists of entities from both Automate and the device and 
compares them, using the key for the device entity. 

The key is typically the unique identifier (ID) for the record in the device we're syncing with. 
For example, for Cisco UCM, the ID is the *pkid*, which is the internal Cisco UCM database ID.

For users, a sync builds up the list of ``device/cucm/Users``
in Automate and then requests from the Cisco UCM the lists of users it currently has
for the comparison. Differences in the lists are handled based on the sync type.


.. rubric:: Related topics

*
  .. raw:: latex

     Data Sync Overview in the Core Feature Guide

  .. raw:: html

     <a href="concepts-data-sync.html">Data Sync Overview</a>

*
  .. raw:: latex

     Cisco UCM Change Notification Sync (CNF) in the Core Feature Guide

  .. raw:: html

     <a href="/src/user/cisco-ucm-cnf-syncs.html">Cisco UCM Change Notification Sync (CNF)</a>


.. _data-sync-type-pull-from-device:

Pull from Device
..................

In a *Pull from Device* sync type, the Automate resource is updated where the same key is present in both 
lists. In this case, the device data is the master and the Automate system model data is updated with the 
device data.

For example, let's say new data is added to the Cisco UCM, so that the Automate system data
state for a Cisco UCM ``device/cucm/User`` does not show instances that are shown on the Cisco UCM.

In this case, a *pull* data sync synchronizes the system data with the
|Unified CM| data. For example, a user's Department may be updated on the
|Unified CM|, but the update only shows on the system after a *Pull from Device* sync.
If a user resource is created in |Unified CM| but not in Automate, this adds the
``device/cucm/User`` instance into Automate at the level the *pull* sync was run from,
for example, at the customer level.

When deleting a Automate resource from the device, so that the key is in the
Automate list but not in the device list, a *pull* sync removes the resource
in Automate. For example, if the resource is a user in Automate but not in
|Unified CM|, the *pull* sync removes the ``device/cucm/User`` record in Automate.

To restrict the number of records removed in Automate, ensure you have the
following named macro at the hierarchy where the sync takes place:

`PULL_SYNC_DELETE_THRESHOLD_<device_type>`

.. raw:: latex

   For details, see Pull Sync Delete Threshold topic in the Advanced Configuration Guide.

.. raw:: html

   <p>For details, see <a href="reference-macros-sync-delete-threshold.html">Pull Sync Delete Threshold</a>.</p>


When pulling device data, for example LDAP users from an LDAP device, the
results returned to Automate depend on the LDAP server configuration. For example,
if the returned results exceed the LDAP server configured maximum, and if the server
does not support paging, an appropriate error message is returned.

For Microsoft 365 syncs, a **Max page size** (default 1000) setting can be adjusted
if the error "Template output exceeded memory limit" is shown. 

.. raw:: latex

   For details, see *Microsoft tenant setup* in the Core Guide.

.. raw:: html

   For details, see <a href="microsoft/ms-tenant-setup.html">Microsoft tenant setup</a>.


.. _data-sync-type-push-to-device:

Push to Device
..................

The *Push to Device* sync type (available only to Cisco UCM device types), synchronizes devices with 
the Automate system data state, which is the primary data state. 

* When deleting device data from Automate so that the key is in the *device*
  list but not in the Automate list (for example, delete user in Automate),
  the user is removed from |Unified CM|. The user will not
  exist on the device or on Automate.

* When adding new device data to Automate so that the resource
  shows instances that are not shown on the device, a *push* data sync
  synchronizes the device data with the Automate data. For example,
  adding a ``device/cucm/User`` instance to Automate and running a *Push to Device*
  sync adds the user record to |Unified CM|.

Keys found in both lists are ignored. Existing records are not updated in either direction.

In the ``device/cucm/User`` example, if the same user exists on both Automate
and on |Unified CM|, no update occurs in either direction. Detailed settings
may still not match after a *Push to Device* sync.

.. important::

   Consider data dependencies between different models when running a *Push* sync. For example, 
   data dependencies may exist between users and phones in the Cisco UCM. In this case, if a user is 
   associated to a phone (via the associated devices on the user), you can't add the user if the phone does 
   not yet exist in Cisco UCM. However, for *ownerID* on the phone, pushing the phone first will fail 
   since the user isn't in place. That may require running the *Push* sync multiple times so it
   loads in the required order.Alternatively, you may need to modify data (such as removing device association) 
   to allow the *Push* sync to succeed. 

.. note::

   The keys list sync logic described in this topic implies that in case of a reversion
   of the |Unified CM| to restores/inactive partitions, the end-state of the relevant pkids
   may differ to their state the last time Automate was in sync with |Unified CM| (before a restore),
   particularly if testing occurred in between. This means you may, for example, have
   a user with the same username in both Automate and |Unified CM|, but if that user's pkid in
   |Unified CM| now differs to the one in Automate from previous
   syncs or interactions, they will be seen as different users even though they have the same
   usernames.

.. * Merge with Device - This is a combination of pull and push data synchronization and takes place in both directions.
.. Merge data synchronization synchronizes data between the system and the |Unified CM| data without overwriting or deleting any data.

.. * If you create a resource where the key is in the 'device' list but not in the Automate list (for example, the entity is in |Unified CM| but not in Automate), then the Automate resource is created. For example, adding a user in |Unified CM| will create the ``device/cucm/User`` record in Automate after the merge sync.
.. * If you add to a device so that the key is in the Automate list but not the 'device' list (for example, the entity is in Automate but not in |Unified CM|), then the device resource is created.  For example, creating a ``device/cucm/User`` record in Automate will create the user in |Unified CM|.
.. Keys found in both lists are ignored, for example no updates will be made for existing records in either direction. So in the ``device/cucm/User`` example, if the same user exists in both the Automate and |Unified CM|  system, then no update occurs in either direction. In other words, detailed settings may still not match after a Merge with Device sync.



.. _data-sync-type-cnf-sync:

Change Notification (CNF)
..........................

The CNF sync type is available only to Cisco UCM device types.

A CNF sync type is a pull sync of changes stored in the local collection that
is updated by the Change Notification Collector service.

For more details on Change Notification Sync, see the related topics in Data Sync section of the
Core Feature Guide.


.. _data-sync-type-purge-local-resources:

Purge Local Resources
..........................

A *Purge Local Resources* sync type deletes all resources or instances of device information that exists 
in the system are deleted. Entities in the device are not deleted.

..  note::

   The default *purge* syncs created when adding a Cisco UCM, CUC, LDAP or CCX server
   type are disabled by default. To use the *purge* sync, the "Remove" check box
   must first be cleared on the "Disabled Operations" tab of the relevant sync.

This sync type is typically used when cleaning up the system. The system displays a warning
before executing an enabled *purge* sync.

See the following sample device type syncs:

* HcsPurge-{{CUCMHostname}}--{{CUCMClusterName}}-DS
* HcsUserPurgeDS-{{CUCMHostname}}--{{CUCMClusterName}}
* HcsPhonePurgeDS-{{CUCMHostname}}--{{CUCMClusterName}}
* HcsPurge-{{CUCXHostname}}--{{CUCXClusterName}}-DS
* PurgeUccx-{{UCCXHostName}}
* HcsLdapUserPurge--{{UniqueID}}
* PurgeSpark{{CustomerName}}



.. |VOSS Automate| replace:: Automate
.. |Unified CM| replace:: Cisco UCM