Microsoft Sync Overview#
Overview#
The Automate system interfaces and syncs with a range of Microsoft applications, depending on the specific setup and use in your environment. The table describes how these applications sync in to Automate:
Application |
Synced in via |
Notes |
|---|---|---|
Microsoft Entra ID (Office365) |
Graph API |
Syncs in Msol users. Used for managing licensing. |
Microsoft Teams |
PowerShell |
Syncs in Csol users that have a matching Msol entry. Used for managing voice, voicemail, and collaboration. |
Microsoft Exchange Online |
PowerShell |
Note
Azure Active Directory is now Microsoft Entra ID.
Related Topics
Sync from Microsoft Entra ID / Office365#
Syncs from Microsoft Entra ID (Azure Cloud) sync in Msol users and are used to manage licensing.
This sync fetches users and licenses from the Microsoft Entra ID instance for the tenant. This can either be the source AD or it could be synced with an on-prem AD which is the real source. A combination of sources is also possible depending on the setup.
The licensing for cloud services is controlled on this user object in Microsoft Entra ID.
Primary purpose of syncs with Microsoft Entra ID:
Provide the user objects and details that are in the Microsoft Entra ID environment, including existing settings and licenses.
Fetch license details for the Microsoft tenant - which licenses are valid for the tenant, current available/used statistics, as well as the service plans for each License SKU.
The setup and timing of the sync can depend on how much of this data is required for your environment and how often you’re viewing and using the data. For example, if you’re using Automate to assign licenses to users as part of user onboarding, then syncing in users and licenses is important and needs to be timely in order to pick up new users in the system that need licenses. However, if licensing is being handled outside of Automate, this sync is less critical in terms of frequency.
As this is a sync with the corporate AD, it is recommended that you apply a filter to control what’s being synced in to the system. Filters can limit the number of users synced in to save load on the system, to reduce the time it takes for a sync to execute, and to enhance data security by ensuring that only relevant data (required for Automate’s management purpose) is synced in.
These types of filters are set up via model instance filters (MIFs) in Automate.
The table describes common use cases for filtering in syncs:
Geography |
Filter users by geography - certain cities and/or countries - potentially for a limited rollout of voice services in some geographies/sites, but not others. |
Licensing |
Filter users based on O365 licensing. You can have a simple filter that only pulls in licensed users (with any license), or you can have a more specific filter to identify licenses you want to match. The “IsLicensed” field is a filterable field for MsolUser and is the easiest filter (any license). Building a list of license IDs and permutations is more complex, but can be useful to narrow the sync down to a specific service (for example, those with licenses related to voice services such as E5, Phone System, or Calling plan). When contemplating this condition, consider whether you’re managing users without a Phone System - for example, MS Exchange, or MS Teams users without Phone system, and so on - in which case a specific license filter may not be appropriate. |
Combination |
You can filter also use a combination of filters, that is, users from a certain geography, with an appropriate license. |
Automate includes example model instance filters for the scenarios outlined above. You can clone and use these model instance filters, and adjust as needed - including your City name(s), country name(s) or License SKU ids, for instance.
Planning for a First Sync#
If your environment has more than a few thousand users, it is recommended that you plan your sync strategy and not just do a full sync.
Use these guidelines to plan an approach based on the number of users you wish to pull in, and adjust the sync before you start pulling any users in - for example:
Start with a sync of licenses only (without users) to determine which licenses are in the environment, and the corresponding IDs, to set up in your model instance filters (if filtering by license).
Consider if your deployment will only include certain geographic regions and you can set up filters for those regions (maybe even starting with a single region initially to test and finalize setup).
Once you have your filters in place, run the sync, then adjust as needed.
Microsoft Teams#
This sync pulls in a range of elements from the MS Teams tenant to allow Automate to manage the Microsoft Teams tenant. Fetched elements include, for example, users, voice, voicemail, collaboration, resource accounts, numbers, policies, and teams.
Note
When syncing in data from msteamsonline/CsOnlineUser, Automate sets the default maximum number
of records fetched at 130 000. Paging is supported, and records are sorted on the UserPrincipalName (UPN).
You’ll need to contact VOSS support to reduce this value since a lower value may impact performance.
Users are the main consideration for setting up this sync since creating and updating users is related to the licensing of users in Microsoft Entra ID. A user won’t appear in MS Teams unless they have a relevant license assigned in Microsoft Entra ID/O365. Once a license is assigned, Microsoft automatically creates the users as a base user into MS Teams. Base users must be synced in to Automate before they can be managed, for example, to assign policies and enable services for these users.
Default Model Instance Filter for CsOnlineUsers#
Automate v21.4-PB5 ships with a new default model instance filter, CsOnlineUserAutoFilter__[tenant_name], which is
designed for Microsoft Teams syncs from msteamsonline/CsOnlineUser. This model
instance filter is set up at the same hierarchy as the tenant, and is used to sync in only
those CsOnlineUsers with a UserPrincipalName (UPN) that matches the UPN of Msol users.
The CsOnlineUserAutoFilter__[tenant_name] model instance filter specifies
the UPN values of all Msol user instances in the cache to allow only those CsOnlineUsers with a UPN
corresponding to a matching Msol user in the cache to be present in
Automate.
By default, the CsOnlineUserAutoFilter__[tenant_name] model instance filter is updated by
events or by the sync workflow, whenever a Msol user is created, updated, deleted, or purged from the cache:
When … |
Then … |
|---|---|
Msol user is created |
The UPN is added to the model instance filter. |
Msol user is updated |
The model instance filter is updated if the UPN changes. |
Msol user is deleted |
The UPN is removed from the model instance filter. |
Msol user is purged from the cache |
The UPN is removed from the model instance filter. |
Note
When running the default model instance filter the first time, CsOnlineUser cache instances are
purged or added (as applicable) so that only those CsOnlineUsers with UPNs matching an Msol user in the
cache are present in the system. This will remove unused data, streamline the cache, and improve usability.
Automate 21.4-PB5 ships with a tool that allows an admin user to reinitialize the model instance filter if it appears to be incorrect.
You can also add additional filters to the CsOnlineUserAutoFilter__[tenant_name] model instance filter. For
example, you could add “country equals US” to this filter so that the result syncs in
all CsOnlineUser records that have a UPN matching an MsolUser in the cache, AND
have a country that is “US”.
Default Data Syncs#
Automate syncs in all 0365 and Teams data via the default data syncs that are created when a tenant is
added. No Model Type List (MTL) is applied to the default syncs, which means that Automate syncs
in all supported data for the tenants, including all CSOnlineUsers, that is, both enabled and
disabled users (accountEnabled=true and accountEnabled=false).
Since user accounts remain in MS Teams even when licensing to MS Teams is removed from the user or the user is removed from Microsoft Entra ID/O365, syncing in all users in the default sync allows Automate to identify any numbers currently assigned to disabled subscribers. This ensures that the number inventory is accurate, and prevents assignment of duplicate numbers. In addition, the Microsoft Entra ID (M365) data syncs have Quick Import enabled, which improves the performance of the sync.
To limit the sync to enabled users only (accountEnabled=true), you can
apply a model instance filter to future syncs; that is, set up a sync
with a model instance filter by using accountEnabled equals true as a condition. You may also consider
menu filters and other configuration to separate disabled accounts from enabled accounts.
As with the Microsoft Entra ID sync, you may consider filtering for setting up the MS Teams sync in your environment. As the MS Teams integration currently uses PowerShell, there are additional considerations for filtering and setup to ensure the syncs are optimized to be efficient, and performance is acceptable.
PowerShell Filtering Guidelines for CsOnlineUser and MsolUser Syncs#
PowerShell supports filtering with the equals condition on some fields. It is recommended
that you plan your filters and conditions based on these fields due to the filtering that
happens on the Microsoft end. This will significantly improve the PowerShell response time and
limits the amount of data being passed between the systems for processing.
Syncs using these conditions are faster and consume less bandwidth.
The table describes the fields that PowerShell supports for filtering, for CsOnlineUser and
MsolUser:
|
|
|
|
Note
There are an additional 15 read-only extension attributes to filter on for Microsoft subscribers (when MS Exchange is installed). The values for these fields are customizable via your Active Directory server only (not in Automate), and are synced in with MS Entra ID via the MS Graph API.
The values for these attributes can be modified in Automate via the following device
models, provided the MS Entra ID Active Directory server is not synced with an On Premise Active Directory
server: device/msexchangeonline/UserMailbox and device/msexchangeonline/SharedMailbox
You can find more information about these On Premises Extension Attributes in the following topic:
If your requirements can’t be met via the fields above, you can filter
on other fields on the CsOnlineUser and MsolUser objects. This filtering is applied to the results provided
by PowerShell back to the system using the PowerShell filter. This results in the additional
non-PowerShell criteria being treated as an AND criteria with the PowerShell filter.
The filter is configured via the same model instance filter mechanism used for all other sync filtering. Automate takes the model instance filter and breaks it down to use it in the following way:
The model instance filter is checked for any criteria that uses the fields supported for PowerShell filtering. It builds the PowerShell command and issues it using those fields as a filter.
The condition is
equalsregardless of the condition set in the model instance filter for that field (PowerShell only supports theequalscondition).The result is compared against the Automate cache for users that have been added, updated, or deleted.
With this resulting list of users, any further criteria from the model instance filter is applied to the returned data.
Conditions for these fields are applied as configured in the model instance filter
This results in an
ANDcondition between the PowerShell filter criteria and the “regular” model instance filter criteria.
Although combining criteria in a model instance filter allows for less complexity and more predictable results, it is recommended that you avoid doing this, if possible. If you need to mix criteria, it is recommended that you test this carefully for expected results. For example, consider a filter with:
filter on country (can happen in PowerShell filtering) and telephoneNumber (not in PowerShell filtering):
The first list of users from the device would be filtered based on country (PowerShell Filter)
code does the compare to see the resulting set of users that are added/updated/deleted
telephoneNumber filter is then applied.
The result of the model instance filter is users where country and telephone number match
Note
It is possible to filter by UserPrincipalName (UPN) in the model instance filter.
Due to size limitations, this
is implemented slightly differently in terms of capturing the list of UserPrincipalNames
to filter.
Some common user cases and scenarios taken individually or in combination. The criteria needs to be
equals, and multiple values can be provided - either as AND or as OR, depending on the
setup in the model instance filter. These cases can all be handled via PowerShell
filtering so are the preferred use cases:
Filter users by geography - Country |
Pull in users from a subset of countries in the environment |
Filter users by geography - UsageLocation |
Similar to country, to pull in users in certain UsageLocations |
Filter users by geography - City |
Possible but less efficient as still requires pulling all the users into Automate, then filtering by city. Thus, best if used in conjunction with another PowerShell filter field. |
Filter users by department |
Pull in users assigned to certain departments |
The system includes a few sample MIFs for these use cases. The samples can be cloned and adjusted to meet specific needs.
Example model instance filters included in Automate:
|
A simple model instance filter that provides an example of how to filter on users matching United Kingdom OR United States |
|
A model instance filter that includes an example of how to filter on users matching a country and multiple cities in the country. |
Allowlist and Denylist Considerations#
Automate ships with allowlist entries for Microsoft Entra ID and MS Teams users to help optimize syncs for these applications.
Note
You can find out more about allowlist and denylist data sync settings in the Core Feature Guide.
You can modify the default allowlists and denylists for your scenario and environment, for example, to meet your flow through provisioning requirements. The predefined allowlist and denylist entries that ship with the system are typically suitable for most cases, and will account for the fact that sync workflows for a user may not be triggered even though a change for the user was pulled in during the sync. VOSS may update these default lists from time to time during upgrades if future features/functionality require it.
Delete Protection for Syncs#
While there are a number of considerations for the setup of Microsoft syncs with Automate, it is also important to consider delete protection. In this case, the relevant macros for Microsoft Entra ID and MS Teams syncs are the following:
PULL_SYNC_DELETE_THRESHOLD_MSTeamsOnlinePULL_SYNC_DELETE_THRESHOLD_MSGraph
These macros help to prevent mass deletes above a configured threshold defined in the macro (default 10). You can adjust the values for these macros initially, if required (depends on churn in your environment) or adjust them later as needed, based on the sync messages.