Microsoft syncs

Overview

This topic provides an approach to syncs with auto filtering for Microsoft.

Using filters in syncs

It is recommended that you define a filtering strategy employing filtering criteria that can drive filtering on the Microsoft Graph API side.

Applying the filter on the Microsoft side in this way returns matching users only, thus minimizing the number of records that need to be processed, and reducing sync time and load on the system. For example, let’s say there are three hundred thousand users in your tenant and you only need fifty thousand users in the system to be managed. In this case, it is recommended that you apply API filtering to retrieve only the relevant fifty thousand users, while preventing import of the additional two hundred and fifty thousand records that you don’t need.

Supported fields

At the time of writing (for 25.2) the following fields are supported for Microsoft Graph API filtering:

CsOnlineUser

• UserPrincipalName (UPN) - also supports the in condition and is implemented to allow for as many UPNs as needed in the filter

• SAMAccountName

• City

• Country

• UsageLocation

• Department

• FeatureTypes - equals is interpreted as contains; that is, this featuretype is among the user’s featuretypes

• AccountEnabled

MsolUser

• UserPrincipalName (UPN) - currently only supports up until the maximum size of a URL and cannot be used for too many UPNs

• City

• Country

• IsLicensed - returns users with any licenses

• Licenses.Skuld - equals is interpreted as contains; that is, this license is among the user’s licenses

Building your MsolUser model instance filter (MIF) with these fields and using the required criteria ensures that you’re using the API filtering in Microsoft Graph where possible.

Note

Filtering on the Microsoft Graph API is case-insensitive, which helps to prevent errors where, for example, User Principle Names (UPNs) are not uniform.

If filtering on a field or using criteria that is unsupported on the Microsoft Graph API, then this is applied within Automate after retrieving all records from Microsoft.

Filtering methods

There are two approaches to filtering:

• Auto filter - filter CsOnlineUser based on MsolUser

• Traditional syncs - independent filters

Auto filter - filter CsOnlineUser based on MsolUser

The Auto-filter feature for Teams/CsOnlineUser sync filters CsOnlineUser based on MsolUser. Auto filter is the recommended default if filtering is used. Auto filter simplifies setup and operations and should be reviewed as a path to move to for existing customers/systems that might have implemented other filtering approaches prior to Automate 25.1.

Auto filter provides the ability to focus filtering around the Microsoft Entra User (MsolUser) resources, which means that the system only pulls users from Teams (CsOnlineUser) if the user exists in Automate (filtering based on UPNs). As a result, you won’t sync in any Teams users that haven’t first been synced from Microsoft Entra.

Sync setup and performance is streamlined, making it easier for admins to set up and manage filters on the syncs. Admins can build effective filters for various needs, based on the rich set of fields available on Microsoft Entra User. The Teams sync utilizes this data set rather than another filter, and excludes any non-essential data. Syncing with auto filter impacts only the device/mstteamsonline/CsOnlineUser model. No other Teams models are affected.

Model instance filters and related filtering capabilities are supported for all other Microsoft Teams models, but not CsOnlineUser.

To use the auto filter, you apply filters to the O365/MSOL user sync, which has more fields to filter on, and use the Auto-filter feature for Teams/CsOnlineUser sync. This will not support additional filters on the CsOnlineUser.

The table describes the advantages and disadvantages of the Auto-filter feature for Teams/CsOnlineUser sync:

Advantages

Disadvantages

Focuses filter activity on the user model that has the most fields to filter on - Easy to create a plan to “stamp” users if the base populated fields are inappropriate. For example, end customers can “stamp” users for managed services to their MSP. Provides significant control for managing the dataset size for various reasons, such as performance or security (to only see relevant data, for instance) Provides a more effective method for limiting the Teams sync and improving sync performance (not limited to filtering based on CsOnlineUser fields)

It is not possible to apply additional CsOnlineUser filters to the sync if you’re using the auto filter. This is due to how filtering happens on the PowerShell side, and the risk of unpredictable results.

Traditional syncs - independent filters

Traditional syncs (using independent filters) refer to the way that syncs have typically been set up and managed in Automate. In this method, the sync is set up with separate model instance filters for the Microsoft Entra and Teams side, and you would try to align these model instance filters where possible. This sync works when a field you want to filter on is the same on each model (for example, City or Country), but are often not the ideal fields to filter on.

Advantages	Disadvantages
Can offer more control in scenarios where there is a need to have different sets of users from Microsoft Entra compared to Teams.	Difficult for admins to set up, to ensure accuracy, and to maintain Inefficient performance Many filters need to be applied on Automate, which means all users are pulled then filtered. Often requires trade-offs (pulling in too much data due to lack of effective fields to filter on)

Filtering and user types

The following user types should be considered when defining filters, regardless of the approach you take:

• Real users

• Service accounts

The table describes filtering considerations for the user types:

User type	Considerations
Real users	Real users need to be in MS Entra to have Teams, so should not be an issue to have the user come in via Entra and identify an appropriate filter. The goal is to minimize the users being pulled in for performance/security reasons, so identifying a filter based on your data to best achieve this. This may be considered the base filter.
Service accounts	The potential challenge with this user type is how these users fit into the MsolUser filter. Resource accounts - these often won’t have the same field(s) populated as real users (for example, City), as they are auto-created and not effectively managed. Criteria therefore needs to be added to the filter to incorporate these if they don’t fit into the “real user” filter. At the highest level, these can be filtered on the Department field to be synced in. This means all resource accounts are pulled in instead of a potential relevant sub set. However, if the Entra users for these accounts are tagged correctly then the filter can be refined. If all resource accounts are pulled in, this is typically a far smaller set of data and less sensitive than real users so the lack of potential filtering is probably acceptable considering the admin effort of tagging those Entra users correctly. Meeting Room and Common Area Phone users - These instances will also have Entra users for each instance. Typically, these are more like “real users” as they need to be explicitly added, they usually have a site (location), and other business context that can be added in Entra. Thus, if they align with the “real users” filter, nothing further is required. However if they don’t align with “real users”, your base filter may need to be extended to ensure these users are encapsulated. Important The CsOnlineUser record for service accounts are pulled in from Teams to enable accurate inventory and management of the user accounts.

MsolUser filter logic in the model instance filter

Your MsolUser filter logic in the model instance filter (MIF) should consider the following:

• Criteria required for “real users” based on the fields supported on MsolUser.

  This can include fields populated naturally in the directory (such as City or Country), or by using a specific tagging approach (such as ExtensionAttribute1 = Managed)

• Criteria for service accounts (resource accounts, meeting rooms, and common area phones)

  Where these can align to the “real user” filter, this is preferred. However, if alignment is not possible, additional criteria may need to be added to the filter (for example, Department = Microsoft Communication Application Instance).

Recommended setup

This section describes the recommended, baseline setup to get started with your sync scenarios, and may be applied in various scenarios, for example, Enterprise with a tenant, or Provider with multiple end customers. You can adjust the recommendations as needed.

Setup recommendations are provided for:

• MS365 syncs

• Teams syncs

MS365 syncs

MS365 syncs focus on the resources that the system pulls via the MS Graph API, including tenant resources, such as Entra users, licenses, or groups, and limited Teams elements.

This section will give some overall guidance on the suggested baseline sync setup and considerations.

User filtering (MsolUser)

Unless you really need to see all Entra users in the system or the user base is very small, then some filtering should always be applied to the syncs for Entra users. This is to minimize the time and resources spent on syncing as well as to improve security as only relevant user data is in the system.

Common scenarios to consider in your Model Instance filters (MIF) for MsolUser (Entra Users) to pull into the Automate system:

• If you’re handling Microsoft licensing outside of the Automate system, or if you’re only applying incremental licenses, such as Calling Plan, MCOEV, or extra group licensing:

  • Apply the isLicensed = true option

    This option only syncs in users that have some sort of license assigned from Entra. Users without any licenses are excluded.

  • If you intend to use Automate to fully license users from scratch, then leave out this filter so that users without licenses are synced in to be visible for provisioning.

• Additional criteria to match users required - some examples:

  • Specific license(s) - for example, E5 or MCOEV

  • City(s)

  • Country(s)

  • Field identified to be used for filtering - ExtensionAttribute1, Custom Attribute, other schema field - populated with some specific value, and so on.

• System Accounts

  • Resource accounts - if the previous filters won’t match your resource accounts, then adding additional filters for this. For instance, Department = Microsoft Communication Application Instance

  • Meeting Room/Common Area Devices - attempt to match your additional criteria for users above, but if not, include any additional criteria to identify these (could be by specific license, for instance).

Note

• A default filter is applied on the API to only sync in users with UserType = ‘Member’ - this excludes external/guest accounts from the sync.

• Consider the length of your filter when using fields that are filtered on the API (recommended). There is no documentation from Microsoft on the maximum length of the filter so testing will be key to ensuring your filter doesn’t cause an error from Microsoft (typically indicates invalid filter in error message). If you need a long filter and it causes an error on the Graph API, you will need to break your MIF up into multiple syncs. For this reason, it’s recommended that you avoid very long MIFs that have multiple different criteria, if possible.

If using multiple criteria, ensure you match the logic needed - AND conditions compared to OR conditions. Multiple top-level criteria are treated as an OR condition where multiple inner level criteria are treated as AND.

You can also set up multiple syncs around users to have the criteria separate, if needed. In this case, carefully test the remove and purge users that don’t match the settings on the sync to ensure the behavior is as expected. The “auto-filter” capability will work even if you have multiple different user syncs set up around users.

Note

Refer to the documentation on model instance filters for more information.

Recommended sync setup for MS365

The system automatically creates a Full and User sync variant. It is recommended that you ignore these syncs and instead set up the specific syncs needed for your setup. Auto-created syncs will reappear if you delete them and can potentially be updated during upgrades.

The table describes the recommended syncs to build as a baseline, and for scheduling purposes:

Sync	Description
M365 Sync Excluding users	Use this sync for all elements except users as that will be handled via User sync. This can be run on a longer schedule (for example, once a week or month), or even adhoc when needed (for example, a new license is added to the system).
M365 Sync User Add/Remove	This will be the main sync setup that runs regularly to bring in new users or to remove users offboarded outside the system. As this focuses on just adding and removing users it keeps the sync load and runtime minimal.

Note

A sync that updates users is not been included since, typically this does not need to be scheduled. When opening users in the system, the latest settings are pulled in from Microsoft so it can be viewed that way. If you find a need to run update syncs regularly (for example, you want to utilize flow through provisioning to move users between sites based on updates, or many changes are being made outside the system), this can be set up and scheduled as needed. However, note that update scenarios are the most time-consuming in the sync as each individual record needs to be compared for changes.

Additional syncs may be set up for adhoc administration use. In this case, ensure they have similar settings and the appropriate Model Instance Filter (MIF) if the sync involves users.

M365 Sync (excluding Users or excluding Users and Groups/Teams)

This sync handles elements such as licenses, Teams, and Groups. The main purpose would be to bring in new instances or to remove old instances of these elements. Updates can be handled by opening the entity in the Admin Portal if or when the entity is managed. If there are additional elements that you don’t want to sync in, you can build an appropriate Model Type List for those elements. However, the result should at least be that MsolUser is not part of this sync.

The table describes the settings to use:

Note

You can clone the auto-created M365 sync as a starting point.

Setting	Description
Model Type List	This should be the list of elements you want to exclude, and should be at least Excluding Users OR alternative Model Type List
Refresh Existing	Set to true (checked)
Model instance filter	Unlikely to be required unless for some specialized need.
Disabled Actions	Update true (checked)
Purge Unmatched Records	true (checked)

M365 sync user Add/Remove:

This sync pulls in new users and removes users that no longer match the filter (or were removed from Entra AD).

The table describes the settings to use:

Note

You can clone the auto-created M365 user sync as a starting point.

Setting	Description
Model Type List	M365 User Data (with just msgraph/MsolUser in it)
Refresh Existing	Set to true
Model instance filter	The user MIF you set up with your user filter criteria
Include the standard user workflows
Disabled Actions	Update true (checked). Note that in this case, you will need to uncheck Remove.
Purge Unmatched Records	Set to true (checked)

Teams

These syncs focus on the resources that the system pulls via PowerShell from MS Teams, that is, most Teams and voice elements. This section provides overall guidance on the recommended baseline sync setup and considerations.

User filtering

For user filtering, it is recommended that you use the auto-filter that will automatically only pull Teams users that the system has a corresponding MsolUser record for. This enables you to use filtering capabilities available on the MS 365 sync (see above) to also filter the Teams users. In this case, there is no additional filtering required.

If you’re not using the auto-filter, then, as for the MS365 sync, you will need to define an appropriate model instance filter to match the Teams users to sync in. This should only be done if you require different filter criteria for Teams than you have for Entra users.

Note

The system automatically applies an additional filter over the auto-filter or Model Instance Filter (MIF) that is configured:

• AccountType, for example, User or ResourceAccount - this ensures only these account types are synced in, which filters out elements such as external/guest accounts

And;

• softdeletiontimestamp=null - this eliminates the accounts that have been removed/deleted but still exist in Teams as ‘soft deleted’.

Recommended sync setup for Teams

The system automatically creates a Full and User sync variant. It is recommended that you ignore these sync variants and load/set up the specific syncs required for your setup. The auto-created syncs reappear if you delete them and can potentially be updated during upgrades. For this reason, they should be ignored.

The table describes the recommended syncs to build as a baseline, and for scheduling purposes:

Sync	Description
Teams Sync Excluding users	This will be the sync to use for all elements except users as that will be handled via User sync. This can be run on a longer schedule (for example, once a week or month) or even adhoc when needed.
Teams Sync User Add/Remove	This is the main sync setup that runs regularly to bring in new users or to remove users offboarded outside of the system. As this focuses on just adding and removing users it keeps the sync load and runtime minimal.

Note

• A sync that updates users has not been included as typically this doesn’t need to be scheduled. When opening users in the system, the latest settings are synced in from Microsoft so it can be viewed that way. If there is a need to run update syncs regularly (for example, when many changes are being made outside the system), this can be set up and scheduled as needed. However, update scenarios are most time-consuming in a sync as each individual record needs to be compared for changes.

• You can set up additional syncs for adhoc administration use, if required. In this case, ensure they have similar settings and the appropriate Model Instance Filter (MIF) if the sync involves users.

• For planning, (if not using auto filter), if you’re using fields that can be filtered on the PowerShell side (for example, City), there is a limit to the length of filter that can be applied. This is an undocumented PowerShell limitation, and failure messages may be unclear (for example, they may refer to relevant messages about the filter to a message containing “Expected literal (number, boolean, or null)”. Internal system testing indicate 97 conditions for a single field is the limit. For instance, 97 cities passed, whereas 98 cities caused a failure; 97 cities and 97 countries passed and 98 of either failed. For this reason, it is recommended that you test and validate scenarios for your tenant if you’re going to require long filters.

Teams sync excluding users

This sync handles all Teams elements, except users. The main purpose of this sync is to sync in new instances or to remove old instances of these elements (updates can be handled by opening the entity in the Admin Portal if/when they manage it). If there are additional elements that you don’t want to sync in, you can build an appropriate Model Type List for this purpose. However the result should at least be that CsOnlineUser is excluded from this sync.

The table describes the settings to use:

Note

You can clone the auto-created Teams sync as a starting point.

Setting	Description
Model Type List	To define the list of elements that should be excluded; should be at least Excluding Users OR alternative Model Type List
Refresh Existing	Set to true (checked)
Model instance filter	Unlikely to be required, unless for some specialized need.
Disabled Actions	Update true (checked)
Purge Unmatched Records	Set to true (checked)

Teams sync user add/remove

This syncs in new users and removes users that no longer match the filter (or were removed from Teams).

The table describes the settings to use:

Note

You can clone the auto-created Teams user sync as a starting point.

Setting	Description
Model Type List	Teams user data (contains just msteamsonline/CsOnlineUser)
Refresh Existing	Set to true
Model instance filter	If using auto-filter, leave blank. If not using auto-filter, select your MIF for user filtering.
Include the standard user workflows
Disabled Actions	Update true (checked). Note that you will need to uncheck Remove if using a MIF.
Purge Unmatched Records	Set to true (checked)

Scheduling syncs

Consider the following with regards to scheduling syncs:

• MS365 user sync

  • Run and complete the MS365 user sync before running the Teams user sync. This ensures that any new or removed users in Entra are in the system and can drive the auto-filter behavior during the Teams user sync.

  • Run the MS365 user sync more frequently than the Teams sync to minimize delays for user onboarding.

    The MS365 sync brings users into Automate and can then be run through the onboarding process (whether admin initiated or via flow through provisioning).

    Typically, the Teams user is not required in the system to make onboarding possible. The onboarding workflow target syncs the Teams user in during the workflow.

• Teams user sync

  • May not need to be run as frequently as the MS365 sync.

  • If you’re performing many user transactions outside the system (such as assigning numbers), the sync may need to be more frequent, for example, to ensure the accuracy of the number inventory.

Typically, updates won’t need to be scheduled as the system updates data when records are opened. However, update syncs can be set up and scheduled as needed.