[Index]

Model: tool/BulkLoad

Bulk Loading a File

Full HTML Help

Overview

A bulk load template is a Microsoft Excel .xlsx format spreadsheet workbook that contains a single sheet, and is used for bulk loading data into Automate.

A tabbed workbook may contain two or more template sheets (one sheet per model). When using a tabbed workbook, bulk load transactions are carried out from left to right, starting with the far left tab, and ending at the far right tab. For example, when adding a site under a customer in a bulk load, ensure you add the customer sheet to the left of the sheet containing details of the associated site, so that the customer detail is loaded before the site.

You can use any filename for the bulk load workbook, but since the same file can be loaded multiple times, it is recommended that you use unique names to differentiate bulk uploads.

Bulk load limitations

Automate's bulk load automation templates employ advanced features, such as configuration templates (CFTs), customizable field display policies (FDPs), and GUI rules.

For some resources, generated bulk load templates won't produce the provisioning results that may be achieved when using the GUI to upload and configure data. This topic provides an overview of the bulk load limitations to consider for such scenarios.

The list below describes the general bulk load limitations:

• Certain fields link together different resources

  These fields might be hidden in the GUI, or they may be read-only. In generated bulk load templates, these field are currently exposed as mandatory fields. The fields and the specific conventions that are used in the template to link the fields together are highlighted in notes specific to the resource. For example, the value for remote destination name should be specified as RDP-<username>.

• Certain fields are derived from other system data

  Notes specific to the resource highlights where to obtain possible values for such fields. Examples are key-value type fields of a phone's vendor configuration settings.

• Restricting data to a specific set of fields on a sheet for modification purposes may not always succeed. Instead, use all the data for a bulk loader where possible.

  Exporting an instance as a bulk loader and changing the relevant columns is advisable for most complex cases.

  Due to the workflows and logic included with relations when using a modify loader, the loader may not only include the field(s) that you wish to change.

  For instance, relation/MicrosoftSubscriber does work with this approach. If you need to create a modify loader on a relation, test your use case by using only the fields you want to change and validate that the resulting configuration is correct.

  If the use case is not supported, you can potentially modify the data or device model(s) that make up the relation instead. However, be aware that you are not changing a field that has dependencies on the workflow updating related entities (like assigned numbers and number inventory).

  If you have any questions, please contact your local VOSS teams or VOSS support for assistance.

• GUI rules defined in the user interface that aren't replicated in the backend workflow must be considered in the loader to achieve the same provisioning results as the GUI.

  Examples - GUI rules may:

  • Set a default value for a visible field (fixed value or derived from other data in the system). Include this column and corresponding value in the loader for this to be provisioned.
  • Set a value for a hidden field. Include the column and corresponding value in the loader for this to be provisioned. Note that this means that fields may be included in the loader that would not be visible in the user interface.
  • Make a field visible depending on some condition such as the value of another field (for example, a checkbox being selected). Include the column in the loader, and populate it under the appropriate conditions.

The image shows that a GUI rule may, for example, disable input fields based on the state of a checkbox. On the worksheet, the selected checkbox is represented as TRUE in the column. Columns associated with the disabled fields should not be filled.

Bulk loading files

This procedure uploads two or more .xlsx worksheets in a bulk upload.

Prerequisites:

• File formats must be .xlsx.
• Ensure any referenced configuration templates are available.
• Verify details in each file, and ensure they contain all required information.
• Remove any comments from the worksheets, for example, comments showing as a marker in the cell with a pop-up.
• To send empty values, in the relevant cell of the value column:
  • type <NULL> in the cell

To bulk load files

1. In the Admin Portal, go to the Bulk Load page.

2. Set the hierarchy to the level where you want to add bulk data.

3. On the Bulk Load page, click Choose Multiple; then, browse to the files you wish to upload.

4. Choose an option:

   • To remove any files, click the X icon adjacent to the filename.
   • To remove all files, click Remove All, and if required, choose alternative files.
   • To execute the bulk load transaction immediately, select Execute Immediately, else, clear the checkbox and specify an execution date, time, and timezone.

5. Click Bulk Load.

6. Wait for the transaction to complete, then view results.

   Note

   • The Execute Bulk Load sub-transaction list will show the transaction for each row of the sheet.
   • If the transaction is scheduled for a future date, you'll need to check on it at the scheduled execution date and time. Scheduled bulk loads display in the list view of the schedule, with the name and upload date of the sheet.
   • The Bulk Load button is temporarily disabled until the sheets that have been bulk loaded and are in progress have completed. To continue bulk loading while sheet loading is in progress, clear (remove) the current sheet or choose Remove All and select a new sheet to load.

Bulk load transactions

Full HTML Help

Overview

Once you run a bulk load transaction, you can view its transaction details on the Transaction log (accessible via the toolbar).

Related topics

Transaction logging and audit in the Core Feature Guide

View a transaction in the Core Feature Guide

View bulk load transaction details

Click the toolbar icon to go to the Transaction Log. Bulk load transactions show in the log:

• In the list view, the bulk load is shown in the Action column of the log. If the bulk load was scheduled, this is shown as a schedule with the detail column indicating it to be a bulk load. The Action column will show "Execute Bulk Load" or "Execute Schedule" respectively.

• The submitted, start, and stop time for the entire bulk load transaction is also shown.

• The Detail tab displays the name of the bulk loaded file as the workbook sheet number and the number of successful rows out of the total, for example:

  [ 8/9 ] succeeded from [ 1 ] sheet in data_Users_bulkloadsheet.xlsx.
  

Checks are made to validate the user's access profile, the provided hierarchy information, and data constraints for the bulk load transaction when updating the target models. The parent bulk load transaction will show the error message if this validation fails and no rows will be loaded.

Where rows are loaded, each row in the bulk load sheet appears as a sub-transaction within the bulk load transaction. The Message dialog displays the number of successful and failed rows loaded.

Failed asynchronous transactions display below the sub transactions. If the number of failed asynchronous transactions exceeds 10, these details display on a new tab on the page.

For each loaded sheet, bulk load transactions are run in series for each row. Multiple bulk load sheets can be loaded and these transactions will load in parallel.

Sheet rows can be processed in parallel. The sheet should then not contain multiple, sequence dependent models. Refer to Bulk Load Sheet Layout.

For each row of the bulk load sheet carrying out the default add action, a Create action is shown on the list of transactions. Sheet rows that led to a successful Create action have a Success status, while rows that failed show a Fail status. If a row fails, the load process continues. For failed actions, the transaction can be selected to show the error message.

If one or more rows of the sheet failed to load, the Bulk Load Sub Transaction shows a Success status, while the Log list will show "error" for failed rows.

On the list of sub transactions, you can inspect the details of each sub transaction. For example, the submitted, start, and stop time for the bulk load sub transaction corresponding with a row on the bulk load sheet is shown. In the case of a failed sub transaction, further information about the failure - such as the error message and row data - is shown in the sub transaction.

A canceled bulk load transaction means the Processing worksheet sub transaction, as well as all sub transactions within the worksheet transaction in a Processing or Queued state, will fail.

For parallel transactions, multiple resource transactions may be in a Queued or Processing state. By default, 14 rows are processed in parallel. Refer to Bulk Load Sheet Layout for details.

If a worksheet transaction fails as a result of bulk load transaction cancellation, subsequent worksheet tabs in the bulk load workbook will not be processed by the bulk loader.

Model Details: tool/BulkLoad