.. _nbi-ch-technical-description: -------------------------- NBI Technical Description -------------------------- NBI provides functionality to extract subscriber billing related information from VOSS Automate whenever a subscriber change occurs, and to send this information (in a payload) to a northbound billing system. NBI includes a call back service, which is utilized by the northbound system to provide status updates for each billing message sent. Billing-related changes include changes made to both subscribers directly or to their associated devices and services. NBI monitors for changes by connecting to one of the VOSS Automate nodes and observes transactions within the VOSS Automate Transaction Log. Connectivity details are discussed below and are defined in the NBI configuration file. Access to NBI for administration functions is through the NBI Admin GUI (Web Portal). Major Functions and Features ============================= This section describes NBI in terms of its major functions and features: * Voss NBI Module * NBI Administration Portal (GUI) * Access Control to NBI GUI * Real-time Billing Payload * Billing Reconciliation Support (SDE Report) * Extract Function (EXTRACTOR PROCESS) * Transmit Function (NOTIFIER PROCESS) * Call Back Function (CALLBACK PROCESS) * Tracking Table * Connectivity * Provisioning Control (Site Transition flag) * Configuration * Diagnostics * Provisioning Control (User State) and managing Go-Live Dates * Managing Site and Subscriber Information .. _voss-nbi-module: VOSS NBI Module ---------------- NBI is a single server appliance that runs on the Ubuntu operating system: * Small footprint - limited disruption to existing service (this is an add-on) * Server separation - reducing load on the live VOSS Automate application NBI consists of the billing payload and three functions to Extract changes, Transmit payloads northbound and, to receive status updates back (Call Back). A single NBI (only) is connected to a VOSS Automate cluster. .. figure:: /src/images/nbi-image002.png NBI Administration Portal (GUI) -------------------------------- The NBI Web-based Administration Portal (GUI) provides the ability to: * View details on all sent messages - information is extracted from the Tracking Table (including message status) * Control default Subscriber States (new Subscriber detected) * Set Subscriber State (Pending, Live), and manage Go-Live dates * Manage Subscribers individually, or through Bulk Load * Manage overall Site Transition status (on/off) Additionally, flexible filters provide the ability to easily search and locate in the different views. It is also possible to export (XLS, JSON, CSV) and import data. Access Control to the NBI GUI ------------------------------ Secure credentials are required to access the NBI Admin GUI. The same administrator login details are used for the main VOSS Automate application. Real-time Billing Payload --------------------------- Whenever a subscriber is changed, a billing information payload is generated in near real-time and is sent northbound. This billing information payload, which may be customized according to the requirements of the receiving system, contains information about: * The Subscriber * The action taken (Add/Change/Delete) * The set of associated devices and services that have been provisioned or changed Billing Reconciliation (SDE Report) ------------------------------------ NBI may be configured to generate a *billing friendly* report export, which may be used by the northbound system to run a lower frequency reconciliation process. This JSON format report has a configurable schedule, and can be made available to all customers or to a selection of customers. The content is configurable based on the requirements of the northbound system, and the output is deposited to a local NBI media directory. Extract Function (Extractor Process) -------------------------------------- NBI detects changes to the live service by polling for new entries in the *VOSS Automate Transaction Log*. Once a change is detected (known as a *Trigger Event*), NBI uses the standard VOSS Automate APIs to extract subscriber and service information and builds a standard, JSON format billing payload. Payloads are stored in the *Tracking Table*, and are marked as *READY* for sending by the Transmit Function. .. note:: The full list of Trigger Events that cause NBI to create a payload are documented in the appendix. To maintain the chronology of events, NBI processes top level transactions sequentially from the VOSS Automate database. The system includes mechanisms to prevent processing being blocked by large bulk load and data sync jobs that may be running on the transaction log. If NBI is taken offline or loses the connection with VOSS Automate, it retains a record of the last transaction processed and *catches up* once it is brought back online. NBI typically connects to Secondary Database Nodes so that undue load is not placed on the main Primary Database Node in the VOSS Automate cluster (although this is configurable). NBI automatically connects to the Primary Database Node for write operations and replication-lag-sensitive read operations. In a typical installation, to provide redundancy and load distribution, a list of Secondary Database Nodes is configured in addition to the Primary Database Node. The VOSS Automate *Transaction Log* is polled to detect changes to the live system. To reduce load on the node, the polling period defaults to 5 minutes. A regular, scheduled job ('nbi_sync'), keeps the NBI synchronized with VOSS Automate (specifically, for Customer, Site and Subscriber information). This job may run overnight, and provides a key mechanism to get new Subscribers loaded into NBI. Transmit Function (Notifier Process) ------------------------------------- The Transmit Function regularly searches for payload messages in the Ready state, and then sends these northbound (via HTTP, or secure HTTPS), over a RESTful API service to a well-known destination URL. Payload transmission may be buffered or re-sent in case of underlying network issues or if the northbound system is busy. Once the northbound system receives the messages, they are marked as *UserInProgress*. If network failures persists, messages are marked as *SendFailed*. .. note:: The Transit Function automatically retries sending a payload message several times, in case of intermittent issues on the underlying network. Call Back Function (Callback Process) --------------------------------------- NBI supports a Call Back function and tracking mechanism, which the northbound system can utilize to communicate various status updates or error reports back to NBI, against each billing payload message sent. The Call Back Function marks messages with these statuses: * *UserProcessed* - completed successfully * *UserFailed* - in this case, the northbound system indicates an associated Error Status Tracking Table ---------------- NBI tracks all billing events and messages in its *Tracking Table*, which contains records representing billable events and their associated payloads (including status information and error details, if any). .. note:: The contents of the Tracking Table and associated fields are documented in the appendix. The *Tracking Table* is held on the VOSS Automate node(s), and so benefits from the redundancy (multiple nodes) and backup mechanisms found on these nodes. The NBI appliances itself holds no data billing payloads and their history. Connectivity ------------- NBI connects to VOSS Automate using a combination of the VOSS Automate API and direct access to the mongo database (where all NBI data is kept – there is none on the NBI appliance itself). * Access to the VOSS Automate API is made using a service account (configured on VOSS Automate and at the Provider Level in the hierarchy), and through the VOSS Automate Web Proxy Node. * Access to the mongodb is made through direct access to the VOSS Automate Unified Nodes, with Secondary Nodes being preferred (with the ability to move to another node if the selected node is not available). NBI connects to the northbound system (for sending payloads) over HTTP or HTTPS. All connectivity details are defined in the NBI configuration file. Provisioning Control (Site Transition Flag) -------------------------------------------- Sites may be set into a *Transition* state from the NBI GUI, whereby sending of billing payloads for that particular site is fully disabled, regardless of whether the Subscriber is in a *Live* state. This feature is provided as a fail-safe, at a site level, and may be deployed when transitioning an existing customer onto the VOSS Automate service or during the initial provisioning of new customers. Configuration --------------- A configuration file on the NBI appliance manages the configuration of NBI operational parameters. This file is initially configured on NBI installation, and may be updated from the command line (see later commands). Diagnostics ------------- * Message reporting screens to track billing payloads and status * Integration and alarming with NOC systems (SNMP) * Log files are maintained and can be accessed from the command line interface (for each of the main functions, and are date/time stamped) * Commands to check on status and general health of the server appliance Provisioning Control (User State) and managing Go-Live Dates -------------------------------------------------------------- Subscribers (as seen by NBI) are assigned one of the following states: * Test * Pending * Live In a *Test* or *Pending* state, any changes made against the subscriber do not initiate a billing payload, and no entry is made into the Tracking Table. Billing payloads are only sent for subscribers that are in *Live* stage. .. figure:: /src/images/nbi-image-004.png The use cases are as follows: a) During initial provisioning for a customer, when subscribers being on-boarded, the billing flow may be disabled. Subscribers are automatically discovered as they are built into VOSS Automate, and are placed in a *Pending* state (no payloads sent) b) Once a *Go-Live* billing date is agreed with the customer, sites and associated users may be promoted into a *Live* state. All subsequent changes will then automatically flow into billing (payloads sent). This promotion may be made manually or pre-set through configuring a go-live date (schedule) against each subscriber. The default subscriber state (the initial subscriber state) is configured at a site level and may be set to any of Test, Pending or Live: ========= ===================================================== Test Subscribers with a Test State are for NBI testing purposes only; payloads are never sent northbound. Once a Subscriber is placed in a Test State, this state can never be changed. Pending A Subscriber in a Pending State is staged during the initial onboarding of a customer, and can then be promoted into a Live State (either manually or on a Go-Live schedule). Live Subscribers in a Live State are processed and billing payloads are sent automatically to the northbound system. ========= ===================================================== Managing Site and Subscriber Information ----------------------------------------- * Use the **Sites** menu to manage sites (associated default subscriber state, transition flag). * Use the **Subscribers** menu to manage subscribers (associated activation state, go-live date, send new payload). Subscribers can be freely moved between *Pending* and *Live* states (and back again). Run Time Configuration Parameters ================================== The parameters described in this section can be configured on the NBI and are contained within the NBI configuration file. .. note:: Most of these parameters may be updated from the utility found on the command line interface. Where this is not possible, contact VOSS Support for assistance (these parameters require command line root access). ================================================================= ============================================================================================== **Parameter** **Configuration File Extract (bdeconf.json)** **Poll Interval** :: "pollInterval": 5000 "sets poll interval in milliseconds to check for new transactions in the VOSS Automate Transaction Log. Default is 5 minutes **Batch Size** :: "batchSize": 100 "number of records to process in a single batch when doing bulk processing **Location of VOSS Automate application and access credentials** :: "vossApiRootURL": "http://:/api", "Administration credentials for NBI to login into the VOSS Automate application. This should be created as a service account with no password expiry and at Provider Level (e.g. nbiadmin)" :: "vossDbDSN": "mongodb://@:27020, :27020, :27020, :27020/VOSS? authSource=admin&ssl=true&replicaSet=DEVICEAPI&readPreference=secondaryPreferred" "Access details for the VOSS Automate database – note each unified node is specified for redundancy with the secondary being preferred" ================================================================= ============================================================================================== ===================================================================== ============================================================== **Parameter** **Configuration File Extract (bdeconf.json)** **Call Back function** :: "callback": { "address": "0.0.0.0", "port": 5009, "username": "voss", "password": "callbackpass", "url": http://:5009/callback } **Sandbox** :: "sandbox": { "address": "127.0.0.1", "port": 5006 } **Location of the northbound service (URL) and access credentials** :: "notifier": { "url": "https://:", "useCerts": true, "indicates if certs are deployed (or not)" "key": "/opt/billing-data-extract/certs/.key", "ca": "/opt/billing-data-extract/certs/ca.crt", "cert": "/opt/billing-data-extract/certs/.crt" } ===================================================================== ============================================================== Hardware and Network Specification (Single Node Server Appliance) ================================================================== ============= =================================== **Resource** **Specification** CPU 2 vCPU @ 2GHz with no reservation Memory 4GB Disk 80GB Network 1 Gbit/s minimum ============= =================================== ========================== ========================== ======================= **Source (Function)** **Destination** **Destination Port** NBI (Extract Function) VOSS Automate HTTPS TCP 443/8433 NBI (Extract Function) VOSS Automate TCP 27020 NBI (Extract Function) VOSS Automate SSH TCP 22 NBI (Transmit Function) Northbound CRM HTTPS TCP 443/8443 Northbound CRM NBI (Call Back Function) TCP 5009 Command Line Management NBI SSH/SFTP TCP 22 NBI (Services) DNS TCP/UDP 53 NBI (Services) NTP UDP 123 NBI (Services) SNMP Server / NOC SNMP TCP/UDP 161, 162 SNMP Server / NOC NBI (Services) SNMP TCP/UDP 161, 162 ========================== ========================== ======================= ======================================== ========================== **Network Bandwidth Requirement** ======================================== ========================== Flow - NBI to / from VOSS Automate 350 KB/s bandwidth Flow - NBI to / from northbound system 150 KB/s bandwidth ======================================== ========================== Command Interface =================== Maintenance is carried out from a platform user login application command line, either via SSH or from the VM console command line. The password is configured during installation and can be changed using *system password*. On initial login, the system displays a banner indicating the general system health. .. figure:: /src/images/nbi-image-006.png **Appliance Commands** The table describes the appliance commands: ================================ =================================== network - network management keys - ssh/sftp credentials notify - notifications control log - manage system logs diag - system diagnostic tools schedule - scheduling commands snmp - snmp configuration system - system administration drives - manage disk drives security - security update tools app - manage applications user - manage users ================================ =================================== Logs are provided for the three main NBI functions: * Extractor * Notifier * CallBack processes **NBI-specific Commands (billing-data-extract)** The table describes the NBI-specific commands (billing-data-extract): ================== ================================================================================================================== config View and Edit billing-data-extract configuration file run_sde Method for running SDE to produce reconciliation files. This command can be put on schedule to execute nightly. syncnbi Ensure that NBI is in sync with VOSS Automate database syncnbi-dryrun Dry run to report on sync status between NBI and VOSS Automate database test_connection Test remote VOSS Automate database and VOSS Automate API connection message-tracking Manage and prune message tracking database ================== ================================================================================================================== Security Considerations (Certificates) ======================================= The upstream connection and passage of billing payloads may be secured, where required, to use HTTPS over the northbound connection. In this case, a suitably signed certificate should be produced and then installed onto the NBI – into a folder location and then this recorded into the NBI configuration file. .. note:: These certificates need to be copied to, set up, and configured correctly on the northbound system. Installation Requirements ========================== The virtual machine is deployed using the OVA template, which is supplied by VOSS. Installation of the NBI follows this sequence: 1. Configure VOSS Automate for NBI: * Open firewall ports for access to mongodb (NBI data collections) on Unified Nodes. * Configure NBI service account (limited access profile, password without expiry, at Provider Level). 2. Install NBI appliance OVA and set network configuration parameters. 3. Install billing-data-extract application on NBI appliance. 4. Install certificates (as required). 5. Configure and connect billing-data-extract application to VOSS Automate and northbound system. - Location of Unified Nodes (IPs) and Web Proxy - URL for northbound system - Service access account for VOSS Automate - mongodb access credentials - Certificates 6. Provision and restart NBI appliance for operation. .. note:: .. raw:: latex Refer to the NBI Install Guide for full instructions. .. raw:: html Refer to the NBI Install Guide for full instructions.