High Availability Operational ‘OT’ Technology Networks
Experience has identified two main schools of thought that relate to OT network implementation. The first considers the network as a device or set of devices that, when connected together, provide connectivity and therefore become a network. Where connectivity drives the purchasing decision little consideration is given to much other than initial cost. The other approach treats the automation ‘OT’ network as a piece of infrastructure that is critical to the successful operation of the business, this purchasing decision takes a longer view on factors relating to the security and manageability of the infrastructure.When connectivity is selected above infrastructure the outcome is acceptance of risk with no mitigation possible. Some environments have a appetite for risk that allows least upfront cost, to drive unmanaged devices to run a plant; this is not the case for Critical Network Infrastructure. Critical Network Infrastructure demands the network is treated as a project that encompasses elements of design, manufacture, testing and training supported by robust project documentation. IT4A specialise in network solutions for Critical Network Infrastructure.
Risk based design
Network features exist to mitigate risk. Why implement 10Gbps if 10Mbps is all that is needed? Implement 10Mbps when 10Gbps is needed and you will soon see the risk was not worth taking. The leading vendors of industrial networking products recognise the risks that relate to OT networks are different to more generic office IT networks. Features such as mSec path recovery, zero packet loss, precision time protocols are all specialist features that mitigate specific risks. There are also the generic features that consider transmission speed and interface compatibility, management features that allow good administration, and security features that keep autonomous application data apart and passwords encrypted. There are many more features that exist to allow network designers to craft a topology that meets the needs of the application at a level of risk that the Customer has an appetite for.
A key area that often gets overlooked at the product selection and network design stage is the monitoring of network behaviour during normal and abnormal operation. IT4A believes awareness is fundamental to a high availability OT system’s design. Knowing what is connected to an OT network and where; monitoring, capturing and trending events, performance criteria and security alerts and having access to the technical competence to remedy is what counts. A stable network should report zero events in normal operation. Any event or alert that is generated as a result of abnormal operation should result in some form of remediation activity. Read more about network monitoring.
The initial outcome of the design stage is:
- The basis of design (BoD) – a document that answers why the network is required in the first place (the user requirement), the threats/risks that may impact availability and, through discussion, the Customer’s appetite for risk that will influence matters of product selection and feature deployment.
- The functional design specification (FDS) – a document that develops the basis of design. It will describe the technology, the features required, the products that have been selected to realise the functionality within the specific environment and how they devices will be configured to deliver the desired outcome.
Manufacture & Testing
The FDS forms the blue print for the build configuration and subsequent testing of the network. Testing follows a formal process consistent with the Customer’s own standards. The aim is to de-risk the site installation and commissioning stages by proving the features described in the FDS have been deployed correctly through configuration and achieve the desired outcome. This testing is often witnessed by the Customer or their representative at IT4A’s Offices.
Moving from factory test to site introduces a new set of challenges setting the system to work. The lines of demarcation become blurred on site as we work to commission the network across site cabling and power. The successfully completed Site Acceptance Test (SAT) provides a milestone that the site network is operating to the approved design. Further integrated testing or live migration may follow the SAT.
Strategic decisions require the buy-in from project sponsor through to project engineering and the operation & maintenance teams. The heart of a successful project, especially when relating to Critical Infrastructure, is the preparation and distribution of high quality project documentation that supports the commercial case.
From a tactical perspective IT4A’s experience when troubleshooting failed or failing networks is the lack of relevant and current documentation to support the troubleshooting process. This incurs delays and ultimately cost, as no point of reference exists. This is normally the result of an OT network being implemented for low level connectivity rather than critical infrastructure.
Once commissioned, our engineering team probably knows as much as there is to know about the customer’s infrastructure – certainly more than the customer. A transfer of knowledge is crucial if the network is to be maintained and incorporated into OT infrastructure. IT4A believe the best approach to educate the operations and maintenance teams on site is to be clear about what they are expected to do and give them the tools and knowledge to do this. This requires some clarity on roles and responsibilities if we are to do this well.
All IT4A engineers are security cleared to BPSS (Baseline Personal Security Screening) with further Security ‘SC’ Clearance held by IT4A’s Secure Projects team (I.e. Nuclear Industry).