The cloud allows resource and labor costs to be shared across multiple sites and with many customers. This article examines device-centric Internet of Things (IoT) in the industrial space, defined as connecting individual measurement and control devices to the cloud to support different business objectives. This is in contrast to other Industrial IOT (IIoT) solutions that may require a more significant system footprint at the site to support local control and safety requirements, integrated to higher level business functions in the cloud.
Device-centric IIoT architectures are useful for different scenarios:
- Small independent systems in which the customer may have only a few devices
- Distributed systems such as gas distribution networks in which hundreds or thousands of measurement devices are distributed across a wide geographic area
- Embedded systems, where measurement and control devices are embedded in third-party original equipment manufacturer’s (OEM) offerings, such as skid-mounted equipment
- Secondary monitoring for devices that are part of a larger local system but also connect to the cloud for management and health monitoring
Process measurement and control devices in this category tend to be low cost ($1,000 to $2,000), often sold through distributors and systems integrators, and used by relatively small and lean organizations. These characteristics put constraints on an IoT solution, requiring it to be inexpensive to buy, simple to implement and maintain, and requiring little or no Honeywell involvement to sell.
This article discusses unique challenges in this segment including:
- Support for a variety of connection models depending on device capabilities and power
- Low sell prices and indirect sales channels dictate that IoT capabilities in this segment must be self-serve
- Customers lacking the necessary IT resources to deploy and maintain complex solutions
- Support for hundreds of thousands of devices across thousands of customers, requiring a multitenant architecture that supports easy scalability while ensuring data privacy and integrity
- Much of the value proposition for IoT in this segment of being able to share device data and/or management with third parties, suppliers and distributors, OEMs, system integrators, and the customers’ customers, which requires a strong multi-party approach to security
Proof of concept
The background is a recently completed proof of concept (POC) for the Process Measurement and Control business of Honeywell Process Solutions. The portfolio includes a range of measurement and control devices and small systems. The POC explored new use cases and services that can be enabled by connecting these devices and small systems to the cloud and some of the challenges involved.
The POC focused on two representative product lines, paperless recorders and electronic volume correctors (EVC) for gas metering. Paperless recorders are small-scale process historians that capture historical process data from other local measurement and control devices for short-term use and act as a feed for long-term data archives. Recorders are typically sold as stand-alone products but may be bundled with other devices as part of an OEM offering or system integrator solution. A given site may have one or up to 100 or more of these devices. A companion PC package supports data analysis and long-term archiving.
EVCs provide pressure and temperature correction for gas meters and support sending corrected gas consumption data to centralized data centers for billing purposes via dial-up connections. They are typically installed by gas utility companies at industrial and institutional sites such as manufacturing plants, hospitals and campuses. A utility company may install and manage hundreds or thousands of these devices.
The POC demonstrated how connecting an individual device, such as a paperless recorder, to the cloud can add value in many ways:
- Access to process data anytime, anywhere including notifications about abnormal conditions
- Ability to manage and configure the device remotely
- Reduction in IT costs for related software solutions
It also demonstrated how IoT approaches can help customers manage large fleets of distributed devices including:
- Remote monitoring of device status and facilitating service rounds
- Enabling the outsourcing of device service
- Supporting data-as-a-service scenarios in which the device is owned and managed by a third party
Not covered in the POC is the potential to support analytics on device data for demand forecasting, predictive maintenance, optimization and other operations, but this would be a natural extension. Other key aspects of device-centric IoT were explored during the POC.
A key requirement for device-centric IoT is, naturally, the ability to securely connect devices to the cloud. Different connectivity topologies are described in this section, but the technical details of communication protocols and transport security are not included.
Devices are candidates for direct connection to the cloud if they have the following characteristics:
- The device is always connected to the internet, allowing full bidirectional interaction. This can be a wired or wireless connection. The main implication is that the device has sufficient power to maintain a connection.
- The device can tolerate communication outages by buffering data and status.
- The device has sufficient processing power to host the required IoT communication stack.
- The device or the process element(s) it represents has sufficient business value as a stand-alone entity. Otherwise, marshalling multiple devices together through a gateway might be more economical.
- The paperless recorder is an example of a device that meets these criteria.
- Indirect connection via gateway
- In some cases, connecting one or more devices to the cloud indirectly via an “edge gateway” may be necessary or desirable. This may be the case for a number of architectural reasons:
- Devices may have low power availability, limited processing ability and/or no buffering capability.
- Device product cost can be reduced by delegating optional IoT functionality to a separate gateway.
- Communication channels and related management for multiple devices can be optimized with a shared gateway, which also presents a focused target to secure.
- Existing installed devices may support local communication protocols but not IoT connectivity.
Edge gateways may support connecting to devices via a number of local communication protocols — such as ModBus, FieldBus, OPC or 4-20 mA — and to the cloud via appropriate IoT protocols. The gateway acts as a bidirectional multiplexer and buffer between multiple devices and the cloud. Gateways may be purpose-built devices or more general purpose devices, such as remote terminal units.
Periodic connection via dial-up
Remote and/or geographically distributed devices are often connected to networks and the cloud via cellular or landline modems. While this provides a mechanism with which the devices can send their data to a remote host, these devices are often too low-powered to maintain this communication and may buffer data but only connect to the network to upload it once per day. While this may be adequate for reporting purposes, it makes remote monitoring and management of these devices challenging because any device commands would only be pulled down at the same frequency. Long cycle monitoring such as watching battery life can be handled, but situations requiring interactive communication require a different approach.
One solution is to use a mobile device (phone or tablet) as a proxy between the device and the cloud, using a low-energy Bluetooth connection to talk to the device and the phone’s cellular connection to talk to the cloud. This does require physical proximity, which can be expensive — sending a technician into the field should be a last resort. One of the value propositions for IoT is to use monitoring and analytics to identify and predict maintenance requirements to help optimize field support.
Once a device is physically connected to the internet, it needs to be provisioned as an IoT device. Provisioning is the act of establishing a device as a participant in an IoT solution from both technical/communication and business/entitlement perspectives. Customers grant the device permission to connect to the cloud, and the IoT solution grants customers access based on their entitlements. Low cost and simplicity requirements dictate that the provisioning process be as automatic as possible.
First, the device or gateway needs to be able to connect to the cloud, which requires that it know which endpoint to use and to have appropriate keys and/or certificates with which it can be validated. One solution is to ship the device from the factory with the endpoint and keys preconfigured. While this could allow the device to connect automatically to the cloud, in practice the customer will want control over this and explicitly enable it if desired.
One challenge with a pre-provisioned device is that it assumes a fixed endpoint for the cloud where in reality the correct endpoint may depend on the geographic region or which services the customer has purchased. For example IoT Hub services in Microsoft Azure have distinct endpoints for different regions. A solution is to have a dedicated global provisioning service that can receive initial communication from the device then redirect it to the correct endpoint based on the customer’s order. This process can also replace temporary factory-issued keys/certificates, including with customer-provided keys if desired.
The other side of provisioning is to associate the device with a customer. The device is inherently unaware of its ownership — many of these devices are sold through distributors and cannot be pre-provisioned for a specific customer order. Having the device assert its ownership based on user configuration introduces the potential for impersonation, so the safest approach is to rely on a pre-provisioned device ID (key and/or serial number). The provisioning service can match the device ID with a customer order for the same ID via integration with a backend order management systems. Once customers are identified, their entitlements can be established via integration with the backend license management systems.
An alternative to pre-provisioned devices is for the provisioning process to be managed by a mobile app. A user makes a local connection to the device, for example via Bluetooth, and a network connection to the cloud. After authentication the mobile app can act as a broker to establish a secure connection between the device and the appropriate cloud endpoint. This approach aligns with the general trend toward using mobile devices for local configuration and management of devices.
The provisioning process described above is self-serve, meaning the user can complete the process with a minimum of effort and no Honeywell involvement. This lends itself to trial or “freemium” licensing where purchasing the hardware device automatically entitles the user to free, if limited, access to cloud services. A good analogy is Google or iCloud Drive where users of the Google or Apple ecosystems get a certain amount of cloud storage for free, with options to pay for more. In the Honeywell context, providing remote access to current (and limited historical) device data demonstrates the value of cloud access and can create pull for paid services such as long-term history, notifications, remote configuration and device management, and data analytics.
Integration with business systems
Device provisioning touched on two integration points between the cloud platform and the business systems. Integration with order management is used to correlate devices and customers and license management dictates what cloud services the customer is licensed to use.
Initially customers may be given a trial license, but they can upgrade to a standard license and purchase options such as notification and service apps. Staying with the self-service approach, they can purchase and renew licenses via an e-commerce solution. Depending on the licensing model, the customer may be billed on an annual or usage basis.
Once licensed, they have online access to documentation and support materials via a content management system. The customer may be automatically enrolled in the customer management system and can optionally sign up for aftermarket support services and access to a customer support portal.
All of these backend systems need to be integrated with the IoT cloud platform using standard IT approaches. With this integration, they are available to support multiple line of business solutions in the cloud.
Once devices are provisioned, they support bidirectional communication with the cloud. Communication may use different transport protocols and message formats such as AMQP, MQTT and OPC UA. Communication from device to cloud is often called telemetry and from cloud to device command and control. In preferred implementations, all communication is initiated from the device including command and control, where the cloud places commands into a queue to be pulled down by the device, so that the device never accepts unsolicited communication. Azure IoT Hub is an example of a solution that implements this pattern.
The device sends messages to the cloud to communicate its status, latest values, alarms and events (if monitored by the device), and configuration changes. The device also needs the ability to upload historical data in a batch, for example to recover after a communications outage. As messages arrive at the cloud, they are processed according to event rules, which may trigger downstream processing, such as alarm and event notifications, real-time analytics, data storage, and message archiving. Once at rest, the data can support visualization, reporting, offline analytics and machine learning.
The cloud can send messages to individual devices to update configuration, initiate backup and restore, update firmware, and restart the device. Ideally multicast communication is supported to allow operations, such as time synchronization between the cloud and multiple devices.
A major component of the IoT solution is storage of device data in the cloud to maximize visualization and reporting, to support analytics, and for audit and archival purposes. Each of these benefits has different requirements for querying, data granularity and retention time. While a general mantra of cloud computing and big data is “store everything,” costs and functionality considerations must be addressed.
Cloud platforms support a many storage options — including blob storage, table storage, Document DB (NoSQL),, Azure SQL and HD Insight (Hadoop) — as well as in-memory caching and support for third party and open source storage solutions. Each technology has pros and cons and different associated costs. Document DB, for example, can be several orders of magnitude more expensive than blob storage. While per unit storage costs are generally cheap in the cloud, the data volumes can be extremely large. Analysis of the POC found that the initial storage approach chosen compared unfavorably with traditional database architectures from a cost perspective. Alternate approaches were identified but not implemented.
Experience with the POC suggested that a practical approach was to use a mix of storage technologies, each for its own purpose. For example, blob storage is well suited for archiving telemetry messages verbatim but poor for querying. Azure SQL manages meta-data well. This requires a high degree of referential integrity but is expensive for large data volumes. Hadoop is ideal for huge volumes of arbitrary data and supporting offline analytics but less well suited for simple reporting.
This approach may result in the same data being stored in multiple places, which is a departure from traditional data management practices. It also suggests taking advantage of the extensive processing power of the cloud to process and pre-aggregate data when appropriate. For example, rather than storing years of data at one second intervals, data could be stored at that frequency for a relatively short period then aggregated into minute or longer averages for longer term storage.
A key distinction between system-centric and device-centric IoT is the scope of a given customer. System-centric solutions have relatively fewer customers with larger systems. Device-centric solutions have many more customers but smaller scopes ranging down to individual devices. While hosting some system-centric solutions independently for major customers (often privately within the company’s infrastructure) is practical, doing this for many device-centric solutions is not economically feasible.
The solution is to implement a multitenant architecture in the cloud, allowing many customers to share the same cloud resources, so system and support costs are spread across them. Data privacy, however, is a key concern with this approach, no less so for small customers than for larger ones.
Each customer is considered to be a tenant, and system resources and functions are segregated by each tenant as needed. Static and persisted data must be strictly partitioned by the tenant. Different implementations are available depending on the data storage technology used. Some storage approaches, such as Azure SQL, natively support tenant partitioning. Others require this to be designed into the solution. Likewise, data security may be enforced by the database itself but more often by data access services.
Data partitioning is also important for scalability and availability. Partitions can be assigned to different resources in a data center or even across different data centers as needed. Larger customers may desire separate, dedicated storage,which is a potential premium offering.
Unlike stored data, data in motion from multiple customers may be comingled in streams and event queues as it is processed. The system is free to assign messages to processing resources without regard to ownership in order to manage load. An exception might be cases in which messages are secured with customer-specific keys, where dedicating processing resources to that customer may be more efficient.
Customers need to be able to manage their own users and permissions. For many customers, users can be managed directly in the IoT solution, for example using Azure Active Directory. By default multiple customers are managed in a single directory, with an overall security administrator delegating limited administration rights for each customer. Optionally a company may have its own dedicated directory that can be used, for example when the company already uses Azure Active Directory for other solutions. A more advanced customer might choose to federate security in IoT with an enterprise active directory so that users and groups can be managed within the enterprise but used in the cloud.
Some of the strong value propositions for cloud solutions require the customer to share data with third parties. This might include:
- Sharing quality or compliance data with customers and regulators
- Sharing process and performance data with process supplier and OEMs for analytics and optimization
- Sharing system health and diagnostics with equipment suppliers and outsourced support service providers
This requires a multiparty approach to data security in which a customer can grant selective permissions to third parties as needed. This requires a model that allows permissions to be assigned at a more specified level than just each customer. A gas utility company, for example, may have gas meters and correctors at thousands of end user sites — hospitals, colleges, light industry — and may want to share data with each user about the specific site(s). The security model needs to support assigning permissions to groups of users down to the device or even device property level. Ideally, permissions could be assigned on a role basis. For example a device manager can be given permission to view and modify device settings, and a process user can be given read-only access to measurement data. The POC determined that the Azure implementation of roles was not sufficiently flexible, so a permissions model was developed in the POC solution and mapped to Azure Active Directory groups for user assignment.
Mobile devices play an important role in device-centric IoT. Independent measurement and control devices have traditionally been configured locally using built-in user interface (UI) panels or custom handheld configuration tools or by plugging them into a laptop computer. As in other domains, a move to replacing these single-purpose tools with mobile apps is growing, allowing the user to interact with the device from (optionally hardened) smartphones or tablets via Bluetooth.
The POC explored two scenarios in which mobile access added value to devices. The first was to provide users with 24/7 access to device information and notifications. Using a smartphone, the user can browse the devices and easily pull up current process status and historical trends. In addition, some applications allow the user to subscribe to notifications about abnormal process or device conditions and provides additional context in the form of data trends and related events to help the user quickly assess the situation and determine what follow-up action is required. Some apps also support collaboration with other users through comments and event ownership.
Mobile applications also facilitate servicing distributed devices. A service technician, possibly from a third party service company, can receive notifications about device issues and immediately call up device details to investigate. Configuration changes and software updates can be performed remotely without needing to visit the device. If a visit is required, the technician can use geographic coordinates to map a route for servicing multiple devices in the same area. Once there, the technician has easy access to technical documentation, how to videos and even instant messaging with support via the app.
In addition to supporting local, stand-alone device access and configuration, the mobile device can also play a role in provisioning the device into an IoT solution as described previously in this article. Together, the local app and the cloud can eliminate the need to have a UI on the device, enabling a new generation of lower cost, “headless” measurement and control devices.
One of the value propositions for IoT is to make use of data to make more informed decisions through the application of analytics. Device-centric IoT provides a mechanism for pulling together data sets from large numbers of geographically dispersed sources.
The POC did not explore this area but identified a potential future phase applying Azure HD Insight and other tools to problems such as a gas utility company forecasting gas demand based on consumption data from multiple end-user meters along with other sources such as weather information, or an OEM running fleet analytics across multiple customer installations.
One of the learnings from the POC was that cost is an important factor in choosing the right architecture and should be considered from the beginning of development. While this is true to an extent for any software development, the need is more acute when developing a cloud offering, particularly on a third party PaaS platform, such as Azure. Processing and storage costs can vary considerably depending on which platform services are selected and how they are used. On the positive side most PaaS services are priced on a unit basis so that costs scale proportionally with usage without requiring significant upfront investment.
An often over-looked cost is the development and operations (DevOps) resources required for routine maintenance, upgrades and enhancements, and end-user support. This expense may often be equal to or greater than the cost of the PaaS platform and does not scale linearly. Ideally these costs should be spread across multiple IoT solutions and many customers to reduce the cost per customer.
The PMC POC demonstrated that device-centric IoT can support new service offerings for customers. Key requirements were identified including:
- Supporting a variety of deployment topologies
- Self-serve provisioning and licensing
- Integration with business systems
- Heterogeneous data storage options
- Multitenant support respecting data privacy
- Role and resource based security models
- Mobile first user experience
- Cost optimized architecture
An architecture that addresses these concerns can support multiple IoT solutions for different business lines.
Matthew Burd is a senior principal engineer with Honeywell Performance Materials & Technologies, based in Calgary, Canada. His career with Honeywell has spanned more than 30 years, during which time he has contributed to and led application developments in oil movements and storage, process history, manufacturing execution systems and mobile applications. Burd has won a number of Honeywell awards including a Technical Services Citation and a New Product and Application Development Award, and he holds a Bachelor of Applied Science in mechanical engineering from the University of Waterloo. He may be reached at firstname.lastname@example.org.
For more information about the Process Measurement and Control business of Honeywell Process Solutions, visit honeywell.com.
Author’s Note: I would like to thank Joe Pane, Chris Gilbert, Dave Granatelli, Josh Worrill and Rhett Newman for their contributions to the POC project that inspired this article.