The role of Z/IP gateways


#1

This is probably a very simple question, and I may be overthinking things, but I’m trying to properly understand the role of Z/IP gateways. I understand their functionality, but I think I’m confused as to their role.

My undestanding is…

  • Z-Wave devices talk to a Z-Wave controller with a matching HomeID
  • ZIPGateways allow the encapsulation of the Z-Wave command class over IP connections, usually carried over DTLS
  • ZIPGateways target Z-Wave devices as an IP address but they need a Z-Wave controller to proxy the connections to translate that in to the RF based frames to be sent to the actual devices
  • ZIPGateways may connect to one or more Controllers

Is this all correct so far?

I’m trying to understand the place of zipgateways in the home environment when you consider devices such as the Samsung SmartThings Hub which acts as a Z-Wave controller but also allows remote access. Granted, it doesn’t provide Z/IP (so far as I am aware) but the use case appears to be the same (if only for small 1-home deployments… the SmartThings hub allows effective IP based communication with Z-Wave devices.

So would zipgateways really only make sense for very large deployments or OEMs wanting to create their own hub similar to the Samsung one?


#2

They do, but they do more than just that. Z-Wave devices can talk to each other (so long as they’re all in the same HomeID) without having to involve a controller. A Z-Wave motion sensor can directly trigger a Z-Wave light bulb, there’s no requirement for the sensor to report to a controller, and then have the controller trigger the bulb - although this is certainly possible. Z-Wave networks can have many controllers.

They do. A better way to think of zipgateway is as a Z-Wave controller who’s interface is Z/IP messages received over IP.

zipgateway *is* a Z-Wave controller. It does the actual “translation” between IP and Z-Wave RF. zipgateway needs access to Z-Wave hardware, which is why you need a Bridge Controller UZB stick to work with it with the publicly-available RaspberryPi SD card image.

A zipgateway connects exclusively to a single piece of Z-Wave hardware. What you could have, is a Z/IP client that connects to multiple _zipgateway_s - be they in the same Z-Wave network, or otherwise.

Like you mention, zipgateway is useful for OEMs building hardware that behaves as a Z-Wave controller. This device that controls the Z-Wave hardware using zipgateway, if it exposes its functionality to the IP network, is useful for Z/IP client developers. You asked for some scenarios in your previous post, so I"ll try to exemplify with a couple of them. I’m over-simplifying the scenarios, but they should give you an idea of what’s possible and where zipgateway fits:

  • You’re an OEM building router/gateway-type devices. Your customers want Z-Wave “in the box.” You redesign your hardware to include a Z-Wave module. Instead of having to implement software support for the module and design an API for your customers, you install zipgateway as another one of your OS’s services (like you would a web server, for example) and tell your customers to develop their applications (middleware, uplinks, apps, web UIs, etc.) using Z/IP.

  • You’re a software development house. You see an opportunity to develop a middleware that supports Z-Wave devices. You’d like to support a wide array of hardware platforms and provide a standardized interface to control Z-Wave devices. Leveraging zipgateway in your middleware, you provide clients a standardized Z/IP interface and support any hardware platform that runs zipgateway. Because the communication with zipgateway happens over IP, there’s no requirement for your middleware to be “in the box” but could also be hosted in the Cloud somewhere. The same is true for the middleware’s UI/clients - they can be websites somewhere else in the Cloud, or Apps in a smartphone communicating directly to the middleware or through your Cloud - all the pieces can be replaced by something else, so long as the also speak Z/IP.

  • You’re an app developer. Instead of limiting your App to work only against the API (and thus, their “box”) of a specific gateway manufacturer (to re-use your example, the SmartThings Hub), you use Z/IP and instead support any and all Z-Wave controllers that expose Z/IP to the IP network, regardless of the manufacturer or their proprietary APIs.

  • You’re hardware manufacturer, producing end devices. You want your new product, a fancy in-wall thermostat with WiFi and a touch display, to be able to control Z-Wave devices, maybe some TRV valves, and get data from Z-Wave sensors, maybe temperature and humidity. You could add a Z/IP client to your software stack and control and receive information from Z-Wave devices by leveraging a zipgateway that’s in the same WiFi network, and is joined into the Z-Wave network that has the devices you’re interested in.

Looking at this from the perspective of “Z-Wave Public”, the combination of the zipgateway SD card image and Command Class documentation that was made available allows you to write software that communicates with a zipgateway in order to interact/manage the Z-Wave network it’s part of - Z/IP Clients - that fit in all the scenarios described above.


#3

As always, thanks for the comprehensive reply :slight_smile:

zipgateway is a Z-Wave controller. It does the actual “translation” between IP and Z-Wave RF. zipgateway needs access to Z-Wave hardware, which is why you need a Bridge Controller UZB stick to work with it with the publicly-available RaspberryPi SD card image.

You actually answered my next question already! :wink: I’ve installed the reference zipgateway Raspberry Pi image but noticed that my existing Z-Wave USB dongle isn’t supported (it’s an Aeotec Z-Stick Gen 5). I’m curious as to what is different about the UZB dongle… in my head the logic and translation is dong from Z/IP to Z-Wave by the software, though I suspect I’m missing something.

Sounds like I need to put my hand in my pocket and buy a UZB dongle…


#4

Okay, I’m sorry for this, but I’m still not quite understanding something… again, correct me if/when I’m wrong :slight_smile:

Correct use of a zipgateway
In a large hotel scenario there would be one zipgateway per room. Hotel staff would control the rooms by using either…

a) an app to talk to directly to each zipgateway using Z/IP
b) a web portal, communicating over HTTP to the staff members mobile app, with the portal then sending Z/IP packets to each zipgateway.

Is that correct?

TLS vs DTLS
If zipgateways use DTLS for LAN connections and TLS for WAN connections how is it determined what to communicate over? Will a zipgateway always accept connections on DTLS and TLS and leave it to the client to pick the best procotol dependant on their location or type of network?

Z-Ware Portal
Is ‘Z-Ware’ purely a sample/reference portal that could interface with a zipgateway? Does it communicate using Z/IP to the zipgateways?

Am I correct in thinking that a zipgateway MUST have its private PKI certificate (created by Sigma Designs) loaded in to the portal in order to authenticate it?


#5

That’s right, zipgateway requires the Z-Wave module to be loaded with a “Bridge Controller” library. Your Aeotec Z-Stick is probably a “Static Controller” which doesn’t have some of the functionality zipgateway requires. When buying the UZB dongle, besides making sure it’s a Bridge Controller, check that its frequency matches the region you’re in.

If you buy the UZB from DigiKey, do take note of this thread: Bridge Controller UZB Apparently, DigiKey’s system has no way of allowing certain products to be sold without signing an NDA/Dev. Kit agreement, so you’ll need to place the order over the phone in order to explain your situation.


#6

Yes, both are possible.

For the scenario in (a), the hotel could also authorize an app on a customer’s phone to communicate directly with the zipgateway in his room, let’s say by providing it with that zipgateway’s DTLS PSK, allowing the customer to control the room through his phone.

For the scenario in (b), the _zipgateway_s could connect to the Portal over a TLS tunnel - the Z/IP packets are sent trough this tunnel, avoid headaches like NAT traversal. Provisioning of the zipgateway’s can happen securely over this tunnel using the Gateway Command Class.
The communication to the staff member’s apps could be over HTTP, maybe the phones use a REST-like API like the one provided by Z-Ware’s Web API, or a custom-built one.

This setup is either performed by manually modifying zipgateway’s .cfg file, or over the Gateway Command Class. If properly configured, zipgateway will allow both connections, say a Portal over the TLS tunnel and DTLS from the LAN WiFi.

The Z-Ware Portal is exactly a sample/reference implementation of a very simple Portal that can communicate with _zipgateway_s. It can be setup in “local” mode, where zipgateway’s communicate only over DTLS (this is the setup included with the RPI SD card) or in “portal” mode, where all communication happens only over the TLS tunnel.

For “portal” deployments, you only need Sigma’s certificate if you’re planning on connecting to Sigma’s Demo Portal, which hosts the IFTTT integration examples that the guide on “Z-Wave Public” explains how to setup. The idea is that in “real” portal deployments, both the certificates on the portal and the ones in the _zipgateway_s are generated by you. This way, only your _zipgateway_s can authenticate with your portal, and only your portal can be used with your _zipgateway_s.


#7

Thank you :thumbsup:

Need some time to digest this… :slight_smile:


#8

What are ‘Z-Ware apps’, as described in this YouTube video…?

They are described as sample apps for the Z-Ware portal and the video makes it sound like they are intended for developers to pick and, quickly customise and then push out.

It sounds like the code (SDK) is available as part of the controller developer kit. However, if the Z-Ware portal is only intended as a simple/reference design, what are these Z-Ware apps really intended to work with? The video mentions pre-certification (if no code changes are made) but if you need the app to work with anything other than the Z-Ware sample portal it sounds like code changes are almost essential.

Have I misunderstood? Is ‘Z-Ware’ the marketing term used for the Z/IP and zipgateway SDK?


#9

Z-Ware is *not* the marketing term for the Z/IP and zipgateway SDK. Z-Ware is a Z-Wave middleware. It is comprised of a “business logic” component, called Z-Ware C Library, that handles communication with a zipgateway. This component in turn communicates with a web server over FastCGI. The web server hosts a number of web applications (Z-Ware apps), the ones mentioned in the video. This complete Z-Ware “stack” is included in the RPi image, along with the documentation for building more of these web applications using the Web API that Z-Ware provides (which in turn calls down to the FastCGI). There is, for example, a web application that presents the user interface in a manner that is suitable for interaction from a mobile phone. There’s another for TVs, another for computers, and a “developer” interface.

The “pre-certification” mentioned in the video should be understood as the whole Z-Ware stack having gone through Z-Wave Certification already, and passed. If you were to deploy the provided Portal as-is and use Z-Ware as your middleware, also as-is, you’d be deploying a setup that has already been certified, and are therefore guaranteed to pass Certification. You still need to submit the product for Certification, and have it tested, though.

Like you mention, you’ll likely want changes to the UI and possibly the middleware. Maybe you want to add support for one of the less-popular Command Classes for which support isn’t provided yet. Maybe you’d like to split the way devices are shown in the UI and differentiate between “preferred devices” from manufacturers you have a partnership with and “other devices.” These changes nullify the pre-certification, as it is entirely possible that mistakes have been made in the implementation of the new Command Class, or the UI has been modified in ways that make it no longer conform to Z-Wave Certification. Just like in the case where you made no modifications to Z-Ware, you still need to submit the product for Certification, but there’s no guarantees anymore that everything is in proper working order.


#10

Okay, thank you. My UZB is en route so I should have a chance to play soon.

For what it’s worth… I do think current documentation and videos leave a lot to be desired…

Z-Wave
Z-Wave Plus
S2
Z/IP
ZIPR
zipgateway
Use of TLS vs DTLS
Z-Ware
Z-Ware C API
Z-Ware portal
etc…

…these are all things that I’ve had to peice together over weeks of reading around. Granted much of this would be explained if I signed up for a dev kit but as a interested individual who may or may not like to develop apps and solutions, trying to understand how everything fits together is… not easy!


#11

Hopefully @hanskroner or someone isn’t sick of my ignorance and is able to help answer a few more questions :slight_smile:

In the past few weeks I’ve managed to get the reference zipgateway fully working on my Raspberry Pi. I’ve got a few nodes (switches and smoke alarm) added to the zipgateway controller and I’m able to read the sensors and turn on/off the switches using both the reference Z/IP client and the sample Z-Ware Android application.

I have also compiled the Z/IP reference client for my laptop (MacOS) and I’m able to connect to the zipgateway successfully (albeit with one minor problem*).

On to the questions…

  1. Z/IP apparently works over UDP or TCP…if this is true, do I need to modify the config to connect over TCP? When I attempt to connect over TCP (TLS, port 443) I get an SSL handshake error and the reference client on my Mac does not connect

  2. The IPv6 subnet allocated to the Z-Wave nodes is different than the IPv6 subnet the rest of my wifi and my laptop run on. A look at the adaptors on the zipgateway show…

br-lan Link encap:Ethernet inet6 addr: fe80::36e:b225:xxxx:xxxx/64 Scope:Link inet6 addr: fd00:a622:3f5d:4590:16f3:xxxx:xxxx:xxxx/64 Scope:Global

and

wlan0 Link encap:Ethernet inet addr:192.168.0.251 Bcast:192.168.0.255 Mask:255.255.255.0 inet6 addr: fe80::1b64:92d3:xxxx:xxxx/64 Scope:Link inet6 addr: fd67:eacc:b504:1:a0b6:xxxx:xxxx:xxxx/64 Scope:Global

This seems to match what is in the ‘Z/IP Gateway Bootstrapping’ document.

I can only connect over wifi to the zipgateway using the wifi adaptor’s fd67:: address. *While this is fine, performing a list command shows no nodes (since, presumably, it is using mDNS/DNS-SD to locate devices).

Do you know how to get around this issue? Should I change my wifi IPv6 subnet to be the same as the zipgateway? What about existing network installations? Can the zipgateway be configured to use a different IPv6 network range?

  1. I’m still a little foggy on how the zipgateway resolves the IPv6 addresses… I would have expected to see ARP table entries for all the Z-Wave nodes but there are none there. So, in the same vein as the above question, how can remote clients connect to the HAN IPv6 addresses?

  2. Sorry to labour this point… but I’m still stuggling with the concept of Z/IP at all. Even the sample Sigma Designs mobile app [for Android] uses ZWare, so security is handled by TLS and credentials (instead of DTLS and a PSK). My mobile can connect to the zipgateway and see all of the nodes and I don’t have to worry about IPv6 addressing, the ability for mDNS to function, and then the problem that every Z/IP client may see a list of different nodes depending on whether the mDNS query was successful. I can see Z/IP clients being the ‘master control centre’ in a large environment (e.g. a factory) where it connects to dozens of zipgateways but this then leads me to my next question…

  3. Z/IP uses a single PSK for encryption. How is authentication handled? Going back to my factory example, how could I give a janitor the ability to turn on lights in a certain part of the building, but not turn off essential factory machines? Presumably the PSK would need to be ‘hidden’ away in bespoke front end code which also handled different users and the ability to control various nodes.

As always, thank you for your time in helping me better understand Z-Wave!


#12

Good to hear that you’ve been making some progress :smile: Some of the questions start getting off the “role of zipgateway” topic of the thread, so I won’t go too deep into them. If you feel you still need further clarification, just start up a new thread and we’ll pick it up there. This way the forum stays readable and searchable in the future :wink:

Z/IP works only over UDP. UDP provides the same “stateless” view of the world the Z-Wave RF protocol uses and is a better fit. zipgateway can be setup to have a TLS tunnel as an entry point, but this is just a tunnel - the payload inside is still UDP packets.

This is the way zipgateway is designed to operate. You can configure the IP prefixes for the LAN and HAN subnets in zipgateway.cfg but they’re intended to be different from your LANs existing subnet (and each other’s).

In IPv6, ARP is replaced by NDP (Neighbor Discovery Protocol) and things work differently than they do in IPv4. This is a long and winding topic to discuss, but the short version is that clients can’t connect to the HAN addresses in the majority of cases, there’s no easy fix for it, and fixing it currently requires access to the router (the router needs to be told to route all requests to the HAN subnet through the zipgateway’s IP address). The “Home Gateway Initiative (HGI)” has been hard at work for a while trying to come up with a good, secure solution to this issue.

Z/IP is the standard way to transport Z-Wave messages over IP. Nothing more. The mobile app (which isn’t really a mobile app, it’s an app that opens a webpage) is not communicating directly with zipgateway - it’s communicating to a server (the Z-Ware Portal) which in turn communicates with a zipgateway (this last leg of the communication is with Z/IP). It doesn’t make a lot of sense to use Z-Wave Command Classes to talk to a middleware/Portal - but it does make sense to use them to talk to Z-Wave devices.

Authentication isn’t handled. DTLS is there to ensure that other devices that share your WiFi/LAN cannot communicate with a zipgateway unless they’re provided the PSK. In your factory example, I doubt the janitor’s mobile app would be communicating directly to a zipgateway (or a couple of them) that runs the factory floor. It’s more likely that his app would be communicating with a middleware/Portal (which would handle authentication) and this middleware/Portal then communicates with the zipgateways (again, using Z/IP) .


#13

Thanks so much for the quick reply and, yes, understand you not wanting to get off-topic on this thread :slight_smile:

Okay, cool. No probs. Buuuuuuuuutttt…

I’m still confused :slight_smile: If mobile apps will probably always use the ZWare Portal and if the Z-Wave HAN IPv6 subnet is seperate to the LAN IPv6 subnet… what devices/clients are even intended to actually act as the Z/IP client?

I still don’t understand the use case for Z/IP

Z/IP is the standard way to transport Z-Wave messages over IP.

Okay, so let me be blunt… what’s the point? What’s the use of transporting Z-Wave messages over IP? Why not just always use a gateway (whether that’s ZWare or something proprietary)?

For the scenario in (a), the hotel could also authorize an app on a customer’s phone to communicate directly with the zipgateway in his room, let’s say by providing it with that zipgateway’s DTLS PSK, allowing the customer to control the room through his phone.

To pick back up on a previous reply, this scenario would never work, right? I know I may have picked a bad example, but a hotel could never give out a PSK since it would need to be changed every time a new guest stayed in the same room.

It seems to me like having a gateway with web server that provided API access with JSON payloads (to enumerate the nodes) would have been a far simpler and more scalable solution.

Again, I feel like I’m still missing the real world use case of Z/IP. There’s probably a glaringly obvious example I’m not seeing :slight_smile:


#14

I think you might be missing the point of Z/IP because you’re fixating on higher levels of the stack - the mobile apps and the web APIs. Z/IP solves for IP communication the same problem that Z-Wave RF solved for IoT device manufacturers: It gives devices a common, standardized language to speak with.

I’ll point you back to the reply I gave previously on this thread about the different use cases. Mobile apps don’t always have to got through a Portal, though lots certainly will. There’s products out there that are basically Android tablets that never leave your home and might even be fixed to a wall - they’re prime candidates for being Z/IP clients. Without going that far, the app that typically connects to a Portal would do well to also be a Z/IP client so it can provide some local functionality when you’re home in case the Internet uplink is down. zipgateway’s can network amongst themselves using Z/IP, providing an IP backbone for a Z-Wave network.

If you still can’t see the point, ask yourself what would you do if you were building a product and Z/IP wasn’t around. What would 3 other competitors to you do? Would an average customer be able to buy a product from you and another from one of your competitors and get them to work together - any one of them, and all of them? You’ll probably come up with a proprietary API, and if the world is lucky you’ll open source it with the, almost standard, poor documentation. The only way a customer will ever get products to sort-of-work-together-in-the-most-basic-of-cases is if they buy an additional “translator bridge” device that has half-assed support for 7 different RF protocols and 13 different proprietary APIs and provides whatever minimum-common-denominator functionality all their supported platforms could share. See the problem? This is the interoperability Z/IP aims to provide for IP devices that want to interact with Z-Wave devices - just like Z-Wave RF does for communication between Z-Wave-enabled devices. Instead of having to focus your development and maintenance efforts on all sorts of different APIs, you support one standard and focus the rest of your time in areas that actually add value to your product.


#15

Thanks Hans. I think the wall mounted tablet in the home being interoperable with lots of Z-Wave devices (and zipgateways???) is the best use case I’ve heard yet.

However, it still seems like an elegant solution to a problem that isn’t quite there.

Z/IP solves the interoperability problem… but developers still need to learn to use Z/IP and a zipgateway is still required in the home. Standard Z-Wave controllers won’t work.

Alternatively Z-Ware could be used exclusively… now, instead of having to learn Z/IP, developers just learn Sigma Designs single API. Now anything that supports HTTP could be used as a client and the gateway now just converts the HTTP/API calls in to native Z-Wave RF packets.

Z/IP isn’t an generic protocol that supports Zigbee, BTLE and Wifi connected devices, for example. So if I were developing a controller or a whole home automation system I still need to look at making use of 3rd party API’s and, potentially, multiple gateways for the different RF networks. As a manufacturer or developer, I’m not sure how Z/IP helps.

I think we’re getting to the point where we may have to agree to disagree, but I do very much appreciate your input and help thus far.


#16

You got it :slight_smile: Hopefully there more you dive into Z-Wave the more you’ll see where Z/IP fits. From a purely Z-Wave perspective, interoperability with other wireless automation technologies is secondary to interoperability across Z-Wave devices. If that’s the chasm you’re trying to bridge, Z/IP will not get you all the way there - that’s not what its trying to do.


#17

Hi…i am a new user here. As per my knowledge Z-Wave devices can talk to each other without having to involve a controller. A Z-Wave motion sensor can directly trigger a Z-Wave light bulb, there’s no requirement for the sensor to report to a controller, and then have the controller trigger the bulb - although this is certainly possible. Z-Wave networks can have many controllers.