VPNs for Virtual Workshops & Interops

The term "interop" as we know it today came from a trade show of the same name founded by Dan Lynch in the late 1980's. Lynch has experienced a lot of pain while at SRI and USC-ISI getting interoperable implementations of TCP/IP working between companies. The "INTEROP" show brought together many competitors to connect their different devices together on the "InteropNet", common network for vendors to communicate over. If you found an incompatibility with someone else's devices, you could walk over to their booth and work out a solution.

In a similar fashion, in the early days of professional live IP video solutions, many interops / workshops / "dirty hands" style events were held. Starting in 2015, events were held to achieve interoperability in PTP/SMPTE ST 2059, VSF TR-03 that later evolved into SMPTE ST 2110, and the AMWA NMOS APIs. The experience at these events lead to the public multi-vendor demonstrations at the IBC and NAB IP Showcases. It was always a very exciting experience to have 20-50 engineers from different companies from all over the world in the same location at the same time for several days working together to interoperate. A great time to talk shop, solve problems, and have a few drinks with dinner. We were very happy to host many of these events at the Disney facility in The Woodlands, Texas.

But despite the incredible value of these events in improving interoperability of standards and specifications, it was very costly to ship personnel and equipment halfway around the world to the interop site. As the Advanced Media Workflow Association (AMWA) was working on its NMOS APIs, we wondered if there was a "virtual workshop" option. Instead of having people travel huge distances to hook their computers into a network, could we just hook them together remotely over the Internet? While this was not such a great option for ST 2110 due to multi-Gbit/s bandwidth and low latency required for effective PTP synchronization, the NMOS APIs (over REST & MQTT) were using very little bandwidth and could handle some latency.

A Virtual Private Network (VPN) came to mind as a solution. But which VPN to use? Many people use VPNs to connect over the Internet to their corporate network, or use VPNs to anonymize their Internet traffic. But we were seeking a VPN that was a "mesh network", where any participant could host servers/clients on the VPN and have connectivity to the servers/clients of the other participants. We also wanted a VPN that would allow us to run DHCP servers, DNS servers, and perhaps even do multicast DNS. In a perfect world, we wanted a VPN that could "sneak" through corporate firewalls (although it turns out that corporate firewalls are pretty smart). Finally, we wanted a software solution, so that participants would not have to purchase a particular kind of VPN hardware to use it.

Of the options, AMWA chose the SoftEther VPN, which is an open-source, layer 2, cross-platform, free, and multi-protocol VPN system from University of Tsukuba, located in Japan. SoftEther VPN uses Ethernet over HTTPS, which for many firewalls appears like any "normal" HTTPS web page and makes it through unscathed. However, some smart corporate firewalls can still guess SoftEther from its packet transport fingerprint and in such circumstances, users may need a special tunnel made for the VPN. SoftEther also has options to allow more "traditional" VPN clients to connect via L2TP/IPsec, MS-SSTP, or OpenVPN.

SoftEther is very easy to use, with a nice GUI for server control. It has an easy to install software VPN client on Windows. Participants can also run their own local SoftEther server hub with a "cascade" connection to the main hub. In this way, local LANs can easily be bridged over the VPN.

The AMWA VPN SoftEther server runs on an AWS t3.small sized Linux EC2 instance. This costs about $0.02 per hour, but to date has met all of the bandwidth requirements of the bursty HTTP REST or MQTT API calls of the NMOS APIs. The VPN server instance is in an auto-scaling group of size 1, so in case the instance goes away, it will be spun up again right away. A Windows "Scheduled Task" writes the VPN server configuration file to S3 on a regular basis so that config changes will be reflected if the instance has to be re-launched.

Running on a second EC2 instance on the VPN is the Internet Systems Consortium (ISC) DHCP server and ISC BIND9 DNS server. This allows VPN users to receive a DHCP IP address, and for NMOS clients to discover the location of NMOS services using DNS Service Discovery (DNS-SD). Workshop participants can use nsudpate to add their services to the DNS server.

During a virtual workshop (which generally lasts a week), participants have a daily Zoom check-in videoconference to go over the objectives for the day and to discuss issues as a group. Outside of the group videoconference, participants use Slack for text chatting, file exchange, and Slack Calls among sub-groups.

Virtual Workshops have added a new facet to AMWA specification development and testing. By having a lightweight way for multiple vendors from around the world to try out interfacing with each other, specifications can quickly be checked for accuracy, clarity, and appropriateness. And while we miss those face-to-face contacts, in the age of tight travel budgets and Covid-19, the virtual workshop lets "the show go on".