NSS Labs Announces Industry’s First SD-WAN Group Test Results

by | Aug 8, 2018 | Globe Newswire

3 products receive Recommended Rating; 4 products receive Verified Rating; and 2 products receive Caution rating

AUSTIN, Texas, Aug. 08, 2018 (GLOBE NEWSWIRE) — NSS Labs, Inc., a global leader and trusted source for independent, fact-based cybersecurity guidance, today announced the results of the industry’s first Software Defined-Wide Area Network (SD-WAN) Group Test. In this groundbreaking, inaugural test, nine of the industry’s leading SD-WAN products were examined to help enterprises understand the merits of products in the market and identify the capabilities best suited to meet their use case requirements.

SD-WAN is becoming increasingly mainstream and the products validated in the 2018 NSS Labs SD-WAN group test offer some impressive capabilities for enterprises and service providers.

The capabilities that were assessed by NSS Labs include: 

  • Remote Initial Configuration
  • WAN Impairment and Link Failover
  • Dynamic Path Selection with SLA Measurements
  • Path Conditioning and Application-Aware Steering
  • Link Saturation and Congestion
  • Application-Aware Traffic Steering
  • Performance by Application Flows
  • Raw Packet Processing Performance – UDP Throughput and UDP Latency
  • Maximum Capacity
  • HTTP Capacity and limits
  • Application-Aware Response Time
  • Security Effectiveness

While security effectiveness was not included in the final scores, it was evaluated in products offering security capabilities.

Adoption of SD-WAN has reached an inflection point and nearly every distributed business is deploying, evaluating, or planning to implement an SD-WAN as part of its IT vision. The traditional WAN architecture is becoming increasingly expensive and ineffective as the adoption of cloud-based applications is changing the type of traffic remote branch offices are experiencing. This growth in the adoption of cloud-based applications, be they software-as a service (SaaS) or applications hosted in an infrastructure-as-a-service (IaaS), also disrupts the prominence of Multiprotocol Label Switching (MPLS)-based WAN connectivity to the branch.

The global market intelligence firm IDC estimates that the SD-WAN market will grow at a compound annual growth rate of 69%, reaching $8 billion by 2021.1 According to an NSS Labs research survey, 87.8% of U.S. enterprises surveyed indicate that their organization have or will adopt SD-WAN in the next 12 months. Of the enterprises currently deploying SD-WAN, 52.9% plan on expanding their use of the technology.2 

The NSS Labs SD-WAN Group test provides the most comprehensive technical research available of SD-WAN products on the market today.  Based upon hundreds of hours of rigorous testing, the SD-WAN Group Test includes detailed individual test reports for nine products, plus three comparative reports: a TCO Comparative Report, a Performance Comparative Report, and the NSS Labs Value Matrix Comparative Report.

Key Findings:

  • NSS Labs testing uncovered few outright flaws in the products tested, indicating that as a technology category, SD-WAN is primed for widespread adoption.
  • NSS Labs research indicates that the SD-WAN market will continue to evolve, and both SD-WAN and security will become integral components of the modern WAN Edge architecture.
  • Testing revealed the areas of primary differentiation for SD-WAN  products to be Quality of Experience (QoE) for VoIP, QoE for video, and network performance.  There was also a wide disparity in pricing and total cost of ownership (TCO) as would be expected in a high growth emerging market space.
    ° On a QoE scale of 1 – 5 for VoIP, with 3.4 being the minimum viable NSS Labs considers to meet the use case for VoIP, and with a theoretical maximum score of 4.41, the tested range was 2.49 to 4.38 with eight of the nine tested products scoring above the minimum of 3.4.
    ° On a QoE scale of 1 – 5 for video, with 3.4 being the minimum viable NSS Labs considers to meet the use case for video, and with a theoretical maximum score of 4.53, the tested range was 1.10 to 4.47 with seven of the nine tested products scoring above the minimum of 3.4.

Quality of Experience (QoE) refers to the measure of satisfaction an enterprise experiences with a voice or video service. NSS Labs tested nine SD-WAN products using a QoE scale ranging from 1 – 5 for VoIP and video. NSS Labs considers a QoE score below 3.5 for VoIP and video represents a significantly degraded voice call/video stream, out of a maximum achievable score of 4.41 and 4.53 respectively.

“The SD-WAN market holds strong potential and there are several impressive technologies available today for companies seeking to enhance or replace traditional WAN deployments,” said Jason Brvenik, Chief Technology Officer at NSS Labs. “The NSS Labs 2018 SD-WAN Group Test is the industry’s first baseline assessment of these technologies. Our testing validates that this is a technology that is ready for prime time. We encourage enterprises to examine our findings for insights regarding the capabilities, performance, and cost of solutions as they continue to evolve their WAN architectures.”

Products Tested:

  • Barracuda NextGen Firewall F-Series F80 v7.1.1
  • Citrix Systems Inc. NetScaler SD-WAN 10.0.0.207
  • Cradlepoint AER2200-600M v6.5.0
  • FatPipe Networks MPVPN/SD-WAN v9.1.2
  • Forcepoint NGFW 1101 vSMC 6.3.6, engine 6.3.6.19302
  • Fortinet FortiGate 61E v6.0.1 GA build 5068
  • Talari Networks Adaptive Private Networking (APN) Software vAPN 7.1
  • Versa Networks FlexVNF v120
  • VMware NSX SD-WAN by VeloCloud vEdge version 3.2

Unverified Products:

  • Cisco
  • Silver Peak

NSS Labs is committed to providing empirical data and objective group test results that enable organizations to make educated decisions about purchasing and optimizing security infrastructure products and services. As with all NSS Labs group tests, there is no fee for participation, and the test methodology is available in the public domain to provide transparency and to help enterprises understand the factors behind test results. Click here for more information about our group test policies.

Click here for more information on this test and the test methodology used or to purchase the individual test reports. Click here to download the NSS Labs Value Matrix, which provides a graphical comparison
Of the TCO per Mbps (Value) and the QoE scores for the VoIP and video of tested products.

Additional Resources: 

  1. IDC Forecasts SD-WAN Market to Reach $8 Billion in 2021, July 2017
  2. NSS Labs “Security Controls in the US Enterpris: Network Security Software-Definded Wide Area Network” January 2018

About NSS Labs, Inc.
NSS Labs, Inc. is recognized globally as the most trusted source for independent, fact-based cybersecurity guidance. Our mission is to advance transparency and accountability within the cybersecurity industry. Our unmatched foundation in security testing, along with our extensive research and global threat analysis capabilities, provide the basis for NSS Labs’ Cloud Platform for Continuous Security Validation. This cloud platform empowers enterprises with objective, empirical data and allows them to gain continuous visibility, gather actionable insights, and rationalize investments in their cyber programs. The cloud platform lets enterprises know where they stand by continuously validating the effectiveness of their security products and assessing the impact of unmitigated risks to the enterprise stack. This gives business leaders the relevant information they need to substantiate their security investments. CISOs, Chief Security Architects, SOC and Threat Analysts, and information security professionals from many of the world’s largest and most demanding enterprises rely on trusted information from NSS Labs. For more information, visit www.nsslabs.com.

Contact:
Jessica Johannes
Phone: +1 512-498-7076
[email protected]