For any technology to become mature, a huge amount of tests have to happen behind the scenes. With hardware and software components recently developed and, usually, under very tight deadlines, extensive testing is an assured way to avoid market failure. 5G is no different in that respect. In fact, testing this technology becomes especially relevant because it is not a simple upgrade, it involves much complexity. Those offering 5G are looking at high demand users who may depend on it for enabling further technological advancements. As such, the scope for flaws are practically non existent.
Much of the current 5G device testing has a focus on signaling and raw throughput tests using tools like iPerf. For example, Conformance Testing for 5G (3GPP 38.523) does not have any real world data transfer scenarios. While 5G signaling test is of much significance, there is also a critical need for real data transfer tests from an end user angle. This allows a real, clear, on-ground understanding of how the device will perform.
Wireless devices today are incredibly complex which, in spite of heavy testing, may leave possibilities for issues like memory leaks and race conditions. For mobile devices such issues are usually not observed because mobile phones usually get rebooted for one reason or the other – battery draining out, turning on the airplane mode (soft shutdown), etc. In cases where none of these happen the user might just reboot the phone when s/he thinks it is getting slow or unresponsive.
Unlike mobile devices, the reboot option does not apply to 5G CPE devices. Once installed, these devices are supposed to be out of sight and out of mind. They are meant to work for years, without any external support. Other devices such as TVs, home IoT devices, security cameras, mobile phones, laptops, tablets, smart watches etc are connected to it constantly. Since 5G CPE devices are expected to run for years at a time, without any reboot, issues like minor memory leaks etc. might show up and affect their performance within a few months into usage. These may be taken as symptoms for lower performance, difficulty in connecting new devices, etc. Certain parts of the system get tested only under heavy loads, e.g. buffer limits, queues. Many borderline events like race conditions get hit under heavy loads. Hence, the device behaviour must be tested in suboptimal conditions.
A normal 5G end user (e.g.: a 5G connected home) will have multiple traffics that have different QoS requirements – video streaming, browsing, voice calls, security camera, IoT devices, etc. – running simultaneously. It is critical to simulate real world environments and multi QoS traffic loads in the test lab because the alternative to troubleshoot real field issues would be to send support staff onsite, a rather expensive option.
It is possible to minimize this challenge by running multiple long duration soak tests on different setups under different client load and traffic conditions. Even a seemingly simple action like adding/removing multiple 5G cells might unearth some issues if done long enough. Another example is real time traffic like Skype for Business calls getting dropped during peak business hours when maximum number of employees are using the online resources.
A two week soak test covering the different usage scenarios in a typical home or small office can help bringing out issues. For home use cases, video traffic at different resolutions might be of higher importance whereas for office use cases productivity tools like emails, ftp, Skype for Business calls might carry more weightage. VoIP call drops during peak business hours is a common issue reported from multiple real offices, so that is definitely one use case to cover in the soak test.
5G is still unproven in the real world. Any negative experience, even if it is caused by non-3GPP components in the device, may get blamed on 5G. So, the stakeholders in the ecosystem will have to leave no stone unturned to ensure that whatever they put in front of the customer is their best.
How do you test 5G CPE for long duration?
SWAT WiCheck emulates 100s or 1000s of WiFi clients and helps you to recreate real world scenarios very easily. It helps you to test your 5G CPE for 1000s of hours with multiple users accessing the network simultaneously. It runs real applications(Facebook, YouTube, YouKu, Skype for business calls & more) and measures the end user experience.
Have you got any questions regarding 5G CPE end to end testing for long duration? Please reach out to us HERE
More details of SWAT WiCheck
P.S: Alethea Blog has been selected as one of the Top 25 Wireless Technology Blogs on the web
