India is currently the world’s second largest manufacturer of mobile phones after China. This growth can be attributed to the rapid internet penetration, growing disposable incomes and widening base of tech-savvy consumers. The Indian mobile components manufacturing and assembly market accounted for $25.3 billion in 2018 and is projected to reach $74.1 billion by 2024, at a compound annual growth rate of 19.2 per cent during 2019-2024, which consolidates India’s position as a leading player in this space.

Testing and measurement is crucial for cellular phones in order to ensure product quality and ratify their performance. Therefore, testing of cellular phones is of great significance at several stages in their life cycle; for instance, during the development process, prototype testing is required to accredit the performance against design specification wherein portions of mobile device development are tested. These tests mainly include hardware, software and mechanical testing. Further, when manufactured, the finished mobile devices must undergo factory test to ensure delivery of customer satisfaction. Factory testing is conducted automatically, which includes mobile application testing, hardware and battery testing, signal receiving, network and protocol testing, mobile games and mobile software compatibility testing. Finally, when the devices are to be repaired, tests are performed to diagnose the problem and repair it successfully. As per data by Anritsu, nearly 7 million phones return for repair each year and about half of these are fixed by the service provider or manufacturer.

Categories of phone testing

There are mainly three categories for cellular device testing:

  • Level 1: Involves testing the mobile origination and mobile pages. While the successful completion of Level 1 tests is necessary, it is not sufficient to determine the working of a device on a live system.
  • Level 2: Testing at this level generally involves testing receiver sensitivity, transmitter frequency, handover and maximum power testing in order to evaluate the usability of the phone under various conditions.
  • Level 3: Despite the two aforementioned testing levels, some glitches can still be faced in the device such as digital modulation errors, timing errors and waveform quality problems, which are addressed in level 3 testing by adding modulation and timing analysis.

Mobile application testing

The mobile app market is growing exponentially and so is the competition in the market, leaving the end user with a humongous number of applications to choose from. Application quality drives the success, which can significantly be attributed to mobile app testing. The growing numbers of cellular devices are driving massive fragmentation of operating systems (OS), screen sizes, variations of standard OS and more. There are several efficient tools available for the testing of mobile apps, which include Appium, Calabash, Monkey Talk, Native Driver and See Test.

Types of mobile app testing

  • Compatibility testing: Compatibility testing has the highest stack when it comes to app testing, as it ensures the successful working of the app with selected OS or devices.
  • Installation testing: Essentially done at the initial stage of development, installation testing verifies that the installation process goes on smoothly.
  • Localisation and performance testing: Localisation checks if the app adheres to the local cultural settings and customises apps according to the target country and language. Performance testing essentially covers the device application, server and network performance.
  • Usability testing: Usability test evaluates three main criteria for the target audience – efficiency to test the accuracy and completeness; effectiveness, which checks the resources expended in relation to the accuracy; and customer satisfaction.
  • Conformance testing: Conformance testing checks whether certain standards pertaining to the marketplace and enterprise policy guidelines are being met.
  • Security and stress testing: Data and security standards should be verified as a part of security testing, often done through routing the cellular device’s data through a proxy server like OWASP Zed Attack Proxy and looking for vulnerabilities. Stress testing focuses on finding exceptions, hangs and deadlocks that may go unnoticed during the functional and user interface testing.

There are several other tests performed for the usability and consistency of an app. These include laboratory testing, memory testing, power consumption testing, functional user interaction testing, visual testing, interrupt testing, updates and certification testing. With the introduction and diversified growth of innovative technologies every year, there are several trends to be tapped in the development and testing of mobile apps.

ML and AI

Machine learning (ML)-and artificial intelligence (AI)-based techniques can test suite optimisation and help in identifying inessential and unique tests. Log analytics can be performed by ML, which can be used to identify test cases to be executed automatically. AI can be used to efficiently identify risk areas of the mobile app for prioritising regression testing. Further, AI-powered bots can run exploratory tests or crash tests with greater precision and speed.

Certifaya is an AI-powered app testing bot, that is used to test the behaviour of the app over multiple devices and gives real-time insights into its performance. Certifaya uses ML algorithms to navigate through the app and detect issues and generates reports on the same.

According to Juniper Research, global spending on AI will grow to $7.3 billion per a year by 2022, up from $2 billion in 2018, as there is a surge in companies investing heavily in AI tools. AI/MI are already equipped for getting real-time information and detail analytics and now with mobile apps incorporating these advancements, the testing technique is set to be revolutionised.

Internet of things

Cellular and internet of things (IoT) devices are set to collaborate seamlessly in the future, which will provide users a better experience and greater control over the gadgets via a unified application. IoT testing would be applicable to IoT-based devices, which would test their usability and compatibility with the IoT system.

This will help in the simulation of the sensors, data validation integrity providing user authentication and data privacy. These techniques will also be used to trace the performance of the connected IoT devices in real time.

Impact of 5G

Moving from 4G to 5G is a significant shift in the network ecosystem, which will provide users with higher bandwidth for uploads and downloads. As per the Statista portal, there will be around 100 million 5G connections worldwide by 2021. In order to test 5G-powered mobile apps, test automation, built-in systems and monitoring would be required. Three main aspects of 5G technology have been identified by 5G and 3GPP, which coexist in the same network infrastructure. eMBB provides greater bandwidth and reduced network latency. URLCC is still in the process of being standardised and will provide virtual reality gaming, robot-enabled remote surgery, autonomous driving, etc. Massive machine type communication is a narrowband access type to sense, meter and monitor use cases.

US-based T-Mobile is updating its device testing capabilities to include 5G and has unveiled a new 20,000 square foot facility which will test 5G devices, besides the ones that enable licence-assisted access, narrowband IoT, LTE and 3G. As per the company, the lab consists of more than a dozen testing areas, ranging from radio frequency signal testing to voice call/sound quality, video optimisation and data throughput testing; in-depth testing of software, apps and services; and durability testing including drop testing, water testing and sensitivity to heat. Also, the lab is fitted with equipment to test devices across a range of frequencies from low band and mid-band to millimetre wave – in both its current and planned 5G spectrum.

Blending Agile and DevOps

Agile successfully aids diversified requirements, while DevOps serves the demand for speed. DevOps involves practices that help integrate development and operation activities, reducing the time from development to operations. It deploys the software it is running on, automatically and on demand, at any stage of the mobile app development cycle. DevOps reduces the release time and enables the development of better application. It can also dodge inefficiencies and bottlenecks.

The adoption of Agile and DevOps has helped the industry in developing and delivering quality software faster. However, to fully support DevOps, mobile test automation must be a part of it. The integration between cloud-based mobile device labs such as Kobiton and test automation tools like Katalon will drive the growth of mobile test automation in the near future.

Cloud apps

Cloud apps, along with the growing use of the app programming interface, are enabling successful quality assurance decisions.  The industry is focusing on building cloud apps, which require minimum operational costs and are highly efficient.