Benchmarking Jio: How does Reliance’s new service fair against its established competitors?

Reliance Jio Infocomm Limited, better known as Jio, is changing the definition of speed for mobile data users across India. It is India’s largest 4G network, without any 2G/ 3G-based services. This is a revolution of sorts! Not only is this about data; Jio’s ecosystem of apps and content is pretty amazing too!

PacketZoom carries out billions of file transfers from its servers to mobile devices from all over the world. India's bustling mobile user base forms a significant fraction of LTE traffic from India. Since we make it our business to know mobile networks, we needed to take a deeper look at Jio to optimise our service for Jio subscribers. And while we were at it, why not share our findings with a wider audience.

We measured the time taken to download a 100 KB file hosted on a popular content delivery network (CDN) from thousands of devices across the country. We collect millions of data points every day from different parts of the world from users on different types of networks, and this served as the basis for our exercise.

These are the throughputs that we observed for different networks in units of Kilobytes per second. The higher the throughput, the faster the network.

Results indicate that Jio has the lowest throughput, approximately a quarter that of Vodafone, another popular Indian carrier, and almost one-sixth that of Verizon. It came as a surprise to us!

Having spent the last three years developing a mobile-specific protocol that serves millions of requests every day from across the world and various networks, this intrigued us and we decided to dig deeper. In order to get finer networking stats, we proxied the requests through our own servers, running our custom UDP (User Datagram Protocol)-based protocol.

The first thing we looked at was Network Latency, which is the length of time it takes for a packet of data to be sent plus the length of time it takes for an acknowledgment of that packet to be received. This is also known as the Round Trip Time (RTT) of a network. In traditional networking protocols, the throughput is (roughly) inversely proportional to RTT, meaning that the higher the RTT, the lower the throughput.

The chart above shows the average RTT observed for different networks. Jio and Idea have the highest RTT compared to other networks. This further substantiates the lower throughput of Jio; hence the inverse correlation gets justified.

Since we control both the client and server sides in our UDP-based protocol, one of the advantages is that we are able to get data on packet loss, which is not easy to get from TCP (Transmission Control Protocol). Unless you control the client and server side of TCP stack, there is no way of knowing how many packets were lost due to a lossy network and not because of congestion, but that is something to discuss on a future post.

Mobile networks are known to be lossy in nature. This is one of main factors that affect throughput. In a lossy network, an extra round trip is required to retransmit the lost packet. What further compounds the problem is that traditional networking protocols (such as TCP) consider all types of packet loss as a sign of congestion; this means that even a small amount of packet loss can have a disproportionately high negative impact on the throughput!

Simply put, the higher the packet loss, the lower the throughput.

The chart above shows the average packet loss for each network. Similar to the RTT data, Jio and Idea are the most lossy networks.

RTT and Packet Loss data explains Jio’s lower throughput. It is also a possibility that the bandwidth in general is much lower for Jio than for other networks. While we cannot control the available bandwidth, we can still improve the underlying protocol to squeeze out the best available bandwidth.

The solution lies in understanding the difference between a packet loss due to congestion and a packet loss due to a lossy network (medium interference), and acting on it differently, or being better at estimating the available bandwidth. Ultimately, reducing round trips could do wonders. Our hypothesis- higher packet loss and higher latency results in higher speed improvements.

We ran the same experiment using our own protocol, which is designed for mobile applications, keeping in mind those factors, and here are the results.

Jio and Idea, having the highest latency and packet loss, show the maximum gains as expected. And with lesser latency and packet loss, the gains are also less, proving our hypothesis.

Jio is a new carrier and with further investment in its infrastructure, its performance could improve. However, without any changes to the infrastructure, it is still possible to beat the best network in India just by changing the underlying network protocol to one that is designed specifically to handle challenging mobile network conditions.

Time will tell whether Jio lives up to the hype we have seen over the past few weeks. Till then, let’s ride the Jio wave and see what else Reliance Infocomm has in store for us!

(Disclaimer: The views and opinions expressed in this article are those of the author and do not necessarily reflect the views of YourStory)