Network Performance Testing - The Real Winners February 23, 2017 14:30

This week RootMetrics released their 2nd Half 2016 - Mobile Network Performance results which declared Verizon their scorecard winner.  T-Mobile quickly responded with a blog post first diminishing the RootMetrics testing approach, and then touting the crowd sourcing approaches that put their networks in the best light.  So which of these testing approaches provide the best evaluation of a carrier's network.
 
I would say both are necessary to fully evaluate network performance.  The strength of the crowd sourced data collection approaches (Open Signal and Ookla Speedtest) lies in the fact that they collect an immense amount of data in the most populated areas (where most of the people/their testers live, work, and play).  The users can be indoors or outdoors, in a car or walking down the street.
 
The systematic data collection (drive testing and walk testing) that RootMetrics conducts provides a network performance view across a broad sampling of each carrier's network with an equal samples across all areas.
 
So when I see Verizon winning the RootMetrics Scorecard, I see that as winning the network performance trophy across the entire US map, metro areas, suburban, small towns, Interstate Highways, US Highways, and State Highways.  If I live or travel frequently in rural areas, I would want my carrier to score well on the RootMetrics tests.  But RootMetrics doesn't test in your home, your church, or you office so the network performance in those areas would be best measured by crowd sourced data.
 
Winning the Open Signal and Ookla Speedtest is an indication of how each carrier's networks will perform in the more urban markets (think cities of 70,000 people or more).  If I live in one of these areas and rarely need coverage beyond these more urban areas, T-Mobile performs very well. (I am a T-Mobile customer).
Looking at the OpenSignal results for Kalispell, MT; testing results for T-Mobile and Sprint are not included, only AT&T and Verizon.  This is likely because T-Mobile and Sprint have few customer phones in this market since the only store fronts are AT&T and Verizon.
 
As a T-Mobile customer, I have enjoyed great network performance in all of the cities that encountered in 2016 with a very short list of performance challenges that I think are best measured by the RootMetrics Scorecard.
  • I-90 outside of the Seattle area
  • I-10 between Palm Springs and Phoenix
  • Kalispell, MT (lack of in-building service)

As you can see below, a section of I-10 only has Verizon crowd-sourced data.  Specific to this area, T-Mobile won't have any crowd-sourced data because their customers only have access to voice roaming, no data for several large stretches of I-10.