Live TV streaming metrics
On-demand streaming metrics
How we scored the best live TV streaming services
For live TV streaming services, we focused heavily on the quality of each service’s channel lineups. While each service boasts about the raw number of channels you’ll get, not all channels are created equal, and we didn’t want to give the same weight to Cheddar News as we did to CNN.
Enter TiVo’s Video Trends Report. The report comes out four times a year, and (among other things) asks 3,330 respondents, “Which channels would you be interested in including in your TV package?”
All in all, 112 channels are included in the list. We then ordered these from most desired to least, and assigned values to each channel on the list so that the most popular channel (ABC) was “worth” three times as much as the least popular one (YTV). Once each channel had a score, we simply checked which channels were included in each streaming package and added up the scores. We then standardized them to a 10-point scale for our “channel lineup” scores.
When services had more than one plan available, we averaged the scores for each of their plans. Here’s how these channel lineup scores compare for each live TV streaming service:
|Service||Channel lineup score|
|Hulu + Live TV||7.5|
|AT&T TV NOW||7.1|
|AT&T Watch TV||5.4|
*As of 01/17/20
For our value scores, we simply divided the raw channel score for each service by its price. This gave us a good idea of how much you’d be paying for each channel. We also standardized these scores on a 10-point scale. Like we did with the channel lineup number, if a service had more than one plan, we averaged the scores for each plan. Here’s how each service scored for value:
|AT&T Watch TV||9.5|
|Hulu + Live TV||6.0|
|AT&T TV NOW||4.6|
*As of 01/17/20
There was one small exception here. Because Hulu + Live TV includes Hulu’s popular on-demand streaming service, we subtracted the price of a subscription ($5.99/mo.) from its base price ($54.99/mo.). That raised its value score from 5.6 to 6.0.
With this score, we tried to quantify the quality of supplemental perks that each service offers. We focused on hours of cloud DVR storage included in each service’s base plan, how long recordings store for and the number of simultaneous streams allowed. We also considered any sign-up bonuses here, like Sling’s offer of a free Amazon Fire TV Stick when you prepay for two months of service.
With unlimited cloud DVR storage that stores for nine months and three simultaneous streams, YouTube TV received our highest score in this section with 9.5.
This is the only section that was truly objective. We tested each service out for about five hours each on four devices: a Roku Streaming Stick, Apple TV, Amazon Fire TV Stick and MacBook Pro. We looked for things like an easy-to-navigate interface, helpful recommendations and buffer-free streams.
To get our overall score, we weighted each section according to their importance. Our channel lineup and value scores each accounted for 35%, while extra perks and user experience were assigned 15% each.
How up-to-date are Allconnect’s live TV streaming service metrics?
We do an official audit of our live TV streaming metrics on a monthly basis, but if any major changes occur outside of that schedule, we update them immediately. For instance, AT&T TV NOW increased its DVR storage from 50 hours to 500 hours in January 2020. That instantly raised its “extra perks” scores from 5.3 to 8.7 and its overall score from 5.3 to 6.3.
How we scored the best on-demand streaming services
Movie and TV scores
To evaluate each streaming service’s movie collection, we factored in both quality and quantity. For the sheer raw numbers of movies available, we utilized data from JustWatch.com, which tracks the titles that are on every streaming service.
Like our live TV streaming scores, we awarded more points to the titles at the top of the list than the ones at the bottom. In the top 250 movies, for example, The Shawshank Redemption (number one on IMDb’s list) was worth about three times more than Guardians of the Galaxy (number 250). The same process was repeated for TV shows.
Finally, we added up the scores for each service and standardized them on a 10-point scale. Here’s how the major streaming services stacked up for movies and TV shows:
|Movie score||TV score|
|CBS All Access||1.6||1.3|
*As of 01/17/20
Netflix came in first with 33 movies and 69 TV shows on the list, while Apple TV+ came in last place with zero on movies and TV shows in IMDb’s top 250.
To evaluate each service’s original content, we focused on three metrics: critical reception, audience ratings and awards recognition. For each streaming service, we averaged user ratings from IMDb and Rotten Tomatoes for every original show and movie, provided each title had at least 100 ratings.
To gauge critical reception, we used Rotten Tomatoes’ Tomatometer® score. The metric scores TV seasons and movies once at least five professional critics have weighed in. If an original title had enough critic reviews for a score, we included it.
Finally, we counted the number of Emmys and Academy Awards that each service has garnered since 2013, the first year a streaming service was recognized (Netflix was nominated for four Emmys for House of Cards that year and once for Arrested Development.)
|Emmy Nominees (2013-2019)||Emmy Wins (2013-2019)||Oscar Nominees (2014-2020)||Oscar Wins (2014-2020)|
These three scores were then averaged for an overall original content rating. You can see more details about our findings in our full analysis here.
Like our user experience rating for live TV streaming services, we spent about five hours with each service (and often much more) to determine how user-friendly it is. We alternated between a Roku Streaming Stick, Apple TV, Amazon Fire TV Stick and MacBook Pro to grade how the services fared across devices. We looked for things like easy search and navigation, helpful recommendations and uninterrupted streams.
Unlike our overall score for live TV streaming services, we weighed each category equally for on-demand streaming services.
How up-to-date are Allconnect’s on-demand streaming service scores?
Our TV and movie scores are updated every three months, while our original content ratings are typically updated every six months. That said, we also revise the original content scores when Emmy and Academy Award nominations are announced, and again after the awards ceremonies take place.
Our streaming reviews
To see our TV streaming metrics in action, check out all of our reviews below:
Live TV streaming
- AT&T TV review
- YouTube TV review
- fuboTV review
- Philo review
- Hulu + Live TV review
- AT&T TV NOW review
- AT&T Watch TV review
- Sling TV review