© 2018 Hulu, LLC | all rights reserved
Hulu, LLC | 2500 Broadway, 2nd Floor | Santa Monica, CA 90404 | (310) 571-4700 | hulu.com
1.5 | last updated December 9, 2020
Annually
No part of this publication may be reproduced, transmitted, transcribed, stored in a retrieval system, translated into any other language in any form or by any means, electronic or mechanical, including photocopying or recording, for any purpose, without the express permission of Hulu, LLC.
This document is provided for informational purposes. The information is provided “as is” and without warranties of any kind. Due to continuing product development, the information is subject to change without notice. Hulu, LLC shall not be responsible or liable for any inaccuracies, errors, or outdated information.
This document is intended for advertisers. It provides a summary of the video-streaming measurement processes employed by Hulu, LLC (Hulu). It includes a general description of Hulu’s measurement methodology, filtration processes and reporting procedures. Issuance of this document is in accordance with Hulu’s accreditation conferred by the Media Rating Council (MRC).
Hulu provides an Ad-supported streaming digital video service that offers a selection of TV shows, clips and movies via its video players at hulu.com, embedded Hulu video players hosted at other internet sites, mobile devices via the Hulu mobile application and over-the-top (OTT) internet-connected devices via device-specific Hulu applications. Users of Hulu’s digital video service may elect to view content on hulu.com and through external sites such as yahoo.com. To view Hulu content on web, or any Mobile or OTT internet-connected device, Hulu users must subscribe to the Hulu service. Video Ads are served within the Hulu player at pre- and mid-roll positions. Companion display Ads may also be shown within these placements, or adjacent to the player, in close proximity. Hulu partners with Content Providers (CPs) and distributors to provide digital video content to users. Hulu also produces certain original video content available through its streaming video service. Regardless of the origin of the content, Hulu maintains control of its streaming digital video players and content pipelines, and encodes all video content and advertisements prior to distribution within Hulu’s proprietary video players on Web and within Hulu applications. Access to Hulu content is limited to users within the United States, including US territories and military bases overseas, based on IP-address geolocation as determined by a third-party vendor; other regions are blocked. In most situations, Hulu’s first-party Ad server performs Ad decisioning. However, Hulu also supports third-party Ad serving in certain situations involving pre-established relationships with specified content providers.
The following Hulu metrics are included in MRC-accreditation scope, on the Web, Mobile Web, and Mobile In-App platforms:
The following Hulu reports are included in MRC-accreditation scope:
The following items are excluded from MRC-accreditation scope:
All Hulu beacons originate directly from the client. No server-side beacons are used. No proxies are intentionally used.
Hulu does not use sampling. Measurements are counted directly from client-initiated beacons, after filtration processes mentioned below.
Clients send Impressions and Completions to geographically-dispersed CDNs provided by Akamai. Akamai’s CDNs then forward these beacons to Hulu’s measurement servers.
Client initiate Impressions and Completions by firing a beacon via HTTP GET request. The measurement server returns a 200 to indicate successful reception of the beacon, and writes a line in its log file to record the Impression or Completion. The above is true of Impressions and Completions for all video, rich-media, and display Ads.
Network latency may result in beacons not arriving at the server around the moment an Impression or Completion occurs, but rather at some point afterward. If this is case, the Impression or Completion is counted upon receipt. Network issues may also cause beacons being duplicated somewhere along the way (e.g., by a third-party proxy server) between Hulu’s client and server. Hulu addresses this risk in two ways. First, by never performing beacon retries on any accreditation platform (i.e., Web, Mobile Web, or Mobile In-App). Second, by performing beacon de-duplication on the server side.
Hulu does not compound or grouped beacons. Measurements are counted directly from client-initiated beacons one at a time.
Hulu uses a combination of beacon parameters, including a client-generated auto-incrementing “seq” (i.e., sequence) value, to create uniqueness of Impressions and Completions. Also, in the 200 response to Impression and Completion beacons, the server uses the following Header Controls: Cache-Control: max-age=0, no-cache, no-store; Expires; Pragma: no-cache. These values instruct the client to not cache the GET request, and always fire a new instance of the beacon.
Hulu does not engage in auto-play of Video or Interactive Ads. Playback is never initiated without an explicit user action (i.e., click or tap). Hulu engages in Continuous Play, which is automatic advancement to a new piece of Content after an existing piece of Content comes to an end. Hulu uses a threshold to determine when to show an “Are you still watching?” confirmation. After four hours of continuous play without user interaction, the dialog is presented, and if no confirmation (i.e.,‘Yes’) is received from the viewer, further playback is stopped. Based on a preliminary study, ~18% of Impressions on Web was generated by sessions that included 1+ hour inactivity. ~1% of Impressions on Mobile In-App was generated by sessions that included 1+ hour of inactivity. Hulu is developing and conducting improved studies on viewer inactivity in order to update its continuous-play functionality.
Impression and Completion beacons received from Akamai are immediately written into log files. The log files contain raw beacon data. They undergo processing before they can be consumed by reporting mechanisms. Daily processing allows Hulu’s reports to include the previous day’s data, but not the current day’s.
Hulu’s Mobile application is available to all mobile devices on the platforms mentioned below in the same geographical region as its Web and Living Room offerings. Namely, the U.S. and its territories.
Hulu’s Mobile application runs on iOS and Android.
The use of Hulu’s Mobile application is possible only when the user has proactively downloaded and installed the application. Ad delivery and measurement techniques across application versions have not changed meaningfully in the past several years.
The use of Hulu’s Mobile application occurs only when the user has proactively opened and initialized the application after downloading and installing it. Multiple versions of the same application cannot coexist.
The use of Hulu’s Mobile application occurs only when the application is in focus. Playback automatically stops when the application loses focus. This means that no Impression or Completion beacons can be fired while the application is background’ed. Furthermore, during situations when the device goes offline, playback will stop due to the application’s video buffer emptying. At this point, no Impression or Completion beacons can be fired as playback of both content and ads is halted.
While generally no latency issues are particular to OTT devices, there is one exception in that Roku devices have been noted to experience network issues. Further detail is provided in the SSL section below.
Roku devices have been noted to experience network issues. On these devices, it has been observed that having a large queue of beacons to be fired can sometimes result in some queued beacons simply not firing and eventually being discarded inappropriately. Also high volumes of network activity create latency issues and generally slow processing that are particular to these devices. As a result, as of September 2018, SSL protocol is not available on Roku devices.
Hulu uses a threshold to determine when to show an “Are you still watching?” confirmation. After three hours of Continuous Play without user interaction, the dialog is presented, and if no confirmation (i.e., ‘Yes’) is received from the viewer, further playback is stopped. Studies are performed on an annual basis, to understand viewer patterns and inform decisions on whether adjustments to improve experience are necessary. Continuous Play is used more frequently on OTT, as compared to Web and Mobile. As a result, Hulu’s studies on Continuous Play are conducted on a subset of OTT devices.
Detection of TV-off state, and accordingly pausing playback, is entirely dependent on the capabilities of hardware. Where detection is possible, Hulu apps pause playback. However, there is a population of devices that do not communicate the change of hardware state to higher levels in the tech stack. In these cases, Hulu apps are unable pause playback, although Continuous Play thresholds apply as normal. Hulu has conducted research to gain a general understanding of the aforementioned hardware capabilities and limitations. A study was performed on a broad sample of devices running Hulu’s OTT app. This sample included devices from manufacturers Amazon, Apple, Google, LG, Microsoft, Panasonic, Roku, Samsung, Sony, Tivo, and Vizio. These devices running Hulu were paired with a variety of TVs from manufacturers that included LG, Panasonic, Phillips, Samsung, Sharp, Sony, Toshiba, and Vizio. The study was also ran with and without an AV receiver between the Hulu device and the TV.
A summary of study results is as follows:
In conclusion, there are many permutations of hardware, settings, and use cases to consider. Improving detection of TV-off is a complex industry issue that will require consistent commitment from TV manufacturers.
Please see the Continuous Play section above.
The methodology for measuring clicks is identical to that of Impression and Completion measurement. A client-side beacon is fired whenever the user performs a click action within Hulu’s application. On the server side, this beacon is logged and undergoes processing, eventually being included in reporting.
Ad-blocking software is capable of preventing Hulu’s Ads from being displayed. On the Web platform, browser settings that disable images, JavaScript, HTML5 elements, and / or Flash are capable of preventing Hulu’s Ads from being displayed. When these situations occur, because no Ads are loaded, no Impression or Completions are fired. Ad stitching (i.e., using a single manifest with pre-set content and ads for each stream) is performed on the server side, with the result being delivered to Mobile and Living Room client devices to facilitate smooth playback.
Hulu employs a suite of techniques in an attempt to identify and filter invalid activity. These techniques include but are not limited to known and suspected non-human activity and suspected invalid human activity. Because user identification and intent cannot always be known or discerned by the publisher, advertiser or their respective agents, it is unlikely that all invalid activity can be identified and excluded from report results. Below, Hulu’s various techniques are summarized strictly at a high-level for security purposes.
Hulu practices GIVT. Sophisticated Invalid-Traffic Filtration (SIVT) is not performed.
Hulu performs its own filtration without the assistance of any third parties.
Hulu removes all internally-generated traffic.
Based on User Agent, Hulu performs the IAB dual-pass method to remove all traffic that doesn’t appear on the international lists of spiders and bots, and valid browsers.
Hulu employs ABF to identify anomalies. The specifics of its ABF techniques are kept confidential for security purposes. They are disclosed only to MRC auditors as part of the audit process.
Hulu does not employ any pre-fetching mechanisms. Impression and Completion beacons are fired in real time. This also applies to ads shown as a result of Continuous Play.
Filtration is applied against Hulu’s entire traffic across the Web, Mobile Web, and Mobile In-App platforms. No subsets or sampling techniques are used.
When Impression and Completion beacons are received by the server, they undergo validation to ensure critical parameters aren’t missing. They are also evaluated to see whether parameter values are contradictory. Any beacons that fail either type of validation are discarded ahead of any further processing. In the unlikely event Hulu’s log files are corrupted, Akamai is capable of resending beacons for Hulu to reprocess.
An analysis of IVT during Q4 of 2016 shows that IVT typically fluctuates at immaterial levels.
When significant changes to Hulu’s filtration methods and / or technology are implemented, Hulu issues a statement to clients with active or upcoming Ad Campaigns. This is reserved for situations where notification provides a discernible benefit to clients. For security purposes, Hulu discloses as little as possible when it comes to its filtration techniques. Disclosures will be accordingly high-level.
Hulu reports a variety of metrics to clients on a per-Campaign basis.
The reports in accreditation scope – Post-Buy and Device – are presented to clients at multiple points in the Campaign, depending on Campaign length and client preference. The Post-Buy is always presented after a Campaign ends.
Estimation is not used. Report metrics are composed solely from actual data collected in the measurement process.
While report data accumulates on a day-by-day basis, it is considered final as, once processed, a day’s data is official and generally unchanging. Data revisions are possible but rare.
These are Impressions and Completions, and their derivatives – Completion Rate and Delivered Percent.
Impressions and Completions are segregated in three ways:
Non-accredited metrics are also presented. These include, but are not limited to: quartiles, Clicks, Clickable Impressions and Completions, Click-Through Rate (CTR), Time Spent Viewing, and Unique Users.
Open columns with no metrics or data can also be presented. They are meant to reserve space conveniently for manual data entry. These include Estimated Spend values, Third Party and Discrepancy.
Reports can be:
Per Hulu’s product options, static companion ads like banners and logo are often displayed alongside video and rich-media ads. These companion ads are segregated in reporting as distinct products.
Campaigns parameters are entered by Hulu’s Campaign Coordinators. Before being set live, every Campaign is subject to full review by a Campaign Manager.
After going live, Campaigns undergo pacing monitoring by Campaign Managers to ensure they’re delivering as expected. Optimizations and debugging are performed if necessary.
Revised metrics, including Impressions and Completions, will be communicated when excluded or invalid activity is detected for a Campaign after initial reporting and within three days after the end of the Campaign. The methods of communication include e-mail statement and / or updated reporting.
Campaigns billed against Hulu metrics are never billed for IVT traffic. In other words, Ad Campaign targets are always met after discounting any traffic identified as IVT. However, as mentioned above, clients are always notified of gross vs. net numbers. After reviewing these, clients may contact their Hulu Campaign Manager to discuss any general questions around IVT volume and classification.
Source data for Hulu’s reports is retained indefinitely without expiry. Records of data reissuance are also retained indefinitely.
At the time of this writing, Hulu maintains an important relationship with Akamai.
Akamai provides CDN services, playing a pivotal role in beacon gathering, and consequently has a material impact on Hulu ad delivery. Other business relationships exist but are not listed due to having no influence on Hulu’s ad delivery mechanisms. Per MRC regulations, Hulu has worked with Akamai to complete a qualification process whereby Hulu ascertains Akamai’s stance and progress against anti-IVT measures. While Akamai relies on its clients to make their own IVT policy decisions, Akamai offers a suite of tools to counteract IVT. This suite includes Bot Manager, which allows clients to configure appropriate responses to requests from bots, and use white and black lists. Another tool offered is Kona Site Defender, which provides rate control and analytics, as well as IVT-behavior detection. Akamai has also granted Hulu access to its SOC 2 security-based reportA
Q: What does native frame rate mean?
A: Native frame rate refers to the frame rate the source footage was shot. Whenever possible, we require all videos to be delivered in their native frame rate. This means that no frame rate conversion should be performed, which includes adding 3:2 pulldown for broadcast.
Stress mark should be marked with [capitals] to indicate the primary stressed syllable, as in: news・pa・per [NOOZ-pey-per] in・for・ma・tion [in-fer-MEY-shuhn]
On living room, mobile, and tablet devices, the color gradient overlay is dynamic and will change based on the cover story art. It is not something we can control on our end.
If the tagline/date messaging doesn’t fit within the 11 syllables max, it can be included as text.
On living room, mobile, and tablet devices, the color gradient overlay is dynamic and will change based on the cover story art. It is not something we can control on our end.
Q: When is letterboxing allowed and not allowed?
A: When the native aspect ratio is 1.78:1 or 1.33:1 throughout the entire program, there should not be any letterboxing (black bars on top and bottom), nor should there be any pillarboxing (black bars on either side). We should see an active picture take up the full frame. If the aspect ratio is wider than 1.78:1, such as 2.35:1, matting on the top and bottom is permissible. Additionally, if there is a creative choice to add matting or if there is a mix of native aspect ratios, this is usually waived, but please reach out to your Hulu representative to confirm.
Q: Should the bitrate be constant if delivering ProRes codec?
A: No, since ProRes codec is built to be variable, this is waived.
Q: Can you accept bitrate higher than 30 Mbps?
A: Yes, we can accept bitrate beyond the recommended range for H.264 and ProRes. In the case of ProRes, bitrate will often exceed 30 Mbps due to its variable setting.
Q: Why do you ask for progressive?
A: The Hulu player, unlike traditional broadcast, does not play back interlace scan, so we require that all videos be delivered with their scan type set to progressive. If your video is natively interlaced, you must de-interlace it to progressive and you must employ a de-interlace filter that does not result in blending or ghosting artifacts. We recommend an auto-adaptive de-interlace if available.
Q: Why do you prefer PCM codec?
A: PCM codec is lossless audio quality, so whenever possible, please deliver PCM audio.
Gadi: The development of serendipity in recommendations is very important. An analogy I use is that of a restaurant. If you go to a good restaurant, you don’t always want to see what you’re looking for. You’re looking for surprises. It’s not the expected, it’s the unexpected. You trust in the restaurant’s atmosphere. You don’t know exactly what you’re getting, but you trust their creativity and that you’ll enjoy whatever they serve you.
Do you think it is possible for a streaming service to become that “restaurant” - a trusted source of serendipitous recommendations?
Gadi: I absolutely do. But it is difficult. It’s a long game. It requires them to build trustworthiness with audiences through genuine content recommendations over the years.
Gandi: The development of serendipity in recommendations is very important. An analogy I use is that of a restaurant. If you go to a good restaurant, you don’t always want to see what you’re looking for. You’re looking for surprises. It’s not the expected, it’s the unexpected. You trust in the restaurant’s atmosphere. You don’t know exactly what you’re getting, but you trust their creativity and that you’ll enjoy whatever they serve you.
Do you think it is possible for a streaming service to become that “restaurant” -- a trusted source of serendipitous recommendations?
Gandi: I absolutely do. But it is difficult. It’s a long game. It requires them to build trustworthiness with audiences through genuine content recommendations over the years.
Another expectation among younger generations seems to be personalization. How are you seeing this play out in media?
Jonathan: If you go into the world of advertising and marketing, customized, personalized advertising is more important than ever before. There’s a realization that eight years of funny commercials that we’ve showed everybody probably for the fourth time, doesn’t work anymore. So there’s a lot of companies moving towards specialized advertising.
How does this type of personalization translate to personalizing content beyond advertising? Who’s going to predict what people will want to watch, and do it first?
Jonathan: It’s not about being the first to predict what people want to watch. It's different. It’s about getting viewers to browse. You want to show them the value of all of the money Hulu has spent and the great range of TV and film for them to choose from.
Julie: There are vast differences between the way under 35-year-olds watch TV and over 35-year-olds watch TV. I grew up in a world where there were maybe 10 channels, and my viewing changed as technology and options changed. Younger viewers started from a really different place. Everything has always been on-demand. Anything they ever wanted to see was available to them, and they therefore have different expectations for TV.
How so?
Julie: One of the things we found with Gen Z is that they really want to be immersed in something for a long time. They want to have content that they can live with for a while; it’s like this long, seamless storytelling. They’ll knock off a couple episodes a night and it will last a couple of months. And then they’ll re-watch it a million times over.
What other shifts have you seen happen-with Gen Z but also more broadly-with the rise of streaming?
Julie: For years, television had to deliver a specific rating. Shows had to appeal broadly or else they wouldn’t survive. And those days are long gone because, with streaming services, shows can reach hundreds of thousands of people or tens of thousands of people and still be considered successful. There’s more experimentation with the types of content; we’re not locked into an hour, a half hour, a comedy, etc. The industry can create really niche shows to appeal to niche audiences, but also simultaneously create big, broad experiences that are shared by millions.
Your work focuses a lot on Gen Zs who, for better or worse, are dubbed “digital natives.” How do you think a generation of digitally native audiences view digital content differently than older generations?
Larissa: I think for young people digital content is a way they’re able to explore their own identities through the story... They want to see themselves and their stories in the content that they’re engaging with.
Tell us a little more about this digital content as Gen Z’s form of self-reflection.
Larissa: Digital content is sort of like a currency. I find that young people want to watch things that their friends are watching so that they can have conversations about it. For example, with Euphoria, young people were just kind of in love with the characters. It was very timely and a bit provocative, and then there was a way that they could see themselves in these stories and connect with their friends about the topics and ideas in the show.
And then also they could almost embody these characters in their own life. I really do think that the TV shows that young people are buying into are actually influencing their culture and their trends and even their language that they’re using.
What does the future of personalization look like?
Richard: I think it’s all down to trust. We're going to see more opportunity on platforms like Hulu and Spotify where the user trusts us.
That’s really interesting. Another area we wanted to explore is podcasts, and their relationship to video. For example, the show Homecoming is an adaptation of a podcast; the podcast Office Ladies is a spin-off from a TV show. Why do you think the two formats work so well together?
Richard: Anything at all that drives conversation in pop culture, and TV does a lot of that, is worthy of consideration in a podcast environment. Any of these conversations can become multiple audio streams that evolve with experts, interviews, and all kinds of narrative threads that can flesh out characters, or narrative development, or whatever's happening in those shows.
The use of Graphik is acceptable in cases where the Client cannot supply their own typeface.
