https://netflixtechblog.com/rapid-event-notification-system-at-netflix-6deb1d2b57d1

0*i6o12AqlvqZDMcAj

Netflix has more than 220 million active members who perform a variety of actions throughout each session, ranging from renaming a profile to watching a title. Reacting to these actions in near real-time to keep the experience consistent across devices is critical for ensuring an optimal member experience. This is not an easy task, considering the wide variety of supported devices and the sheer volume of actions our members perform. To this end, we developed a Rapid Event Notification System (RENO) to support use cases that require server initiated communication with devices in a scalable and extensible manner.

In this blog post, we will give an overview of the Rapid Event Notification System at Netflix and share some of the learnings we gained along the way.

With the rapid growth in Netflix member base and the increasing complexity of our systems, our architecture has evolved into an asynchronous one that enables both online and offline computation. Providing a seamless and consistent Netflix experience across various platforms (iOS, Android, smart TVs, Roku, Amazon FireStick, web browser) and various device types (mobile phones, tablets, televisions, computers, set top boxes) requires more than the traditional request-response model. Over time, we’ve seen an increase in use cases where backend systems need to initiate communication with devices to notify them of member-driven changes or experience updates quickly and consistently.

• Viewing ActivityWhen a member begins to watch a show, their “Continue Watching” list should be updated across all of their devices to reflect that viewing.
• Personalized Experience RefreshNetflix Recommendation engine continuously refreshes recommendations for every member. The updated recommendations need to be delivered to the device timely for an optimal member experience.
• Membership Plan ChangesMembers often change their plan types, leading to a change in their experience that must be immediately reflected across all of their devices.
• Member “My List” UpdatesWhen members update their “My List” by adding or removing titles, the changes should be reflected across all of their devices.
• Member Profile ChangesWhen members update their account settings like add/delete/rename profiles or change their preferred maturity level for content, these updates must be reflected across all of their devices.
• System Diagnostic SignalsIn special scenarios, we need to send diagnostic signals to the Netflix app on devices to help troubleshoot problems and enable tracing capabilities.

In designing the system, we made a few key decisions that helped shape the architecture of RENO:

1. Single Events Source
2. Event Prioritization
3. Hybrid Communication Model
4. Targeted Delivery
5. Managing High RPS

Single Events Source

The use cases we wanted to support originate from various internal systems and member actions, so we needed to listen for events from several different microservices. At Netflix, our near-real-time event flow is managed by an internal distributed computation framework called Manhattan (you can learn more about it here). We leveraged Manhattan’s event management framework to create a level of indirection serving as the single source of events for RENO.