how we know where you are in house of cards
play

How We Know Where You Are in House of Cards @zimmermatt Netflix - PowerPoint PPT Presentation

Netflixs Viewing Data Microservices How We Know Where You Are in House of Cards @zimmermatt Netflix Scale @zimmermatt Netflix Scale 62,000,000 @zimmermatt Netflix Scale 62,000,000 50+ @zimmermatt Netflix Scale 62,000,000 50+


  1. Netflix’s Viewing Data Microservices How We Know Where You Are in House of Cards @zimmermatt

  2. Netflix Scale @zimmermatt

  3. Netflix Scale 62,000,000 @zimmermatt

  4. Netflix Scale 62,000,000 50+ @zimmermatt

  5. Netflix Scale 62,000,000 50+ 3,000,000,000+ @zimmermatt

  6. Netflix Scale 62,000,000 50+ 3,000,000,000+ 1000+ @zimmermatt

  7. Netflix Scale 62,000,000 50+ 3,000,000,000+ 1000+ 37% @zimmermatt

  8. Viewing Data @zimmermatt

  9. Viewing Data: History Who @zimmermatt

  10. Viewing Data: History Who, What @zimmermatt

  11. Viewing Data: History Who, What, When 6/10/2015 11:55 @zimmermatt

  12. Viewing Data: History Who, What, When, Where 6/10/2015 11:55 @zimmermatt

  13. Viewing Data: History Who, What, When, Where, How Long 6/10/2015 11:55 @zimmermatt

  14. Viewing Data: Telemetry @zimmermatt

  15. What have I watched? @zimmermatt

  16. What have I watched? @zimmermatt

  17. Where was I at? @zimmermatt

  18. What else am I watching? @zimmermatt

  19. What Happened? Video Bitrate Usage @zimmermatt

  20. Viewing Data Requests @zimmermatt

  21. Viewing Data Requests @zimmermatt

  22. Viewing Data Requests @zimmermatt

  23. Viewing Data Requests @zimmermatt

  24. Viewing Data Requests @zimmermatt

  25. Gen 3 (Existing) System Stateful Tier Events 0 Viewing Service 1 Active Sessions Summarized … Sessions Latest Positions n-2 n-1 History Cache Session Summary Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  26. Gen 3 System’s Architecture & Scale Logs @zimmermatt

  27. Gen 3 System’s Architecture & Scale Logs Viewing History @zimmermatt

  28. Gen 3 System’s Architecture & Scale History Cache Logs Viewing History @zimmermatt

  29. Gen 3 System’s Architecture & Scale Summarized Sessions History Cache Logs Viewing History @zimmermatt

  30. Gen 3 System’s Architecture & Scale Stateful Tier Viewing Service Summarized … Sessions History Cache Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  31. Gen 3 System’s Architecture & Scale Stateful Tier Events 0 Viewing Service 1 Active Sessions Summarized … Sessions Latest Positions n-2 n-1 History Cache Session Summary Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  32. Growth 2015 @zimmermatt

  33. Virtuous Cycle Improved Viewing Personalization Better Experience @zimmermatt

  34. Stateful Instance Count Over Time @zimmermatt

  35. Guiding Lights

  36. Externalize State… Stateful Tier Events 0 Viewing Service 1 Active Sessions Summarized … Sessions Latest Positions n-2 n-1 History Cache Session Summary Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  37. Externalize State… DENIED! Stateful Tier Events 0 Viewing Service 1 Active Sessions Summarized … Sessions Latest Positions n-2 n-1 History Cache Session Summary Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  38. Service Decomposition Video Bitrate Usage @zimmermatt

  39. Service Decomposition Start Stop @zimmermatt

  40. Service Decomposition Start Stop Active Last Viewing Data Sessions Position History Feed @zimmermatt

  41. Service Decomposition Start Stop Event Stream Session Stream State Summary Active Last Viewing Data Sessions Position History Feed @zimmermatt

  42. Service Decomposition Events Collect Process Queries Provide @zimmermatt

  43. Gen 3 Review Stateful Tier Events 0 Viewing Service 1 Active Sessions Summarized … Sessions Latest Positions n-2 n-1 History Cache Session Summary Logs Stateless Tier Viewing History (History, logs, and fallback) @zimmermatt

  44. Gen 4 Generic Architecture Collector Processor Provider @zimmermatt

  45. Gen 4 Generic Architecture Collector Processor Provider Queue @zimmermatt

  46. Gen 4 Generic Architecture Collector Processor Provider Queue Raw Data Materialized Views @zimmermatt

  47. Gen 4 Generic Architecture Collector Processor Provider Cache Queue Raw Data Materialized Views @zimmermatt

  48. Gen 4 Architecture Positions Session Events Viewing History Collector Playback History Provider Log Session Accounting Collector Provider Event Data Session Summary Raw Data Processor History Raw Data Logs Active Raw Data Sessions Materialized View Summarized Sessions Materialized View @zimmermatt

  49. Shadow Testing Session Summary Log Session Events Collector Collector Processor Stateful Tier Events 0 Viewing Service Logs Summarized Event Data Active Sessions 1 Sessions … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Positions Viewing History Stateless Tier (History, logs, Active Playback History and fallback) Provider Sessions History Cache History Viewing History Summarized Logs Sessions @zimmermatt

  50. Shadow Testing: Start Play Event Session Events Collector Stateful Tier Events 0 Viewing Service Event Data Active Sessions 1 … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Stateless Tier (History, logs, Active and fallback) Sessions @zimmermatt

  51. Shadow Testing: Start Play Event Session Events Collector Stateful Tier Events 0 Viewing Service Event Data Active Sessions 1 … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Stateless Tier (History, logs, Active and fallback) Sessions @zimmermatt

  52. Shadow Testing: Start Play Event Session Events Collector Stateful Tier Events 0 Viewing Service Event Data Active Sessions 1 … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Stateless Tier (History, logs, Active and fallback) Sessions @zimmermatt

  53. Shadow Testing: Start Play Event Session Events Collector Stateful Tier Events 0 Viewing Service Event Data Active Sessions 1 … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Stateless Tier (History, logs, Active and fallback) Sessions @zimmermatt

  54. Shadow Testing: Start Play Event Session Events Collector Stateful Tier Events 0 Viewing Service Event Data Active Sessions 1 … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Stateless Tier (History, logs, Active and fallback) Sessions @zimmermatt

  55. Shadow Testing: Read Latest Position Stateful Tier Events 0 Viewing Service Active Sessions 1 … Edge Service Latest Positions n-2 n-1 Session Summary Positions Viewing History Stateless Tier (History, logs, Playback History and fallback) Provider History @zimmermatt

  56. Shadow Testing: Read Latest Position Stateful Tier Events 0 Viewing Service Active Sessions 1 … Edge Service Latest Positions n-2 n-1 Session Summary Positions Viewing History Stateless Tier (History, logs, Playback History and fallback) Provider History @zimmermatt

  57. Shadow Testing: Read Latest Position Stateful Tier Events 0 Viewing Service Active Sessions 1 … Edge Service Latest Positions n-2 n-1 Session Summary Positions Viewing History Stateless Tier (History, logs, Playback History and fallback) Provider History @zimmermatt

  58. Shadow Testing: Stop Event Session Summary Session Events Collector Processor Stateful Tier Events 0 Viewing Service Summarized Event Data Active Sessions 1 Sessions … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Positions Viewing History Stateless Tier (History, logs, Active Playback History and fallback) Provider Sessions History Summarized Sessions @zimmermatt

  59. Shadow Testing: Stop Event Session Summary Session Events Collector Processor Stateful Tier Events 0 Viewing Service Summarized Event Data Active Sessions 1 Sessions … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Positions Viewing History Stateless Tier (History, logs, Active Playback History and fallback) Provider Sessions History Summarized Sessions @zimmermatt

  60. Shadow Testing: Stop Event Session Summary Session Events Collector Processor Stateful Tier Events 0 Viewing Service Summarized Event Data Active Sessions 1 Sessions … Edge Service Latest Positions n-2 Session Accounting n-1 Provider Session Summary Positions Viewing History Stateless Tier (History, logs, Active Playback History and fallback) Provider Sessions History Summarized Sessions @zimmermatt

Recommend


More recommend