Defer and Stream Directives in GraphQL Improve Latency with - PowerPoint PPT Presentation

Defer and Stream Directives in GraphQL Improve Latency with Incremental Delivery Liliana Matos, Director, Front-End Engineering @ 1stdibs Rob Richard, Senior Director, Front-End Engineering @ 1stdibs

About • Online marketplace for luxury items across multiple verticals • Front-End stack: Node, GraphQL, React, and Relay • O ffi ces in New York, Vilnius, Lithuania, and Bangalore, India

What are @defer and @stream? • @defer and @stream are proposed directives to the query TalksQuery { GraphQL Specification to support incremental delivery for talks(first: 6) @stream state-less queries ( label: "talkStream", • Championing since January 2020 initialCount: 3 ) • In collaboration with GraphQL Working Group { name • In this talk, we will discuss: ... TalkComments @defer(label: "talkCommentsDefer") • Motivation } } • Specification proposal overview • Code Examples fragment TalkComments on Talk { comments { • Reference implementation in GraphQL.js body } • Open-source contribution } • Best practices

Why @defer and @stream? • Large datasets may su ff er from latency • All requested data may not be of equal importance • Current options for applications to prioritize data, such as query splitting and pre-fetching , come with undesirable trade-o ff s • @defer and @stream would allow GraphQL clients to communicate priority of requested data to the server without undesirable trade-o ff s

Query Splitting /** Original Query */ query SpeakerQuery($speakerId: String!) { speaker(speakerId: $speakerId) { name ...SpeakerPicture } } • Fetch expensive/non-essential fields in a fragment SpeakerPicture on Speaker { separate query after initial query picture { height width 
 • Trade-o ff s: url } } • Increased latency for lower priority /** Split Queries */ query SpeakerInitialQuery($speakerId: String!) { fields speaker(speakerId: $speakerId) { name } • Client resource contention } query SpeakerFollowUpQuery($speakerId: String!) { • Increased server cost speaker(speakerId: $speakerId) { ...SpeakerPicture } }

Pre-fetching • Optimistically fetching data based on a prediction that a user will execute an action • Trade-o ff s: • Increased server cost due to incorrect predictions

https://www.apollographql.com/blog/introducing-defer-in-apollo-server-f6797c4e9d6e/

What about Subscriptions? • Intention is for real-time and long connections • @defer and @stream , intention is to lower latency for short-lived connections

Specification Proposal for @defer and @stream

@defer directive @defer(if: Boolean, label: String) on FRAGMENT_SPREAD • The @defer directive may be specified on a fragment spread. • if: Boolean • When true fragment may be deferred, if omitted defaults to true . • label: String • A unique label across all @defer and @stream directives in an operation.

// Response Payloads @defer // Payload 1 Example { "data": { "speaker": { "name": "Jesse Rosenberger" query SpeakerQuery($speakerId: String!) { } speaker(speakerId: $speakerId) { }, name "hasNext": true ...SpeakerPicture @defer(label: “speakerPictureDefer”) } } } // Payload 2 { fragment SpeakerPicture on Speaker { picture { "label": "speakerPictureDefer", height "path": ["speaker"], width 
 "data": { url "picture": { } "height": 200, } "width": 200, "url": "jesse-headshot.jpg" } }, "hasNext": false }

@stream directive @stream(if: Boolean, label: String, initialCount: Int) on FIELD • The @stream directive may be provided for a field of List • if: Boolean • When true fragment may be deferred, if omitted defaults to true . • label: String • A unique label across all @defer and @stream directives in an operation. • initialCount: Int • The number of list items the server should return as part of the initial response.

@stream // Response Payloads // Payload 1 { Example "data": { "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true } // Payload 2 { query SpeakersQuery { "label": "speakerStream", speakers(first: 3) @stream(label: “speakerStream", initialCount: 1) { “path": ["speakers", 1], name "data": { "name": "Liliana Matos" } }, } "hasNext": true } // Payload 3 { "label": "speakerStream", "path": ["speakers", 2], "data": { "name": "Rob Richard" }, "hasNext": false }

Response Format Overview • When an operation contains @defer or @stream directives, the GraphQL execution will return multiple payloads • The first payload is the same shape as a standard GraphQL response • Any fields that were only requested on a fragment that is deferred will not be present in this payload • Any list fields that are streamed will only contain the initial list items

Response Format Details • label — The string that was passed to the label argument of the @defer or @stream directive that corresponds to this results • path — A list of keys from the root of the response to the insertion point • hasNext — A boolean that is present and true when there are more payloads that will be sent for this operation. • data — The data that is being delivered incrementally. • errors — An array of errors that occurred while executing deferred or streamed selection set • extensions — For implementors to extend the protocol

// Response Payloads Response Format // Payload 1 { "data": { Example "speakers": [ { "name": "Jesse Rosenberger" } ] }, "hasNext": true } // Payload 2 { "label": "SpeakerPictureDefer", query SpeakersQuery { "path": ["speakers", 0, "picture"], speakers(first: 2) @stream(label: "speakerStream", initialCount: 1) { "data": { name "url": "jesse-headshot.jpg" ...SpeakerPicture @defer(label: "speakerPictureDefer") }, "hasNext": true } } } // Payload 3 { "label": "SpeakerStream", "path": ["speakers", 1], fragment SpeakerPicture on Speaker { "data": { picture { "name": "Liliana Matos" url }, "hasNext": true } } } // Payload 4 { "label": "SpeakerPictureDefer", "path": ["speakers", 1, "picture"], "data": { "url": "liliana-headshot.jpg" }, "hasNext": false }

How to use @defer and @stream in your GraphQL Server • @defer - existing resolvers will work e ff ectively • @stream - need to consider how resolvers return data

Execution { "data": { query SpeakerQuery($speakerId: String!) { "speaker": { speaker(speakerId: $speakerId) { Initial Payload "name": "Jesse Rosenberger" name } ...SpeakerPicture @defer(label: “speakerPictureDefer”) }, } "hasNext": true } } Fork execution to dispatcher { "label": "speakerPictureDefer", fragment SpeakerPicture on Speaker { "path": ["speaker"], Subsequent Payload picture { "data": { height "picture": { width 
 "height": 200, url "width": 200, } "url": "jesse-headshot.jpg" } } }, "hasNext": false }

How to use @stream in your GraphQL Server • Any List field can use the @stream directive • What you return from your resolver matters

List return types in GraphQL-JS • GraphQL-JS supports returning several di ff erent data types in List resolvers. • Array<T>, any Iterable, Promise<Iterable<T>> • GraphQL engine will get all results at once • Initial payload will be held up by this resolver • Subsequent payloads will be sent immediately after const resolvers = { Query: { items: async function (_, { filters }): Promise<Array<Item>> { const items = await api.getFilteredItems({ filters }); return items; }, }, };

List return types in GraphQL-JS • Array<Promise<T>> • GraphQL engine will start waiting for all results • Initial payload will be sent as soon as the "initialCount" values are ready • Subsequent payloads will be sent as each promise resolves • Requires knowing how many results there will be before the resolver returns

Returning Array<Promise<T>> const resolvers = { Query: { items: async function (_, { filters }): Array<Promise<<Item>> { const itemIds = await api.filterItems({ filters }); return itemIds.map(async itemId => await api.getItemById(itemId)); }, }, };

List return types in GraphQL-JS • AsyncIterable, Async Generator function • GraphQL engine will yield each result from the iterable • Initial payload will be sent as soon as the "initialCount" values are ready • Subsequent payloads will be sent as each new value is yielded • Can determine asynchronously if the list is completed