Skip to main content

Paging

nestjs-query supports multiple paging strategies that each have their own pros and cons. This documentation will cover the different paging strategies and their applicable use cases.

The following examples are based on the following TodoItemDTO

todo-item.dto.ts
import { FilterableField, IDField } from '@nestjs-query/query-graphql';import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';
@ObjectType('TodoItem')export class TodoItemDTO {  @IDField(() => ID)  id!: string;
  @FilterableField()  title!: string;
  @FilterableField()  completed!: boolean;
  @FilterableField(() => GraphQLISODateTime)  created!: Date;
  @FilterableField(() => GraphQLISODateTime)  updated!: Date;}

Cursor Based Paging#

By default nestjs-query will expose all query many endpoints as cursor based [connections](https://relay .dev/graphql/connections.htm) that you can use to page through results.

note

When using cursor based connections you are not tied to any particular implementation described below, because of the opaque nature of cursors you can start out with the default and switch to key set based cursors later on without changing your clients.

All cursor connections, regardless of paging strategy, expose the following schema

scalar ConnectionCursor
type PageInfo {  hasNextPage: Boolean  hasPreviousPage: Boolean  startCursor: ConnectionCursor  endCursor: ConnectionCursor}
type TodoItemConnection {  pageInfo: PageInfo!  edges: [TodoItemEdge!]!}
type TodoItemEdge {  node: TodoItem!  cursor: ConnectionCursor!}
type TodoItem {  id: ID!  title: String!  description: String  completed: Boolean!  created: DateTime!  updated: DateTime!}

Offset Based Cursor#

By default all cursors will use a form of offset based paging to back cursors.

ProsCons
Allow for recursive relation pagingPages and nodes may change between queries
Straight forward to implementInconsistent Sorting - natural sorting is used by default
Queries grow more inefficient as you go through more pages

Key Set Based Cursor#

You have the option to specify a key set on your DTO which will replace the offset with a where clause. A keyset is a set of fields that uniquely identify a record (e.g. id).

ProsCons
Consistent Results - All cursors uniquely identify a record allowing for deterministic before/after pagesUnable to do recursive paging for relations
Stable Sorting - All results will be deterministically sortedA unique key set must be specified
Consistent performance with correct indexes in placeUnable to jump to arbitrary page
Suitable for datasets of any size

To enable key set based paging all you need to do is decorate your DTO with the @KeySet decorator.

import { FilterableField, IDField, KeySet } from '@nestjs-query/query-graphql';import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';
@ObjectType('TodoItem')@KeySet(['id'])export class TodoItemDTO {  @IDField(() => ID)  id!: number;
  @FilterableField()  title!: string;
  @FilterableField({ nullable: true })  description?: string;
  @FilterableField()  completed!: boolean;
  @FilterableField(() => GraphQLISODateTime)  created!: Date;
  @FilterableField(() => GraphQLISODateTime)  updated!: Date;}

Sorting and Key set cursors#

When using key set based cursors we must take into account any client provided sorting in order to uniquely identify a record within the sorted set of nodes

For example assume we're using the same DTO as above. If we added a sort by completed a comparision on id would no longer be sufficient

{  todoItems(sorting: [{ field: completed, direction: ASC }]) {    edges {      cursor      node {        id        title        completed      }    }  }}

If we only compared on the keyset [id] our pages would no longer be sorted properly, if we only compared on completed you would not be able to page.

We solve this problem by encoding information about the each field in the sort as well as the key set fields into the cursor so we can page properly.

In the above example the filter from the cursor (when paging forward) would be something like

WHERE (completed > ? OR (completed = ? AND id > ?))

Relation Connections#

Key set paging will not apply to relations because they are recursive by nature.

For example if you query for multiple TodoItems and their subTasks if key set paging was used for the subTasks connection the cursor from one todoItems subTasks may not be applicable to all todoItems

For example, assume you have the following todo items and subTasks

[  {    "id": 1,    "title": "Todo 1"    "subTasks": [      {"id": 1, "title": "Todo 1 - Sub Tasks 1"}, // cursor: "abc"      {"id": 2, "title": "Todo 1 - Sub Tasks 2"}  // cursor: "def"    ]  },  {    "id": 2,    "title": "Todo 2"    "subTasks": [      {"id": 3, "title": "Todo 2 - Sub Tasks 1"},  // cursor: "ghi"      {"id": 4, "title": "Todo 2 - Sub Tasks 2"}  // cursor: "jkl"    ]  }]

If you ran the following graphql query

{  todoItems {    edges {      node {        title        subTasks(paging: {first: 2, after: "ghi"}){          title        }      }    }  }}

The resulting query would look for all subTasks with an id > 3 breaking paging for Todo 1, for this reason the @KeySet decorator is used for all relations.

Paging#

To page with cursors it works the same way for all strategies.

In this example we'll fetch the first 2 records.

{  todoItems(paging: {first: 2}) {    pageInfo{      hasNextPage      hasPreviousPage      startCursor      endCursor    }    edges{      node{        id        title        completed        created        updated      }      cursor    }  }}

Lets take a look the pageInfo from the previous example

{  "pageInfo": {    "hasNextPage": true,    "hasPreviousPage": false,    "startCursor": "YXJyYXljb25uZWN0aW9uOjA=",    "endCursor": "YXJyYXljb25uZWN0aW9uOjE="  },}

Notice how hasNextPage is true and there is an endCursor that can be used to fetch the next page.

{  todoItems(paging: { after: "YXJyYXljb25uZWN0aW9uOjE=", first: 2 }) {    pageInfo {      hasNextPage      hasPreviousPage      startCursor      endCursor    }    edges {      node {        id        title        completed        created        updated      }      cursor    }  }}

We can also page backward by using before and last. In the following example we'll use the startCursor from the previous example and set last to 2.

{  todoItems(paging: { before: "YXJyYXljb25uZWN0aW9uOjI=", last: 2 }) {    pageInfo {      hasNextPage      hasPreviousPage      startCursor      endCursor    }    edges {      node {        id        title        completed        created        updated      }      cursor    }  }}

Offset Based Paging#

An alternative to cursor based querying is to use the OFFSET pagingStrategy. When using the OFFSET strategy queries that return multiple records will return an OffsetConnection that looks like the following.

type OffsetPageInfo {  hasNextPage: Boolean  hasPreviousPage: Boolean}
type TodoItemConnection {  pageInfo: OffsetPageInfo!  nodes: [TodoItem!]!}
type TodoItem {  id: ID!  title: String!  description: String  completed: Boolean!  created: DateTime!  updated: DateTime!}
ProsCons
Easy to understandunstable - pages and nodes may change between queries
Able to jump to arbitrary pagesInconsistent Sorting - natural sorting is used in the strategy
Queries grow more inefficient as you go through more pages
State must be maintained in the client to know last limit and offset in order to page

To enable OFFSET based paging all you need to do is set the pagingStrategy option using the @QueryOptions decorator.

todo-item.dto.ts
import { FilterableField, IDField, QueryOptions } from '@nestjs-query/query-graphql';import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';
@ObjectType('TodoItem')@QueryOptions({ pagingStrategy: PagingStrategies.OFFSET })export class TodoItemDTO {  @IDField(() => ID)  id!: number;
  @FilterableField()  title!: string;
  @FilterableField({ nullable: true })  description?: string;
  @FilterableField()  completed!: boolean;
  @FilterableField(() => GraphQLISODateTime)  created!: Date;
  @FilterableField(() => GraphQLISODateTime)  updated!: Date;}

In this example we'll fetch the first 2 records.

{  todoItems(paging: { limit: 2 }) {    pageInfo {      hasNextPage      hasPreviousPage    }    nodes {      id      title      completed      created      updated    }  }}

In this example we'll also pass in an offset to fetch the next 2 records.

{  todoItems(paging: { limit: 2, offset: 2 }) {    pageInfo {      hasNextPage      hasPreviousPage    }    nodes {      id      title      completed      created      updated    }  }}

Disabling Paging#

caution

This strategy is only recommended if you are sure your dataset is small.

When using the NONE paging strategy the paging argument is removed and all methods will return an ArrayConnection.

ProsCons
All data is returned at onceNot suitable for large datasets

To disable paging all you can set the pagingStrategy option using the @QueryOptions decorator to PagingStrategies.NONE.

todo-item.dto.ts
import { FilterableField, IDField, QueryOptions } from '@nestjs-query/query-graphql';import { ObjectType, ID, GraphQLISODateTime } from '@nestjs/graphql';
@ObjectType('TodoItem')@QueryOptions({ pagingStrategy: PagingStrategies.NONE })export class TodoItemDTO {  @IDField(() => ID)  id!: number;
  @FilterableField()  title!: string;
  @FilterableField({ nullable: true })  description?: string;
  @FilterableField()  completed!: boolean;
  @FilterableField(() => GraphQLISODateTime)  created!: Date;
  @FilterableField(() => GraphQLISODateTime)  updated!: Date;}

When using the strategy queries that return multiple records will return an array instead of a connection.

{  todoItems {    id    title    completed    created    updated  }}