Pagination Strategies: Page-Based and Cursor-Based

Pagination Strategies: Page-Based and Cursor-Based

No REST API should return unbounded lists of data. Returning 100,000 records in a single response will crash clients, exhaust server memory, and saturate network bandwidth. Pagination divides large result sets into manageable pages, and choosing the right strategy has significant performance implications.

Page-Based (Offset) Pagination

Page-based pagination uses a page number and limit: GET /api/posts?page=3&limit=20. This skips (page-1)*limit records and returns the next limit records.

const page = parseInt(req.query.page) || 1;
const limit = Math.min(parseInt(req.query.limit) || 20, 100);
const skip = (page - 1) * limit;

const [data, total] = await Promise.all([
  Post.find(filter).skip(skip).limit(limit).sort(sort),
  Post.countDocuments(filter)
]);

res.json({
  success: true,
  data,
  meta: {
    total,
    page,
    limit,
    totalPages: Math.ceil(total / limit),
    hasNextPage: page * limit < total,
    hasPrevPage: page > 1
  }
});

Limitation: OFFSET-based pagination is slow on large datasets because the database must scan all preceding rows even if it does not return them. Records can also appear twice or be skipped if data is inserted or deleted between requests.

Cursor-Based Pagination

Cursor-based pagination uses the ID or timestamp of the last seen record as a cursor, fetching the next N records after it. It is more efficient and consistent for large, frequently changing datasets.

const { after, limit = 20 } = req.query;
const query = after ? { _id: { $gt: after } } : {};

const data = await Post.find({ ...filter, ...query })
  .limit(parseInt(limit) + 1)
  .sort({ _id: 1 });

const hasNextPage = data.length > limit;
if (hasNextPage) data.pop();

res.json({
  data,
  meta: {
    hasNextPage,
    nextCursor: hasNextPage ? data[data.length - 1]._id : null
  }
});