The N+1 Problem & DataLoader

The N+1 problem is the most common performance pitfall in GraphQL. Understanding and fixing it is essential for production GraphQL APIs.

What is the N+1 Problem?

Consider this query:

query {
  posts {      # 1 query to get all posts
    title
    author {   # N queries - one for each post's author!
      name
    }
  }
}

Let's See It in Action

// Naive resolver - causes N+1
const resolvers = {
  Query: {
    posts: () => db.query('SELECT * FROM posts')  // 1 query
  },

  Post: {
    author: (post) => {
      // Called once for EACH post!
      console.log(`Fetching author ${post.authorId}`);
      return db.query('SELECT * FROM users WHERE id = ?', post.authorId);
    }
  }
};

Server logs with 5 posts:

Fetching author 1
Fetching author 2
Fetching author 1  ← Same author, fetched again!
Fetching author 3
Fetching author 1  ← And again!

Enter DataLoader

[DataLoader](https://github.com/graphql/dataloader) is a utility that:

• Batches - Collects all IDs requested in a single tick
• Deduplicates - Removes duplicate IDs
• Caches - Stores results for the duration of the request

import DataLoader from 'dataloader';

// Batch function - receives array of IDs, returns array of results
const userLoader = new DataLoader(async (userIds) => {
  console.log(`Batched fetch for users: ${userIds}`);

  // One query for all users!
  const users = await db.query(
    'SELECT * FROM users WHERE id IN (?)',
    [userIds]
  );

  // IMPORTANT: Return results in the same order as input IDs
  return userIds.map(id => users.find(user => user.id === id));
});

Using DataLoader in Resolvers

const resolvers = {
  Query: {
    posts: () => db.query('SELECT * FROM posts')
  },

  Post: {
    author: (post, _, context) => {
      // Use the loader instead of direct DB query
      return context.loaders.userLoader.load(post.authorId);
    }
  }
};

Server logs with 5 posts (same query):

Batched fetch for users: [1, 2, 3]  ← Just ONE query with unique IDs!

How DataLoader Works

Setting Up DataLoader Per-Request

// loaders.js
import DataLoader from 'dataloader';

export function createLoaders(db) {
  return {
    userLoader: new DataLoader(async (ids) => {
      const users = await db.users.findByIds(ids);
      return ids.map(id => users.find(u => u.id === id) || null);
    }),

    postLoader: new DataLoader(async (ids) => {
      const posts = await db.posts.findByIds(ids);
      return ids.map(id => posts.find(p => p.id === id) || null);
    }),

    // Loader for one-to-many relationships
    postsByAuthorLoader: new DataLoader(async (authorIds) => {
      const posts = await db.posts.findByAuthorIds(authorIds);
      // Group posts by author
      return authorIds.map(authorId =>
        posts.filter(p => p.authorId === authorId)
      );
    })
  };
}

// server.js
const server = new ApolloServer({
  typeDefs,
  resolvers,
  context: ({ req }) => ({
    user: getUser(req),
    loaders: createLoaders(db)  // New loaders per request!
  })
});

Batch Function Rules

The batch function must follow these rules:

// Correct batch function
const userLoader = new DataLoader(async (ids) => {
  const users = await db.query(
    'SELECT * FROM users WHERE id IN (?)',
    [ids]
  );

  // Create a map for O(1) lookup
  const userMap = new Map(users.map(u => [u.id, u]));

  // Return in correct order, null for missing
  return ids.map(id => userMap.get(id) || null);
});

Real-World Example

// Complete resolver setup with DataLoader
import { ApolloServer, gql } from 'apollo-server';
import DataLoader from 'dataloader';

const typeDefs = gql`
  type User {
    id: ID!
    name: String!
    posts: [Post!]!
  }

  type Post {
    id: ID!
    title: String!
    author: User!
    comments: [Comment!]!
  }

  type Comment {
    id: ID!
    text: String!
    author: User!
  }

  type Query {
    posts: [Post!]!
  }
`;

// Simulated database
const db = {
  users: [
    { id: '1', name: 'Sarah' },
    { id: '2', name: 'Alex' },
    { id: '3', name: 'Jordan' }
  ],
  posts: [
    { id: 'p1', title: 'GraphQL 101', authorId: '1' },
    { id: 'p2', title: 'DataLoader Tips', authorId: '1' },
    { id: 'p3', title: 'REST APIs', authorId: '2' }
  ],
  comments: [
    { id: 'c1', text: 'Great post!', postId: 'p1', authorId: '2' },
    { id: 'c2', text: 'Thanks!', postId: 'p1', authorId: '1' },
    { id: 'c3', text: 'Helpful', postId: 'p2', authorId: '3' }
  ]
};

function createLoaders() {
  return {
    user: new DataLoader(async (ids) => {
      console.log(`Batch loading users: ${ids}`);
      return ids.map(id => db.users.find(u => u.id === id));
    }),

    postsByAuthor: new DataLoader(async (authorIds) => {
      console.log(`Batch loading posts for authors: ${authorIds}`);
      return authorIds.map(authorId =>
        db.posts.filter(p => p.authorId === authorId)
      );
    }),

    commentsByPost: new DataLoader(async (postIds) => {
      console.log(`Batch loading comments for posts: ${postIds}`);
      return postIds.map(postId =>
        db.comments.filter(c => c.postId === postId)
      );
    })
  };
}

const resolvers = {
  Query: {
    posts: () => db.posts
  },

  Post: {
    author: (post, _, { loaders }) => loaders.user.load(post.authorId),
    comments: (post, _, { loaders }) => loaders.commentsByPost.load(post.id)
  },

  Comment: {
    author: (comment, _, { loaders }) => loaders.user.load(comment.authorId)
  },

  User: {
    posts: (user, _, { loaders }) => loaders.postsByAuthor.load(user.id)
  }
};

const server = new ApolloServer({
  typeDefs,
  resolvers,
  context: () => ({ loaders: createLoaders() })
});

Before vs After

Key Takeaways

Next up: When to Use GraphQL - Learn the decision framework for choosing between GraphQL and REST.