Batching - SharePoint/PnP-JS-Core GitHub Wiki

Besides caching another way to improve performance is batching requests. Helpfully the library provides built in support for batching REST requests. This involves creating a batch instance and using it with the inBatch method to batch those requests. It should be the final method in your method chain (OR if you include usingCache these two can be transposed). Don't forget to call the execute method of the batch object when you are ready to execute the requests. The execute method also returns a promise that resolves once all of your other promises are resolved. Also, check out that you don't have to change your code at all other than to insert the "inBatch" call into your chain. Otherwise your promises will resolve exactly as expected.

Simple Example

let batch = pnp.sp.createBatch();

pnp.sp.web.lists.inBatch(batch).get().then(r => {
    console.log(r)
});

pnp.sp.web.lists.getByTitle("Tasks").items.inBatch(batch).get().then(r => {
    console.log(r)
});

batch.execute().then(() => console.log("All done!"));

Using the promise of the batch's execute method you can set a timeout to refresh the data on page, invalidate a cache, update the UI, or anything else you'd like to do once the batch is complete.

Batching POST requests

Batching works with all types of requests not just GET so you can batch updates, adds, and deletes all in the same batch. The inBatch method should always be the last method chained BEFORE the action, such as add(), update(), or delete(). Another way to think of it is it should be the thing immediately before the method to which you chain a then() call.

let batch = pnp.sp.createBatch();

pnp.sp.web.lists.inBatch(batch).get().then(r => {
    console.log(r)
});

pnp.sp.web.lists.getByTitle("Tasks").items.inBatch(batch).add({
    Title: "My Item Title"
}).then(r => {
    console.log(r)
});

pnp.sp.web.lists.getByTitle("Tasks").items.inBatch(batch).add({
    Title: "My Item Title 2"
}).then(r => {
    console.log(r)
});

pnp.sp.web.lists.getByTitle("Tasks").items.getById(1).inBatch(batch).delete().then(r => {
    console.log(r)
});

batch.execute().then(() => console.log("All done!"));

Remember your batch is executed on the server in the order you provide so make sure your batches make logical sense. And think through performance as well, don't make your batches too large or try and do too many things. You may have to test out what exactly those limits are but keep them in mind as you are developing.

Contextual Batching

It is also possible to create a batch directly from a Web or Site object using that instance's url as the base. In these cases all queries must be made within that same web, you can't mix batches. But this is helpful when creating Web instances directly as illustrated below:

import { Web } from "sp-pnp-js";

let web = new Web("{ your web url }");

let batch = web.createBatch();

web.lists.inBatch(batch).get().then(l => ...);

web.inBatch(batch).get().then(w => ...);

batch.execute().then(_ => ...);

Using Batching with Caching

You can use batching and caching together, but remember caching is only applied to get requests. When you use them together the methods can be transposed, the below example is valid.

let batch = pnp.sp.createBatch();

pnp.sp.web.lists.inBatch(batch).usingCaching().get().then(r => {
    console.log(r)
});

pnp.sp.web.lists.getByTitle("Tasks").items.usingCaching().inBatch(batch).get().then(r => {
    console.log(r)
});

batch.execute().then(() => console.log("All done!"));

Large batch processing

One can't wrap an endless number of request in a batch. Batch body (multipart) has its limitations. It's impossible to say a specific limit number of requests in a batch because it depends.

We would recommend a batch size of 50+ requests then tweak depending on environment and requests nature.

The following example shows a way of creating of 700 items in a row using batches and qeues:

import { Web, List, sp } from 'sp-pnp-js';

interface IBatchCreationData {
  item: any;
  result: any;
  error: any;
}

const processBatch = async (
  list: List,
  itemEntityType: string,
  data: IBatchCreationData[],
  index = 0
): Promise<IBatchCreationData[]> => {
  // Size of a batch, should be tweaked for a package size, 25-50 or a bit more should work
  const batchSize = 50;
  if (data.length > index + 1) {
    let batch = sp.web.createBatch();
    for (let len = index + batchSize; index < len && index < data.length; index += 1) {

      let dataItem = data[index];

      let success = (function(res) {
        // ES5 function is on purpose to bind executioning item
        // Promise is resolved on batch is executed, no phisical request per item
        this.result = res;
      }).bind(dataItem);

      let error = (function(err) {
        // ES5 function is on purpose to bind executioning item
        // Promise rejected on batch is executed per item
        // if err happened (e.g. payload contains an error)
        this.error = err;
      }).bind(dataItem);

      // This is executed in batch, no individual requests
      // yet the promise is resolved when batch data came back and is parsed with the library
      list.items.inBatch(batch).add(dataItem.item, itemEntityType)
        .then(success)
        .catch(error);

    }
    console.log(`Processing (${index} of ${data.length})... `)
    let result = await batch.execute(); // Executing batch of `batchSize`
    // callback with process visualisation, e.i. progress bar tick
    return await processBatch(list, itemEntityType, data, index);
  } else {
    return new Promise(r => r(data));
  }
};

// Executing item creation
(async () => {

  let dummyItems: any[] = [];

  // Just a dummy data
  for (let i = 0; i < 700; i += 1) {
    dummyItems.push({
      Title: `Title ${i}`
    });
  }

  dummyItems[1].NonExistingField = 'Should cause an error in second item';

  const list = sp.web.getList(`${_spPageContextInfo.webServerRelativeUrl}/Lists/MyList`);
  const entityType = await list.getListItemEntityTypeFullName();

  let data: IBatchCreationData[] = dummyItems.map(dd => {
    return {
      item: dd,
      result: null,
      error: null
    }
  });

  return await processBatch(list, entityType, data);

})()
  .then(console.log) // array of initial objects with creation results and error if any
  .catch(console.log);