In node, execute ‘require’ the least amount of times possible

More often than not if someone says don’t do this, I need to try it out for myself. It’s Curiosity, I like my knowledge visceral and first hand.

Like most developers, carrying my C# and Java experience into Node.js I naturally declare my variables as late as possible, often during their assignment, because this adds to the legibility of the code. You can see what it is, what its value is, and you have minimized the scope of the variable, and all of these things are good. Off-course these languages enforce using (c#) and import (java) as the top statements in code units. Node.js does not enforce this, and its require statements don’t just pull types into scope, being a dynamically typed language they instead return a variable. Well, it was not long before I started writing code that contained require statements in the body of functions, and this is not advisable from a performance perspective.

Node is asynchronous and single-threaded (for problem-state code), and we should always favor asynchronous code in Node as a result, but require statements are not asynchronous. Require statements, require code units to be located and pulled into the unit you are executing. This is a lot of IO. Then, they require compilation as well so that the next line of code you execute inherits an environment as you may expect it.

Now it makes sense that this is all cached in the system somewhere, so running over the same ‘require’ statement more than once should not cause too much of an issue, but what is significant and what is not significant? Right, it’s time to test it!

I wrote out a simple test and executed it using Node v6.7.0.

Simple Test

/**
 * Quickly compare the impact of multiple executions of require
 */
let inline = true;
let j = 0;
if (inline) {
  var express = require('express');
  var path = require('path');
  var favicon = require('serve-favicon');
  var logger = require('morgan');
  var cookieParser = require('cookie-parser');
  var bodyParser = require('body-parser');
  
  var index = require('./routes/index');
  var users = require('./routes/users');
}

let start = new Date().getTime();

for (let i = 0; i < 10000; i++) {
  payload();
}

let end = new Date().getTime() - start;
console.log(`Elapsed time=${end}`);


function payload() {
  if (!inline) {
    var express = require('express');
    var path = require('path');
    var favicon = require('serve-favicon');
    var logger = require('morgan');
    var cookieParser = require('cookie-parser');
    var bodyParser = require('body-parser');
  
    var index = require('./routes/index');
    var users = require('./routes/users');
  }
  j = j + 1;
}

The results are rather interesting. For 10,000 executions, the response time cost is a choice between ~500ms or ~2ms.

Results Interpretation:

Whether you set the inline variable true or false, the function payload is called, and if statement is tested and an integer j is incremented. Setting this to value true executed the require functions once, and setting these to false executes them 10,000 times. Now, requiring these modules also requires in the background many more modules.

There are 8 top-level require calls but these call in turn others, 91 others in fact. Let’s assume that the simple act of requiring the top-level modules requires all of the children once. There are ~99 require executions per loop of payload. For simplicity sake, let’s consider this to be 10 top level require calls and 100 individual require calls.

If you are an optimist and want to support placing the require functions within the functions so that these may be executed more than once during the life of the application, you can claim that the overhead is very low per execution – 0.0005 milliseconds per individual require operation or 0.005 milliseconds per top level require operation.

If you are a pessimist, you see a 99.6% difference between the calls, and that within a single invocation of a process that takes just 0.5 seconds to execute.

I am a pessimist – lesson internalized – all require statements will be placed at the top of each unit as far as possible from here on out.

Leave a Reply

Your email address will not be published. Required fields are marked *