Why would a NodeJS / Restify * server rarely report EPERM?

advertisements

I'm running a restify server in NodeJS. On very rare occasions, on the order of 0.05% of HTTPS requests cause net.js to report the following error:

Error: accept EPERM
    at exports._errnoException (util.js:742:11)
    at TCP.onconnection (net.js:1280:24)

There is nothing special about the HTTP requests. Before this error is reported, the server may have serviced thousands of requests and even responded to dozens of identical requests. I have not been able to find any information about why a server might generate an EPERM error for a socket that has been successfully accepting connections for several hours.

By the way, this error occurs outside of any execution context of our source code. So it's not as if the EPERM is about our code accessing a file or performing some other system call. The EPERM is happening deep within the NodeJS TCP code when the new request arrives and before our code is invoked.

At first, when the error occurred it would cause NodeJS to terminate. So then I added code to catch application-level exceptions:

process.on("uncaughtException", onUncaughtException );

But since I don't know why this error is happening, it's not at all clear what is the recovery process.

Not sure if it will matter, but here is most of the code relevant to starting up the restify service:

var restify    = require("restify");
// skipping some other init code
// configuration data is read from a JSON file
var serverOptions = {
   name: configuration.server.name,
   version: configuration.server.version,
   formatters: {
      "application/json": jsonResponseFormatter,
      "text/html": textResponseFormatter
   },
   serverOptions.key: fs.readFileSync(configuration.server.sslKey),
   serverOptions.cert: fs.readFileSync(configuration.server.sslCert)
}
var server = restify.createServer( serverOptions );
// skipping middleware inits and URL registrations
server.listen(
     configuration.server.port, // using HTTPS 443
     configuration.server.serverip );

By the way, we are running an old version of NodeJS: v0.11.13. My long-term plan is to upgrade to the latest stable version, but we may not be able to update for a few months.


Let me leave my solution here in case anyone else stumbles across this same problem in the future.

Technically, I did not discover why this error was occurring, but I did find out how to handle the error condition successfully: trap and release. The error must be trapped at the application level because it is being generated deep within net.js outside any try-catch context of my source code. So if I don't trap it, then it will crash my application. But the error is non-fatal and it appears that it can be safely ignored. In testing, the socket continued to receive new connections even after this error occurred.

process.on("uncaughtException", onUncaughtException );
function onUncaughtException(error) {
  // put some code here to log the error occurrence, then ...
  if( error.code==="EPERM" && error.syscall==="accept" ) {
     // non-fatal error: do nothing; just ignore this error
  }
  else {
    // handle other application errors here
  }
}

So while it might still be interesting to know why a server socket can occasionally have an EPERM error, for now I'm satisfied knowing the proper way to handle the error when it occurs.