Wsrtjtyk

I was just experimenting with clustering in Node.js. I wrote a little fibonacci example, where I called the function using a clustered server and a non-clustered server as follows.

// fib.js



module.exports = function () {

    const store = 

    store[0] = 0

    store[1] = 1



    return function fib(n) {

        if (store[n] === undefined) {

            store[n] = fib(n - 2) + fib(n - 1)

        }



        return store[n]

    }

}

// non-clustered server



const http = require('http')

const fib = require('./fib.js')



const f = fib()

http.createServer((req, res) => {

    res.writeHead(200)

    res.end(`From ${process.pid}: ${f(30)}n`)

}).listen(8000)

// clustered server



const cluster = require('cluster')

const http = require('http')

const numCPUs = require('os').cpus().length

const fib = require('./fib.js')



const f = fib()

if (cluster.isMaster) {

    for (let i = 0; i < numCPUs; i++) {

        cluster.fork()

    }

} else {

    http.createServer((req, res) => {

        res.writeHead(200)

        res.end(`From ${process.pid}: ${f(30)}n`)

    }).listen(8000)

}

Now, I tried benchmarking the performance of these two servers using artillery but the non-clustered server seemed to be faster. How come?

// Clustered Server Report



Started phase 0, duration: 1s @ 07:53:24(+0000) 2018-11-23

Report @ 07:53:27(+0000) 2018-11-23

  Scenarios launched:  50

  Scenarios completed: 50

  Requests completed:  2000

  RPS sent: 803.21

  Request latency:

    min: 0.3

    max: 70.5

    median: 20.7

    p95: 38.7

    p99: 45.6

  Codes:

    200: 2000



All virtual users finished

Summary report @ 07:53:27(+0000) 2018-11-23

  Scenarios launched:  50

  Scenarios completed: 50

  Requests completed:  2000

  RPS sent: 796.81

  Request latency:

    min: 0.3

    max: 70.5

    median: 20.7

    p95: 38.7

    p99: 45.6

  Scenario counts:

    0: 50 (100%)

  Codes:

    200: 2000

// Non-clustered Server Report



Started phase 0, duration: 1s @ 07:53:39(+0000) 2018-11-23

Report @ 07:53:41(+0000) 2018-11-23

  Scenarios launched:  50

  Scenarios completed: 50

  Requests completed:  2000

  RPS sent: 806.45

  Request latency:

    min: 0.3

    max: 70.1

    median: 21.4

    p95: 38.3

    p99: 44.6

  Codes:

    200: 2000



All virtual users finished

Summary report @ 07:53:41(+0000) 2018-11-23

  Scenarios launched:  50

  Scenarios completed: 50

  Requests completed:  2000

  RPS sent: 800

  Request latency:

    min: 0.3

    max: 70.1

    median: 21.4

    p95: 38.3

    p99: 44.6

  Scenario counts:

    0: 50 (100%)

  Codes:

    200: 2000

The node cluster documentation says itself, that the worker distribution tends to be very unbalanced due to operating system scheduler vagaries, which affects performance. Also see this comparsion between the cluster nginx and iptables load balancer.

AFAIK performance testing tends to be much more complicated if we want to get meaningfull results. Some mainly statistic related questions about your test:

1. Sample sizing: Why did you choose this sample size? Are you sure that the results will be the same with different sample sizes? Eg.: AFAIK the main advantage of clustering kicks off when the single thread process request handling capability ends.




2. Testing environment: How did you run your test? In which environment? The environment was isolated? On how many core? What kind of processors?




3. Significance: Why do you think your result represents significant difference? How did you test the difference between the samples?