Node.js background processing

I’m new to node.js
, so please forgive what probably is a naive question :) My question is what is the best way to setup a non-UI job written in node? The task I’ve created is used to crawl some web content based upon an Azure queue (the queue message tells the job which content to crawl). All of the examples I see around node are more UI and request based, using http.createServer and listening on a specific port. While I can make this work, this doesn’t seem right, it seems like I just need to create some sort of javascript setInterval loop (or something similar) that keeps looking at my queue. Any suggestions or examples that would push me in the right direction would be greatly appreciated.

Chris

Problem courtesy of: Chris Dellinger

Solution

I’m not really clear on what you’re trying to do, but node doesn’t depend on the http stack at all. If you just want to start node and have it process something, that is pretty straightforward. Your app.js could be as simple as:

var queueWorker = require('worker');

var startWorker = function() {
    if(queueWorker.hasWork()) {
        queueWorker.processQueue(startWorker);
    } else {
        setTimeout(startWorker, 1000);
    }
};

startWorker();

What this is doing is setting up a worker loop where every second it will check to see if there is new work, and if there is start processing it. Once it is done processing the work, go back to the 1 second interval checking for new work.

You would have to create the worker module as the check for hasWork and the processing of said work is application dependent.

If you wanted to get a little more fancy, processQueue could spawn a new node process which is only responsible for actually processing the work, then you could keep track of the number of spawned workers versus CPU limitations and have a relatively simple node app which processes data on multiple threads.

Solution courtesy of: Timothy Strimple

稿源:Node.js Recipes (源链) | 关于 | 阅读提示

本站遵循[CC BY-NC-SA 4.0]。如您有版权、意见投诉等问题,请通过eMail联系我们处理。
酷辣虫 » 前端开发 » Node.js background processing

喜欢 (0)or分享给?

专业 x 专注 x 聚合 x 分享 CC BY-NC-SA 4.0

使用声明 | 英豪名录