HowTo: MongoDB Tailable Cursors in Node.js

MongoDB capped collections are great storing volatile information, like access and debug logs. If you need to monitor regular log file, you would probably type something like this in your command line: tail -f /var/log/access_log. Cool thing is that you can do the same with MongoDB capped collections:

Capped collections are fixed-size collections that support high-throughput operations that insert, retrieve, and delete documents based on insertion order. Capped collections work in a way similar to circular buffers: once a collection fills its allocated space, it makes room for new documents by overwriting the oldest documents in the collection.

Creating a capped collection is easy; so lets create capped collection named log and size of 16MB (the maximum size of BSON object if you did not know):

mongo localhost/test
> db.createCollection( "log", { capped: true, size: 16777216 } )

Capped collections act like any other collections, but they work in first-in-first-out mode - when size limit is exceeded, older entries get removed from the collection.

If you are writing your code in node.js and would like to see the objects as they are being written in the collection, then you can use tailable cursor to keep listening the collection; just like subscribing to events.

Following block of node.js code demonstrates, how you can set up subscriber:

var filter = {};

// set MongoDB cursor options
var cursorOptions = {
  tailable: true,
  awaitdata: true,
  numberOfRetries: -1
};

// create stream and listen
var stream = coll.find(filter, cursorOptions).sort({$natural: -1}).stream();
        
// call the callback
stream.on('data', function(document) {
  console.log(document);
});

In result, whenever any program insert new document(s) into the capped collection, the document will be output to the console.

See this gist for fully working example of MongoDB Tailable Cursor in Node.js.
Until next time.