Premium Hosted Website & Server Monitoring Tool.

(Sysadmin / Devops blog)

visit our website

Blog   >   MongoDB   >   MongoDB Monitoring: Keep in it RAM

MongoDB Monitoring: Keep in it RAM

MongoSV-slides.001

There are a number of built in tools and commands which can be used to get important information from MongoDB but because it is relatively new, it can be difficult to know what you need to be doing from an operational perspective to ensure that everything runs smoothly.

This is the 1st in a series of 6 posts about MongoDB monitoring based on a talk I gave at the MongoSV 2010 conference. View the series index here.

MongoDB Monitoring dashboard + alerting

We provide a MongoDB monitoring service to keep an eye on the health and performance of your MongoDB database cluster automatically and with alerting. Find out more here.

Keep it in RAM

MongoSV-slides.003

The first and most obvious thing to note is that keeping everything in RAM is faster. But what does that actually mean and how do you know when something is in RAM?

In every case, having something in memory is going to be faster than not. However, that’s not always feasible if you have massive data sets. Instead, you want to make sure you always have enough RAM to store all the indexes.

The MongoDB console provides an easy way to look at the data and index sizes. The db.stats() command will analyse the database and give you a range of statistics. The output is provided in bytes for dataSize and indexSize. It may take a few seconds for this command to return for large databases and in the most recent versions of MongoDB, it will not block.

> db.stats()
{
	"collections" : 3,
	"objects" : 379970142,
	"avgObjSize" : 146.4554114991488,
	"dataSize" : 55648683504,
	"storageSize" : 61795435008,
	"numExtents" : 64,
	"indexes" : 1,
	"indexSize" : 21354514128,
	"fileSize" : 100816388096,
	"ok" : 1
}

Here we can see we have around 51GB of data and 19GB of indexes. This means we’d need at least 20 GB of RAM for just indexes and 72GB of RAM for both data and indexes.

For larger data sets like this, a good rule is to ensure you have enough memory for the working set. You define your own working set by looking at the collections you know you want to be kept in RAM and ensuring that there is sufficient RAM for them. You can use the db.collectionName.stats() command on each individual collection to determine its total size.

MongoSV-slides.005

There’s no way to tell MongoDB which collections you want to prioritise for memory but it is smart about its memory management so it will keep commonly accessed data in RAM where possible.

How you’ll know – 1) slow queries

Thu Oct 14 17:01:11 [conn7410] update sd.apiLog query: { c: "android/setDeviceToken", a: 1466, u: "blah", ua: "Server Density Android" } 51926ms 

Although not the only reason, a slow query does indicate insufficient memory. This might be that you’ve not got the most optimal indexes for a query but if indexes are being used and it’s still slow, it could be because of a disk i/o bottleneck because the data isn’t in RAM. Doing an explain on the query will show you what indexes it is using.

How you’ll know – 2) cursor timeouts

cursor timed out (20000 ms) 

These slow queries will obviously cause a slowdown in your app but they may also cause timeouts. In the PHP driver a cursor will timeout after 20,000ms by default, although this is configurable (details).

How you’ll know – 3) disk i/o spikes

MongoSV-slides.008

You’ll see write spikes during normal operations because MongoDB syncs data to disk periodically, but if you’re seeing read spikes then that can indicate MongoDB is having to read the data files rather than accessing data from memory. Be careful though because this won’t distinguish between data and indexes, or even other server activity. Read spikes can also occur even if you have little or no read activity if the mongod is part of a cluster where the slaves are reading from the oplog.

Monitoring disk i/o is easy with a tool like iostat or our own server monitoring service, Server Density.

Reclaiming indexes

MongoDB suffers from a problem with reclaiming index space from deleted documents. Online compaction is being worked on but in the meantime there are a couple of strategies you can use to reclaim that space.

Stay tuned

This is the 1st post in a series of 6 on MongoDB Monitoring. View the full post index and don’t forget to subscribe via RSS or Twitter to be notified of new posts and other cool stuff!