MongoDB Monitoring: rs.status()

By David Mytton,
CEO & Founder of Server Density.

Published on the 12th January, 2011.

rs.status()

There are a number of built in tools and commands which can be used to get important information from MongoDB but because it is relatively new, it can be difficult to know what you need to be doing from an operational perspective to ensure that everything runs smoothly.

This is the 4th in a series of 6 posts about MongoDB monitoring based on a talk I gave at the MongoSV 2010 conference. View the series index here.

MongoDB Monitoring dashboard + alerting

We provide a MongoDB monitoring service to keep an eye on the health and performance of your MongoDB database cluster automatically and with alerting. Find out more here.

rs.status()

If you’re running a replica set then you can use the rs.status() command to get information about it. This can be run from the console on any member of any set and shows the status as that particular member sees it. This means that network partitions can be seen from different sides of the divide.

This output shows the status as a slave in one of our replica sets sees it.

> rs.status()
{
	"set" : "set1",
	"date" : ISODate("2011-01-12T16:09:58Z"),
	"myState" : 2,
	"members" : [
		{
			"_id" : 0,
			"name" : "rs1a:27018",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"self" : true
		},
		{
			"_id" : 1,
			"name" : "rs1b:27018",
			"health" : 1,
			"state" : 1,
			"stateStr" : "PRIMARY",
			"uptime" : 770622,
			"optime" : {
				"t" : 1294848597000,
				"i" : 13
			},
			"optimeDate" : ISODate("2011-01-12T16:09:57Z"),
			"lastHeartbeat" : ISODate("2011-01-12T16:09:57Z")
		},
		{
			"_id" : 3,
			"name" : "rs1arbiter:27018",
			"health" : 1,
			"state" : 7,
			"stateStr" : "ARBITER",
			"uptime" : 770622,
			"optime" : {
				"t" : 0,
				"i" : 0
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("2011-01-12T16:09:58Z")
		},
		{
			"_id" : 4,
			"name" : "rs1c:27018",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 770622,
			"optime" : {
				"t" : 1294848595000,
				"i" : 390
			},
			"optimeDate" : ISODate("2011-01-12T16:09:55Z"),
			"lastHeartbeat" : ISODate("2011-01-12T16:09:57Z")
		},
		{
			"_id" : 5,
			"name" : "rs1d:27018",
			"health" : 1,
			"state" : 2,
			"stateStr" : "SECONDARY",
			"uptime" : 770624,
			"optime" : {
				"t" : 1294841397000,
				"i" : 56
			},
			"optimeDate" : ISODate("2011-01-12T14:09:57Z"),
			"lastHeartbeat" : ISODate("2011-01-12T16:09:57Z")
		}
	],
	"ok" : 1
}

health/myState

This is a numerical value that is either myState for the current member or state for each other member in the set. Health is boolean where 1 = good, 0 = bad, but the state gives more information about the specific state that member is in:

  • 0 = Starting up (phase 1)
  • 1 = Primary
  • 2 = Secondary
  • 3 = Recovering
  • 4 = Fatal error
  • 5 = Starting up (phase 2)
  • 6 = Unknown state
  • 7 = Arbiter
  • 8 = Down

Optime

Replica set members who are not master will be secondary, which means they’ll act as a slave staying up to date with the master. The optimeDate allows you to see whether a member is behind on the replication sync. The timestamp is the last applied log item so if it’s up to date, it’ll be very close to the current actual time on the server.

Our rs1d member is 2 hours behind the master because we set a 2 hour sync delay.

Heartbeat

The whole idea behind replica sets is that they automate failover in the event of failure somewhere. This is done by a regular heartbeat that all members send out to all other members. The status output shows you the last time that particular member was contacted from the current member you executed the command from. In the event of a network partition it may be that some members can’t communicate with each other, and when there is an error you’ll see it in this section too e.g.

		{
			"_id" : 3,
			"name" : "rs2c:27018",
			"health" : 0,
			"state" : 6,
			"stateStr" : "(not reachable/healthy)",
			"uptime" : 0,
			"optime" : {
				"t" : 0,
				"i" : 0
			},
			"optimeDate" : ISODate("1970-01-01T00:00:00Z"),
			"lastHeartbeat" : ISODate("1970-01-01T00:00:00Z"),
			"errmsg" : "connect/transport error"
		},

rs.isMaster()

This command gives you some basic info about the whole set, including whether the current member is master or not and who the other members are.

> rs.isMaster()
{
	"setName" : "set1",
	"ismaster" : false,
	"secondary" : true,
	"hosts" : [
		"rs1a:27018",
		"rs1b:27018"
	],
	"arbiters" : [
		"rs1arbiter:27018"
	],
	"primary" : "rs1b:27018",
	"maxBsonObjectSize" : 8388608,
	"ok" : 1
}

You’ll notice that this command is only returning rs1a and rs1b in the hosts list, even though we have an additional 2 members rs1c and rs1d which are shown in the rs.status() output. This is because these final 2 members are set to hidden in the replica set config because we use them for disaster recovery only and one uses a 2 hour sync delay. We allow our drivers to read from slaves and want to exclude these last 2 members from the pool. This is new functionality in 1.7.

Stay tuned

This is the 4th post in a series of 6 on MongoDB Monitoring. View the full post index and don’t forget to subscribe via RSS or Twitter to be notified of new posts and other cool stuff!

Articles you care about. Delivered.

Help us speak your language. What is your primary tech stack?

Maybe another time