Mongo Monitor Script: Health Check

Description

The Script will search for any working server listed in the mongo.properties. It executes the rs.status() command and check the health of all the server in replication

Step to configure mongo.properties

In mongo.properties we need to set the below properties

<!--[if !supportLists]-->· <!--[endif]-->USERNAME [mongo database user's username]

<!--[if !supportLists]-->· <!--[endif]-->PASSWORD [Password]

<!--[if !supportLists]-->· <!--[endif]-->MONGO_INSTALL_BIN_DIR [Mongo installed bin directory]

<!--[if !supportLists]-->· <!--[endif]-->HOSTNAME [List of all servers (get it by execute rs.status() command on mongo client) separated with ;]

<!--[if !supportLists]-->· <!--[endif]-->DEFAULT_HOST_PORT [Default server setting for health check]

USERNAME=username

PASSWORD=password

MONGO_INSTALL_BIN_DIR=/usr/local/mongo/bin/

DEFAULT_HOST_PORT=hostname:27017

HOSTNAME="host1:27017;host2:27017;host3:27017"

Step to configure mongo-health-chk.sh

Open the mongo-health-chk.sh and correct the path for the mongo.properties

For e.g. . /usr/local/mongo.properties

Setting the CRON job

To execute the script as cron job, run below command on shell prompt

#crontab -e

This will open the cron job file. Make an entry of below cron expression [update the script location path]. This will run the script in every 5 min. you need to pass the log file name/path as parameter while running this script

*/5 0 * * * /path/to/script/mongo-health-chk.sh mongo_health.log

Below is the script

#!/bin/bash

. /home/mongo/mongo.properties

exec >> $1

#################################################################

# CHECK THE HEALTH OF ALL SERVER IN REPLICATION #

# version:- 1.0 #

# Author:- Chandrashekhar Dehankar #

# THIS SCRIPT WILL SEARCH OUT FOR ANY WORKING SERVER LISTED #

# IN mongo.properties. EXECUTES THE rs.status() COMMAND AND #

# */5 * * * * /home/mongo/mongo-health-chk.sh health.log #

# CRON EXPRESSION EVERY 5MIN IS BELOW #

#################################################################

#DATABASE HOST-PORT DEFAULT SETTING

HOST_PORT=$DEFAULT_HOST_PORT

#METHOD TO CHECK SERVER HEALTH

function _do_health_check {

#FIND ANY WORKING SERVER

DATABASE_HOST_SERVER=$HOSTNAME

DATABASE_HOST_SERVER=${DATABASE_HOST_SERVER//;/$'\n'}

for host in $DATABASE_HOST_SERVER

do

#CHECK WEATHER ITS IS UP OR NOT WHAT, NO MATTERS PRIMARY OR SECONDARY

IS_SERVER_UP=$($MONGO_INSTALL_BIN_DIR/mongo $host/admin -u $USERNAME -p $PASSWORD --eval "printjson(db.isMaster().secondary)" )

#FILTER THE RESULT

IS_SERVER_UP=`echo $IS_SERVER_UP grep -o '[^ ]*$'`

if [ "$IS_SERVER_UP" == "true" ] [ "$IS_SERVER_UP" == "false" ]; then

HOST_PORT=$host

break

fi

done

count=0

CURRENT_DATE=`date +%Y%m%d_%H-%M-%S`

echo "--------HEALTH TEST START[$CURRENT_DATE] ----------"

while [ $count -ne -1 ]

do

HOST_NAME=$($MONGO_INSTALL_BIN_DIR/mongo $HOST_PORT/admin -u $USERNAME -p $PASSWORD --eval "printjson(rs.status().members[$count].name)" )

HOST_NAME=`echo $HOST_NAME grep -o '[^ ]*$'`

CHECK_HEALTH=$($MONGO_INSTALL_BIN_DIR/mongo $HOST_PORT/admin -u $USERNAME -p $PASSWORD --eval "printjson(rs.status().members[$count].health)" )

CHECK_HEALTH=`echo $CHECK_HEALTH grep -o '[^ ]*$'`

count=$(( $count + 1 ))

#CHECK ALL THE NODES IN REPLICATION

if [ "$CHECK_HEALTH" != "1" ] && [ "$CHECK_HEALTH" != "0" ]; then

echo "..........All SERVER HEALTH TEST DONE.........."

count="-1"

fi

if [ $count -ne -1 ]; then

echo "----------------------------------------------------------"

if [ "$CHECK_HEALTH" == "1" ]; then

echo $HOST_NAME IS WORKING FINE

else

echo $HOST_NAME IS DOWN

fi

echo "----------------------------------------------------------"

fi

done

}

## check the health

_do_health_check

Mongo Database Backup Script

Description

This script use for taking the mongo database backup using --oplog option, this script create The backup folder with current timestamp, also delete the backup folder older than 2 days for daily backup and delete the folder older than 30 days for weekly backup, This script will Run as part of cron job midnight everyday and once in week. This script find out the secondary server for the backup configured in mongo.properties. If no secondary found then it will use The primary for default backup. The default behavior is daily backup if no input provide.

Configure mongo.properties

In mongo.properties we need to set the below properties

· USERNAME [Backup user's username]

· PASSWORD [Password]

· DEFAULT_HOST_PORT [Default server set for backup]

· DEFAULT_BACKUP_DESTINATION [ Path to backup root folder ]

· MONGO_INSTALL_BIN_DIR [mongo installation bin directory path ]

· HOSTNAME [List of all servers(get it by execute rs.status() command on mongo client) separated with ; ]

USERNAME=username

PASSWORD=password

DEFAULT_HOST_PORT=hotsname:27017

DEFAULT_BACKUP_DESTINATION=backup/dir/name

MONGO_INSTALL_BIN_DIR=mongo/install/dir/bin

HOSTNAME="host1:27017;host2:27017;host3:27017"

Configure mongo-backup.sh

Open the mongo-backup.sh and set the path for the mongo.properties

for e.g. . /path/to/properties/mongo.properties

Setting the CRON job

To execute the script as cron job, run below command on shell prompt

#crontab -e

This will open the cron job file. For daily midnight backup make entry of below cron expression [update the script location path]

0 0 * * * /PATH/TO/SCRIPT/mongo-backup.sh daily

For weekly backup on Sunday (Sunday = 0, Monday = 1......) at midnight make entry of below cron expression [update the script location path]

0 0 * * 0 /PATH/TO/SCRIPT/mongo-backup.sh weekly

The user who is going to run this script should have the permission [ read /write] to mongo.properties file and backup folder and script.

Here is the script

# DATABASE HOST-PORT DEFAULT SETTING

HOST_PORT=$DEFAULT_HOST_PORT

#NO OF DAY FOR DELETE OLD BACKUP [DEFAULT VALUE]

NO_OF_DAY=2

CHECK_BACKUP_TYPE=$1

#METHOD TO TAKE MONGO BACKUP

function _do_backup {

# CREATING FOLDER WITH CURRENT TIME STAMP

BACKUP_CURRENT_DATE=`date +%Y%m%d_%H-%M-%S`

BACKUP_CURRENT_DATE="$(echo ${BACKUP_CURRENT_DATE} | tr ' ' '0')"

UNIQ_DIR=""

if [ "$CHECK_BACKUP_TYPE" == "daily" ] || [ "$CHECK_BACKUP_TYPE" == "" ]; then

BACKUP_FOLDER=DAILY_BACKUP/

BACKUP_DESTINATION=$DEFAULT_BACKUP_DESTINATION$BACKUP_FOLDER

UNIQ_DIR=$BACKUP_DESTINATION$BACKUP_CURRENT_DATE

elif [ "$CHECK_BACKUP_TYPE" == "weekly" ]; then

BACKUP_FOLDER=WEEKLY_BACKUP/

BACKUP_DESTINATION=$DEFAULT_BACKUP_DESTINATION$BACKUP_FOLDER

UNIQ_DIR=$BACKUP_DESTINATION$BACKUP_CURRENT_DATE

fi

mkdir -p $UNIQ_DIR

#FIND SECONDARY SERVER

DATABASE_HOST_SERVERS=$HOSTNAMES

DATABASE_HOST_SERVERS=${DATABASE_HOST_SERVERS//;/$'\n'}

for host in $DATABASE_HOST_SERVERS

do

IS_SECONDAY=$(mongo $host/admin -u $USERNAME -p $PASSWORD --eval "printjson(db.isMaster().secondary)" )

#FILTER THE RESULT

IS_SECONDAY=`echo $IS_SECONDAY | grep -o '[^ ]*$'`

if [ "$IS_SECONDAY" == "true" ]; then

HOST_PORT=$host

break

fi

done

#execute mongo dump command

#$MONGO_INSTALLATION_BIN --host $HOST_PORT -u $USERNAME -p $PASSWORD --oplog --out $UNIQ_DIR

$MONGO_INSTALL_BIN_DIR/mongodump --host $HOST_PORT -u $USERNAME -p $PASSWORD --oplog --out $UNIQ_DIR

}

# THIS METHOD WILL DELETE THE FOLDER OLDER THAN 2 DAY

function _do_delete_old_backup {

if [ "$CHECK_BACKUP_TYPE" == "daily" ] || [ "$CHECK_BACKUP_TYPE" == "" ]; then

NO_OF_DAY=2

echo "deleting daily older backup !!!!!!!!!!!!!!!!!!!!!!"

else

NO_OF_DAY=30

echo "deleting weekly older backup !!!!!!!!!!!!!!!!!!!!!!"

fi

find $BACKUP_DESTINATION -type d -mtime +$NO_OF_DAY -exec rm -rf {} \;

}

## take the backup

_do_backup

## delete the old folder

_do_delete_old_backup