How To configure Centralized Health Monitor of multiple Linux Servers.

It is most important task to check server health and current status of our linux servers. These days most of server hacked due to some application vulnerability. Attacker keep trying to establish unauthorized access of our server. I have faced same problem with most of Linux servers in my organization. So I have developed a small utility to prevent this kind of attacks and stay updated for current Linux System Status.

Lets Start !!!

Required Tools/Script To Configure System Health Monitoring

1. mSysMon : To Generate Human Readable HTML Report.
2. mSend : Smtp Client To Send Email. Installation Document

Scenario : If We have multiple Linux Server [Ubuntu or any Debian Server] and we want to get Email Notification for Server Health Status Report every day.

Server

1. Jumphost : From Jump host we can ssh any instance.
2. Remote Server : We will monitor this host health status.

Jumphost : We need to install mSend on Centralized host [Jumphost] and configure smtp details to get email notification. From this host we should able to ssh remote Server with Password less authentication. and ssh user must have password less sudo privilege. So that this user can generate html report perfectly.

Remote Server : This is Target Host to monitor health status. No Need to install any additional agent to monitor server. our script will install mSysmon utility to generate health status report.

Step — 1 Create Script file rSysmon.sh on Jumphost.

#!/usr/bin/env bash
# Before executing this script. You need to install mSend and configure to send email.
# Syntax :
# bash rSysMon.sh "user@hostip" "your_name@domain.com"
# bash rSysMon.sh "ubuntu@13.83.78.23" "myemail@gmail.com"
export msend_bin=$(which msend)
export REMOTE_HOST="${1:-127.0.0.1}"
export EMAIL_TO=${2:-HarryTheDevOpsGuy@gmail.com}
if [[ -z ${msend_bin} ]]; then
  echo "Please install mSend and configure."
  echo "Follow to Install : https://github.com/harry41/mSend"
  exit 1
fi
monitor(){
  # REMOTE_HOST="${1}"
  # EMAIL_TO=${2:-HarryTheDevOpsGuy@gmail.com}
  REPORT_NAME="ServerHealthReport"
  REPORT_FILE="/tmp/health_reports/${REPORT_NAME}.html"
  LOCAL_FILE="/tmp/${REPORT_NAME}.html"
  msysmon=$(ssh ${REMOTE_HOST} "which msysmon")
  if [[ ${msysmon} == "/usr/bin/msysmon" ]]; then
    ssh ${REMOTE_HOST} "msysmon '${REPORT_NAME}'"
    scp  ${REMOTE_HOST}:${REPORT_FILE} ${LOCAL_FILE}
    if [[ -f ${LOCAL_FILE} ]]; then
      grep 'SM_' ${LOCAL_FILE} > /tmp/remotevars
      source /tmp/remotevars
      ${msend_bin} -t "${EMAIL_TO}" -s "${SM_EMAIL_SUBJECT:- Server Health Report}" -f "${LOCAL_FILE}"
      rm -rf ${LOCAL_FILE} /tmp/remotevars
    fi
  else
    ssh ${REMOTE_HOST} "sudo curl -sL 'https://github.com/harry41/mSend/raw/master/tools/msysmon' -o /usr/bin/msysmon && sudo chmod +x /usr/bin/msysmon"
    ret_val=$?
    if [[ ${ret_val} != 0 ]]; then
      echo "Run Below command to install msysmon on remote server."
      echo "sudo curl -sL 'https://github.com/harry41/mSend/raw/master/tools/msysmon' -o /usr/bin/msysmon
      sudo chmod +x /usr/bin/msysmon"
    else
      echo "Please run again to Generate System Health Report"
    fi
  fi
}
if [[ -z ${REMOTE_HOST} || -z ${EMAIL_TO} ]]; then
  echo "Please pass valid hostname and email"
else
  echo "Please Wait.. : Connecting Remote host ${REMOTE_HOST}"
  monitor "${REMOTE_HOST}" "${EMAIL_TO}"
fi

Lets Test Script

bash rSysMon.sh "ubuntu@13.83.78.23" "myemail@gmail.com"
Or
./rSysMon.sh "ubuntu@13.83.78.23" "myemail@gmail.com"

You Need to pass hostip and your email to receive email notifications.