Installing NetForest with NetDiagnostics on Linux Machine

This is the installation manual for Cavisson Monitoring Agent (CMON) for Linux environment.  The CMON has two components:

  • ND Machine Agent, and
  • ND Java Application Agent.

Prerequisites

  1. ND Machine Agent should be running with the same user name as Java application user**.
  2. JDK 1.8 or later is required.
    • RPM installation will auto detect Java and set JAVA_HOME.
  3. For RPM installation the user needs root / sudo access

** If ND Machine Agent is not running with the same user then the user won’t be able to take thread dump and heap dump of the JVM.

Installation Steps

Below are the installation steps for ND with NF:

Step – 1: Set User

By default, ND Machine Agent service runs with ‘cavisson’ user, which is created as part of RPM installation.

To run ND Machine Agent with same user as Java application, set following environment variables:

export CMON_USER=<user_name>
export CMON_GROUP=<group_name>

Step – 2: Install using RPM

Install using following command:

rpm -ivh “cmon-nd-nf-4.1.7-55.x86_64.rpm”

By default, ND Machine Agent is installed in ‘/opt/cavisson’ directory. To install it in a different directory, use prefix argument of RPM:

rpm -ivh –prefix=/opt/cavisson “cmon-nd-nf-4.1.7-55.x86_64.rpm”

In this document, the following two variables are used:

  1. CAV_MON_HOME: The path where Machine agent is installed.
    • For example: /opt/cavisson/monitors
  1. ND_HOME: The path where Java application agent is installed.
    • For example: /opt/cavisson/netdiagnostics

Configuration

Below configuration is required for setting ND with NF:

ND Machine Agent Configuration

Make changes in the configuration details within the following file: $CAV_MON_HOME/sys/cmon.env

Auto Registration Configuration (Optional)
  • NDE Controller Address (Host and Port): Configure NDE Controller as following:

CONTROLLER=<host:port>

  • Tier Name: Configure Tier Name as following:

Tier=<Tier Name>

If the tier name is not defined, then the server will be the part of ‘Default’ tier.

JVM Configuration

Add following for Heap size setting and enabling JMX port of Machine Agent in CMON_OPTS.

"-Xms512m -Xmx512m -Dcom.sun.management.jmxremote.port=7991
-Dcom.sun.management.jmxremote.authenticate=false”
Sample cmon.env file contents

CMON_USER=cavisson
JAVA_HOME=/apps/java/jdk1.8.0_131
CAV_MON_AGENT_OPTS="-F 1"
CMON_OPTS="-Xms512m -Xmx512m -Dcom.sun.management.jmxremote.port=7991 -Dcom.sun.management.jmxremote.authenticate=false -Djffi.boot.library.path=/export/cavisson/monitors/nf/nfagent/vendor/jruby/lib/jni -Xbootclasspath/a:/export/cavisson/monitors/nf/nfagent/vendor/jruby/lib/jruby.jar -Djruby.home=/export/cavisson/monitors/nf/nfagent/vendor/jruby -Djruby.lib=/export/cavisson/monitors/nf/nfagent/vendor/jruby/lib -Djruby.script=jruby -Djruby.shell=/bin/sh"
CLASSPATH=$CLASSPATH:/export/cavisson/monitors/lib/jruby.jar:.

#NDE Controller Host and Port
CONTROLLER=10.10.40.13:7891

#Tier and Server
TIER=WebStore
SERVER=<ip address or host name of server>
Sample nf.properties file contents

NFAgent considers the config directory provided in nf.properties file under nfConfigDir. Default is conf.d.journald, change it to conf.d.files where we are reading logs from log files.

Example

nfEnv=nf/nfagent/lib/bootstrap/environment.rb
nfVersion=--1.9
nfConfigDir=nf/nfagent/config/conf.d.journald/
nfRunner=nf/nfagent/logstash-core/lib/nfagent/runner.rb

Restart ND Machine Agent

Restart ND Machine agent using following command:

service cmon restart
Or
systemctl restart cmon

ND Java Application Agent Configuration

Add following arguments in Java startup program:

-javaagent:/opt/cavisson/netdiagnostics/lib/ndmain.jar=time,
ndAgentJar=/opt/cavisson/netdiagnostics/lib/ndagent-with-dep.jar,
ndHome=/opt/cavisson/netdiagnostics,agentId=1

There may be multiple instances running on same server, assign unique agent ID for each like 2, 3, and so on.

For each instance, agent configuration file is created as below:

Create ndsettings_1.conf under $ND_HOME/config with example as below.

ndcHost=10.10.10.99
ndcPort=7891
tier=Webstore
server=10.10.10.5
instance=cav-stf-01

Where,

Serial NoParameterDescription
1TierThe tier of the application. For example AppTier, SearchTier
2ServerThe display or actual name of the server where the agent is running. This can take both hostname and IP
3InstanceThe name of the standalone application instance or managed instances. For example “cav-stf-01”
4ndcHostNetDiagnostics Collector IP or hostname
5ndcPortNetDiagnostics Collector port. Use 7891 unless some other port is used.

All fields are mandatory. However, if auto scaling is used, then Server and Instance fields can be omitted and NetDiagnostics Collector will assign these values.

Restart Instance

Once ND Java Application Agent is successfully installed, you need to restart the instance for NetDiagnostics (ND) to be enabled in JVM.

Verify whether ND argument is enabled, by using the following command:

ps –lef  | grep java

Following Output validates the proper installation of Java Agent.

stockadm  22732 22631  3 09:00 ?  
00:19:05 java -Xms512m -Xmx1024m -Xns256m -Xss512k 
-Dweblogic.Name=cav-stf-01 
-Djava.security.policy=/root/oracle/bea/weblogic10.3.5/server/lib/weblogic.policy 
-Dweblogic.ProductionModeEnabled=true 
-javaagent:/opt/cavisson/netdiagnostics/lib/ndmain.jar=time,ndAgentJar=
/opt/cavisson/netdiagnostics/lib/ndagent-with-dep.jar,ndHome=
/opt/cavisson/netdiagnostics,agentId=1 -da -Dplatform.home=/root/oracle/bea/weblogic10.3.5 
-Dwls.home=/root/oracle/bea/weblogic10.3.5/server 
-Dweblogic.home=/root/oracle/bea/weblogic10.3.5/server 
-Dweblogic.management.discover=true -Dwlw.iterativeDev=false 
-Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false 
-Dweblogic.ext.dirs=/root/oracle/bea/patch_wls1035/profiles/default/sysext_manifest_classpath:
/root/oracle/bea/patch_ocp360/profiles/default/sysext_manifest_classpath weblogic.Server

This concludes the successful installation of Java Application Agent.

NetForest Configuration

follow below mentioned steps for NetForest Configuration:

Reading NFAgent Configuration (parser) from NFDB (Default)

To read configuration from NFDB, we have to provide NFDB URL in $CAV_MON_HOME/nf/nfagent/config/nfagent.yml file.

config.store.nfdb: true
config.hosts: ["localhost:9200"]

Note:

  1. If store.nfdb is false, configuration are read from files (see below section).
  2. hosts can have comma separated host list in case of multinode clusters.

For example: config.hosts: [“localhost:9200”, “10.10.10.1:9200”]

Reading NFAgent Configuration (parser) From Files

config.store.nfdb: false

File configurations are stored in $CAV_MON_HOME/nf/nfagent/config/conf.d.files.

Note:

In case of failures i.e. NFAgent is unable to connect NFDB for configuration or configuration index is not present etc., it takes configurations from files as fallback.

Add environment specific information in $CAV_MON_HOME/sys/nf.env file.

SERVER=<server>
DC=<dc>
ENV=<env>
INDEX_PREFIX=<prefix>
# If it's false only OUTPUT_HOST1 and OUTPUT_PORT1 is considered
OUTPUT_MULTINODE=false
OUTPUT_HOST1=<host1>
OUTPUT_PORT1=<port1>

Note:

  1. TIER is taken from cmon.env.
  2. In case of cloud installations, SERVER and ENV is taken from Journald not from nf.env.
  3. In case OUTPUT_MULTINODE is false (i.e. nfdb cluster has only one node), OUTPUT_HOST1 and OUTPUT_PORT1 is considered, and rest are ignored.
  4. Server is optional for cloud environment. It takes hostname as server.

Reload NFAgent

In case of any changes in nf.env or in NFDB configuration index, reload the NFAgent config.

Use below steps to reload NFAgent:

  1. nc <ND Machine Agent ip> <ND Machine Agent port> (set port to 7891)
  2. START_SESSION
  3. NetForestCMD:RELOAD
  4. STOP_SESSION

 This concludes the installation of NFAgent.