Installing and Managing NGINX Amplify Agent
Installing the Agent
In order to be able to use NGINX Amplify to monitor your infrastructure, you need to install NGINX Amplify Agent on each system that has to be checked.
Note. The agent will drop root privileges on startup. It will then use the user ID of the user nginx to set its effective user ID. The package install procedure will add the nginx user automatically unless it's already found in the system. If the user directive appears in the NGINX configuration, the agent will pick up the user specified in the NGINX config for its effective user ID (e.g. www-data). Using the Install Script
The installation procedure can be as simple as this.
Download and run the install script.
# curl -sS -L -O \
https://github.com/nginxinc/nginx-amplify-agent/raw/master/packages/install.sh && \
API_KEY='YOUR_API_KEY' sh ./install.sh
where YOUR_API_KEY is a unique API key assigned to your Amplify account. You will see the API key when adding a new system in the Amplify web interface. You can also find it in the Account menu.
Verify that the agent has started.
# ps ax | grep -i 'amplify\-'
2552 ? S 0:00 amplify-agent
Installing the Agent Manually
Installing on Ubuntu or Debian
Add the NGINX public key.
# curl -fs http://nginx.org/keys/nginx_signing.key | apt-key add -
or
# wget -q -O - \
http://nginx.org/keys/nginx_signing.key | apt-key add -
Configure the repository as follows.
# codename=`lsb_release -cs` && \
os=`lsb_release -is | tr '[:upper:]' '[:lower:]'` && \
echo "deb http://packages.amplify.nginx.com/${os}/ ${codename} amplify-agent" > \
/etc/apt/sources.list.d/nginx-amplify.list
Verify the repository config file (Ubuntu 14.04 example follows).
# cat /etc/apt/sources.list.d/nginx-amplify.list
deb http://packages.amplify.nginx.com/ubuntu/ trusty amplify-agent
Update the package index files.
# apt-get update
Install and run the agent.
# apt-get install nginx-amplify-agent
Installing on CentOS, Red Hat Linux, or Amazon Linux
Add the NGINX public key.
# curl -sS -L -O http://nginx.org/keys/nginx_signing.key && \
rpm --import nginx_signing.key
or
# wget -q -O nginx_signing.key http://nginx.org/keys/nginx_signing.key && \
rpm --import nginx_signing.key
Create the repository config as follows (mind the correct release number).
Use the first snippet below for CentOS and Red Hat Linux. The second one applies to Amazon Linux.
# release="7" && \
printf "[nginx-amplify]\nname=nginx amplify repo\nbaseurl=http://packages.amplify.nginx.com/centos/${release}/\$basearch\ngpgcheck=1\nenabled=1\n" > \
/etc/yum.repos.d/nginx-amplify.repo
# release="latest" && \
printf "[nginx-amplify]\nname=nginx amplify repo\nbaseurl=http://packages.amplify.nginx.com/amzn/${release}/\$basearch\ngpgcheck=1\nenabled=1\n" > \
/etc/yum.repos.d/nginx-amplify.repo
Verify the repository config file (RHEL 7.1 example follows).
# cat /etc/yum.repos.d/nginx-amplify.repo
[nginx-amplify]
name=nginx repo
baseurl=http://packages.amplify.nginx.com/centos/7/$basearch
gpgcheck=1
enabled=1
Update the package metadata.
# yum makecache
Install and run the agent.
# yum install nginx-amplify-agent
Creating the Config File from a Template
# api_key="YOUR_API_KEY" && \
sed "s/api_key.*$/api_key = ${api_key}/" \
/etc/amplify-agent/agent.conf.default > \
/etc/amplify-agent/agent.conf
API_KEY is a unique API key assigned to your Amplify account. You will see your API key when adding a new system in the Amplify web interface. You can also find the API key in the Account menu.
Starting and Stopping the Agent
# service amplify-agent start
# service amplify-agent stop
# service amplify-agent restart
Verifying that the Agent Has Started
# ps ax | grep -i 'amplify\-'
2552 ? S 0:00 amplify-agent
Updating the Agent
It is highly recommended that you periodically check for updates and install the latest stable version of the agent.
On Ubuntu/Debian use:
# apt-get update && \
apt-get install nginx-amplify-agent
On CentOS/Red Hat use:
# yum makecache && \
yum update nginx-amplify-agent
Note. Starting from version 1.8.0 agent is running on Python 3 on supported platforms. In order to keep existing configurations in working state and avoid mixing of older Python 2 agent packages with new Python 3 ones, we've placed new packages in the dedicated location /py3 of the Amplify agent packages repository. Please adjust your local repository configuration accordingly if you are upgrading from versions < 1.8.0. Install script has been updated to configure new installations to point to the appropriate location automatically. Using the Agent with Docker
You can use NGINX Amplify Agent in a Docker environment. Although it's still work-in-progress, the agent can collect most of the metrics, and send them over to the Amplify backend in either "standalone" or "aggregate" mode. The standalone mode of operation is the simplest one, where there's a separate "host" created for each Docker container. Alternatively the metrics from the agents running in different containers can be aggregated on a "per-image" basis — this is the aggregate mode of deploying the Amplify Agent with Docker.
Configuring the Agent
NGINX Amplify Agent keeps its configuration in /etc/amplify-agent/agent.conf. The agent configuration is a text-based file.
Overriding the Effective User ID
NGINX Amplify Agent will drop root privileges on startup. By default it will then use the user ID of the user nginx to set its effective user ID. The package install procedure will add the nginx user automatically unless it's already found in the system. If the user directive appears in the NGINX configuration, the agent will pick up the user specified in the NGINX config for its effective user ID (e.g. www-data). It is really important for the agent and the running NGINX instances to use the same user ID, so that the agent is able to properly collect all NGINX metrics.
In case you'd like to manually specify the user ID that the agent should use for its effective user ID, there's a specialized section in /etc/amplify-agent/agent.conf for that:
[nginx]
user =
configfile = /etc/nginx/nginx.conf
There's an option here to explicitly set the real user ID which the agent should pick for its effective user ID. If the user directive has a non-empty parameter, the agent startup script will use it to look up the real user ID.
In addition, there's another option to explicitly tell the agent where it should look for an NGINX configuration file suitable for detecting the real user ID. It's /etc/nginx/nginx.conf by default.
Changing the API Key
When you first install the agent using the procedure above, your API key is written to the agent.conf file automatically. If you ever need to change the API key, please edit the following section in agent.conf accordingly:
[credentials]
api_key = YOUR_API_KEY
Changing the Hostname and UUID
In order to create unique objects for monitoring, the agent must be able to extract a valid hostname from the system. The hostname is also utilized as one of the components for generating a unique identifier. Essentially, the hostname and the UUID unambiguously identify a particular instance of the agent to the Amplify backend. If the hostname or the UUID are changed, the agent and the backend will register a new object for monitoring.
When first generated, the uuid is written to agent.conf. Typically this happens automatically when the agent starts and successfully detects the hostname for the first time. Normally you shouldn't change the UUID in agent.conf.
The agent will try its best to determine the correct hostname. If it fails to determine the hostname, you can set the hostname manually in the agent.conf file. Check for the following section, and put the desired hostname in here:
[credentials]
..
hostname = myhostname1
The hostname should be something real. The agent won't start unless a valid hostname is defined. The following aren't valid hostnames:
- localhost
- localhost.localdomain
- localhost6.localdomain6
- ip6-localhost
Note. You can also use the above method to replace the system's hostname with an arbitrary alias. Keep in mind that if you redefine the hostname for a live object, the existing object will be marked as failed in the web interface. Redefining the hostname in the agent's configuration essentially creates a new UUID, and a new system for monitoring.
Alternatively you can define an "alias" for the host in the UI (see the Graphs section below). Configuring the URL for stub_status or Status API
When the agent finds a running NGINX instance, it automatically detects the stub_status or the NGINX Plus API module locations from the NGINX configuration. To override the stub_status URI/URL, use the stub_status configuration option.
[nginx]
..
stub_status = http://127.0.0.1/nginx_status
To override the URI detection of the status API, use the plus_status option.
[nginx]
..
plus_status = /status
Note. If only the URI part is specified with the options above, the agent will use http://127.0.0.1 to construct the full URL to access either the stub_status or the NGINX Plus status API metrics.
Configuring the Path to the NGINX Configuration File
The agent detects the NGINX configuration file automatically. You don't need to explicitly point the agent to the nginx.conf.
If for some reason the agent is not able to find the NGINX configuration, use the following option in /etc/amplify-agent/agent.conf:
[nginx]
configfile = /etc/nginx/nginx.conf
Note. It is better to avoid using this option and only add it as a workaround. We'd appreciate if you take some time to fill out a support ticket in case you had to manually add the path to the NGINX config file.
Configuring Host Tags
You can define arbitrary tags on a "per-host" basis. Tags can be configured in the UI (see the Graphs section below), or set in the /etc/amplify-agent.conf file: [tags]
tags = foo,bar,foo:bar
You can use tags to build custom graphs, configure alerts, and filter the systems on the Graphs page. Configuring Syslog
The agent can collect the NGINX log files via syslog. This could be useful when you don't keep the NGINX logs on disk, or when monitoring a container environment such as Docker with NGINX Amplify. To configure the agent for syslog, add the following to the /etc/amplify-agent/agent.conf:
[listeners]
keys = syslog-default
[listener_syslog-default]
address = 127.0.0.1:12000
Restart the agent to have it reload the configuration and start listening on the specified IP address and port:
# service amplify-agent restart
Make sure to add the syslog settings to your NGINX configuration as well. Excluding Certain NGINX Log Files
By default the agent will try to find and watch all access.log files described in the NGINX configuration. If there are multiple log files where the same request is logged, the metrics may get counted more than once.
To exclude specific NGINX log files from the metric collection, add something along these lines to /etc/amplify-agent/agent.conf:
[nginx]
exclude_logs=/var/log/nginx/app1/*,access-app1-*.log,sender1-*.log
Setting Up a Proxy
If your system is in a DMZ environment without direct access to the Internet, the only way for the agent to report collected metrics to Amplify would be through a proxy.
The agent will use the usual environment variables that are common on Linux systems (e.g. https_proxy or HTTP_PROXY). However, you can also define HTTPS proxy manually in agent.conf. This could be done as follows:
[proxies]
https = https://10.20.30.40:3030
..
Agent Logfile
The agent maintains its log file in /var/log/amplify-agent/agent.log
Upon installation, the agent's log rotation schedule is added to /etc/logrotate.d/amplify-agent
The normal level of logging for the agent is INFO. If you ever need to debug the agent, change the level to DEBUG as described below. Bear in mind, the size of the agent's log file can grow really fast with DEBUG. After you change the log level, please restart the agent.
[logger_agent-default]
level = DEBUG
..
[handler_agent-default]
class = logging.handlers.WatchedFileHandler
level = DEBUG
..
Uninstalling the Agent
To completely delete a previously monitored object, perform the following steps:
Uninstall the agent
On Ubuntu/Debian use:
apt-get remove nginx-amplify-agent
On CentOS and Red Hat use:
yum remove nginx-amplify-agent
Delete objects from the web interface
To delete a system using the web interface — find it in the Inventory, and click on the "trash" icon. Bear in mind — deleting objects in the UI will not stop the agent. To completely remove a system from monitoring, stop and/or uninstall the agent first, and then clean it up in the web interface. Don't forget to also clean up any alert rules.
Delete alerts
Check the Alerts page and remove/mute the irrelevant rules.