Troubleshooting Syslog-NG

Basic troubleshooting

Syslog-ng has an advanced internal logging mechanism. In case of event info, warning, error, fatal or debug messages are generated.  All messages generated internally by syslog-ng use a special source called internal(). Syslog-ng is shipped with a configuration which collects the internal logs into /var/log/messages

If you're unsure, find the source uses internal() driver. Search for log statement uses the "internal" source and check the destination.

Start the investigation by checking for issues in internal logs, /var/log/messages.

egrep -i 'syslog-ng.*(error|fatal)' /var/log/messages

Verbose logging

If you are trying to solve configuration problems, the verbose messages are usually sufficient.

You can turn on verbose logging on a running syslog-ng instance with the following command.

/opt/syslog-ng/sbin/syslog-ng-ctl verbose --set=on

When you have finished, you can turn off verbose logs with:

/opt/syslog-ng/sbin/syslog-ng-ctl verbose --set=off

Debug logging

In certain situations you may need to analyze debug messages, for example to troubleshoot connection issues or to verify message processing (only syslog-ng PE6).

You can turn on debug logging on a running syslog-ng instance with the following command.

/opt/syslog-ng/sbin/syslog-ng-ctl debug --set=on

When you have finished turn off debug mode with:

/opt/syslog-ng/sbin/syslog-ng-ctl debug --set=off

Debug logging can produce a huge amount of logs. It is strongly recommended to turn it on only for short troubleshooting sessions.

Trace logging for syslog-ng PE 7

You can turn on trace logging (message processing) on a running syslog-ng PE 7 instance with the following command.

/opt/syslog-ng/sbin/syslog-ng-ctl trace --set=on

When you have finished turn off trace mode with:

/opt/syslog-ng/sbin/syslog-ng-ctl trace --set=off

Trace logging produces a huge amount of logs. It is strongly recommended to turn it on for short troubleshooting sessions.

Starting syslog-ng in debug mode

When your syslog-ng instance fails to start for some reason, you can start it up in debug mode. It is also useful to troubleshoot environmental issues, for example in case of a java destination.

Note, that if you are running syslog-ng with a non-root user, you have to run the following commands with the appropriate user.

If that user does not have permission to open a terminal, go to next section.

Make sure that no syslog-ng instance is running and execute:

/opt/syslog-ng/sbin/syslog-ng -Fedv

Press CTRL + C to terminate syslog-ng, if it was not terminated by itself.

Redirecting the output to a file

Debug mode generates huge amount of log messages. It's recommended to redirect the output to a log file.

/opt/syslog-ng/sbin/syslog-ng -Fedv 2>&1 | tee -a syslog-dbg.log

Starting syslog-ng in debug mode with non-root user without shell

Depending on your system, one of the following methods should work.

With sudo

sudo -u USERNAME /opt/syslog-ng/sbin/syslog-ng -Fedv

With su

su - USERNAME -c "/opt/syslog-ng/sbin/syslog-ng -Fedv"

With systemd

Modify systemd service file located at /usr/lib/systemd/system/syslog-ng.service or /lib/systemd/system/syslog-ng.service, depending on your Linux distribution.

  • Find ExecStart= parameter in [Service] section and add debug options '-d -v'.
ExecStart=/opt/syslog-ng/sbin/syslog-ng -F --no-caps --enable-core $SYSLOGNG_OPTIONS -d -v
  • Reload systemd configuration
systemctl daemon-reload
  • Restart syslog-ng
systemctl restart syslog-ng
  • Check logs
journalctl -u syslog-ng