This article explain how to connect syslog-ng to Apsolab-Server. With syslog-ng there are 3 options for sending logs to Apsolab-Server.

  1. Direct: Configure all remote server (running syslog-ng, rsyslog and syslog) to forward logs to Apsolab-Server and let them parse syslog raw data..
  2. Agentless: Configure all remote server (running syslog-ng, rsyslog and syslog) to forward logs to local syslog-ng (on Apsolab-Server computer).
  3. Agent: Configure each instance of syslog-ng to forwards logs to local XLog-Agent and let XLog-Agent connect with Apsolab-Server.

The first option is more easy to implement.

The second option doesn’t require to install any agent on remote host. But it as some drawback.

  • If you need to change log threshold (filtering), you will have to modify syslog-ng configuration file and restart the service.
  • To avoid filtering modifications, you can forward all logs to Apsolab-Server and let this one drop logs with less priority than selected threshold. This option can cause heavy traffic on the network.

The 3rd option require that you install Apsolab-Agent on remote host. With this option, you configure syslog-ng to send all logs to local Apsolab-Agent and let this one decide which logs to forward to Apsolab-Server and which logs to drop. This option gives the flexibility to change logger threshold from Apsolab-Console without having to change anything on remote host. And your network won’t be filled with useless data.

Check your firewall and selinux. They must allow access to TCP port 5470 and/or 5472 (or any other port you may have configured Apsolab-Server).

 Option 1: Direct

Configure Apsolab-Server to enable syslog parser. Connect to Apsolab-Console as ‘root’ (the admin.) and go to Administration/Settings and then to Services tab. At the bottom, there is a [syslog] section. You can configure one or many UDP ports. Yes, Apsolab-Server can listen for syslog data on many UDP ports. Each port will have its own logger name. This is useful to control log level (threshold) and conduct search query. The port syntax is: port[,port][,port]. Simply add comma between port numbers. Make sure your firewall and selinux are configured accordingly. You don’t have to restart Apsolab-Server but make sure to click on Apply Services.

Open syslog-ng.conf file with some editor. Add the following lines at end of the file:

# Define destination for Apsolab-Server host
destination d_apsolab_server {
    udp("127.0.0.1" port(10514));
};
# Forward to Apsolab-Server.
log {
    source(s_all);
    destination(d_apsolab_server);
};

In this example, we forward logs to localhost on UDP port 10514. Change those values as appropriate.

To make your change effective, restart syslog-ng service.

From this point, you should see a new logger in Apsolab-Console (syslog-10514). You can control log level (threshold) from Logger window.

 Option 2: Agentless

Configure all remote syslog-ng to forward logs to syslog-ng located on Apsolab-Server computer. The following text is an example of syslog-ng configuration to forward logs.

# Local log source
source s_local {
    file ("/proc/kmsg" log_prefix("kernel: "));
    unix-stream ("/dev/log");
    internal();
};
# Define destination for Apsolab-Server host
    destination d_xlogserver {
    tcp("xlogserver");
};
# Forward all logs to Apsolab-Server host.
log {
    source(s_local);
    destination(d_xlogserver);
};

Configure syslog-ng on Apsolab-Server computer.
You can copy/paste the following lines into syslog-ng.conf file and make the appropriate changes. This information is also available under script/syslog-ng. If you copy from this page, make sure the template is contained on single line.

# Accept local logs
source s_local {
    file ("/proc/kmsg" log_prefix("kernel: "));
    unix-stream ("/dev/log");
    internal();
};
# Accept logs from tcp on port 514
source s_port {
    tcp(ip(0.0.0.0) port(514));
};
# Define filter for Apsolab-Server
filter f_xlog {
    level(debug..emerg);
};
# Define Apsolab-Server Template (category: system). Note: replace localhost within loggername tag with local host name.
template t_system{
    template ("<xlog><loggertype>syslog</loggertype><category>system</category><loggername>syslog-localhost</loggername><threshold>info</threshold> <timestamp>$UNIXTIME</timestamp><facility>$FACILITY</facility><host>$HOST</host><sourceip>$SOURCEIP</sourceip> <program>$PROGRAM</program><pid>$PID</pid><pri>$PRI</pri><msg>$MSGONLY</msg></xlog>\n");
    template_escape(no);
};
# Define Apsolab-Server destination. Note: your host name and/or port number may be different.
destination d_xlogserver {
    tcp("localhost" port(5472) template(t_system));
};
# Connect with Apsolab-Server.
log {
    source(s_local);
    source(s_port);
    filter(f_xlog);
    destination(d_xlogserver);
};

IMPORTANT: The ‘loggername’ must be unique within Apsolab-Server domain. So, it is recommended to use something like ‘syslog-hostname’ as logger name. Change ‘hostname’ for the real computer host name.

To make your change effective, restart syslog-ng service.

If you want to assign a different category based on source and/or filter, you will have to create one template and one destination for each category. Then create one log {…}; per category.

Ex: router with ip 10.1.1.1

source s_port{
    tcp(ip(0.0.0.0) 
    port(514));
};

filter f_router{
    netmask(10.1.1.1/32);
};

template t_telecom {
    template("<xlog><category>telecom</category><loggername>telecom</loggername> ... </xlog>");
    template_escape(no); 
}; 

destination d_telecom { 
    tcp("localhost" port(33201) template(t_telecom)); 
}; 

log { 
    source(s_port); 
    filter(f_router); 
    destination(d_telecom); 
};

Option 3: Use XLog-Agent to forward syslog-ng logs to Apslab-Server

If you choose this approach, you will have to deploy xlogagent binary on each remote system. You can find the agent in /opt/apsolab/latest/bin.
You can copy/paste the following lines into syslog-ng.conf file and make the appropriate changes. This information is also available under /opt/apsolab/latest/script/syslog-ng. If you copy from this page, make sure the template is contained on single line.

# Local log source
source s_local {
   file ("/proc/kmsg" log_prefix("kernel: "));
   unix-stream ("/dev/log");
   internal();
};

# Define Apsolab-Server Template (category: system).
template t_system{
    template ("<xlog><loggertype>syslog</loggertype><category>system</category><threshold>info</threshold><timestamp>$UNIXTIME</timestamp><facility>$FACILITY</facility><host>$HOST</host><sourceip>$SOURCEIP</sourceip><program>$PROGRAM</program><pid>$PID</pid><pri>$PRI</pri><msg>$MSGONLY</msg></xlog>\n");
    template_escape(no);
};

# Define XLog-Agent destination. Note: your path may be different.
destination d_xlog {
    program("cd /opt/apsolab/latest/bin; ./xlogagent -d debug -t /var/log/xlogagent-syslog-ng.log -l syslog-{hostname} -x xlogserver -p 5470 >/dev/null" template(t_system));
};

# Connect with XLog-Agent.
log {
    source(s_local);
    destination(d_xlog);
};

Important: The ‘loggername’ must be unique within Apsolab-Server domain. So, it is recommended to use something like ‘syslog-hostname’ as logger name. In the previous example, we use {hostname} tag within logger name. XLog-Agent will do the replacement (change ‘{hostname}’ for local host name).
Apsolab-Server host name is specified with -x option.
XLog-Agent connect with Apsolab-Server via the native service (default port 5470);

To make your change effective, restart syslog-ng service.