Forward the original raw message to a SIEM system

Hi, folks at Balabit/One Identity,

We would like to configure Syslog-ng PE forward the original raw message as it is been received, to a SIEM system.

How should we /opt/syslog-ng/etc/syslog-ng.conf for the source and destination that Syslog-ng PE does not tamper with the original syslog header and message content.

Thank you!

Cheers

Parents
  • Hello,

    Generally it can be done by configuring the source and the destination.

    Add flags(no-parse) option to the source on which you receive the message. That will put the whole incoming log into the MESSAGE macro and syslog-ng generates a syslog header for it.

    support.oneidentity.com/.../para-flags-no-parse

    Add template("${MESSAGE}") option to the SIEM destination that way only the $MESSAGE part will be sent to the SIEM, that is the original message.

    https://support.oneidentity.com/technical-documents/syslog-ng-premium-edition/7.0.18/administration-guide/64#TOPIC-1341569

    Consider, that the modification of the source affects the message processing of all logs.

  • Hello,

    Thanks for the advice~!

    Is this the correct way to configure as such that Syslog-ng PE does not tamper with the original syslog header and message content?

    options {
    stats_freq(0);
    keep-hostname(no);
    chain-hostnames(no);
    use-dns(no);
    };
    
    source from_windowsagent { 
       network ( 
          ip(0.0.0.0) 
          port(601) 
          transport("tcp") 
          flags(no-parse) 
          ); 
    }; 
    
    destination d_siem { 
       syslog ( 
          "x.x.x.x" 
          port(514) 
          transport("tcp") 
          ip-protocol(4) 
          template("${MESSAGE}") 
          ); 
    }; 
    
    log { 
       source(from_windowsagent); 
       destination(d_siem); 
    };

  • You may need the network() driver for the SIEM instead of syslog(), but the config seems to be correct.

  • Acknowledge on your advice to use network() driver instead of syslog()~

    Can we understand what is the different between flags(no-parse) and flags(store-raw-message flag)?

    From the description in the admin guide, it seem to me that both can achieve the objective of sending the original syslog header and message content.

    no-parse: By default, syslog-ng PE parses incoming messages as syslog messages. The no-parse flag completely disables syslog message parsing and processes the complete line as the message part of a syslog message. The syslog-ng PE application will generate a new syslog header (timestamp, host, and so on) automatically and put the entire incoming message into the MESSAGE part of the syslog message (available using the ${MESSAGE} macro). This flag is useful for parsing messages not complying to the syslog format.

    If you are using the flags(no-parse) option, then syslog message parsing is completely disabled, and the entire incoming message is treated as the ${MESSAGE} part of a syslog message. In this case, syslog-ng PE generates a new syslog header (timestamp, host, and so on) automatically. Note that since flags(no-parse) disables message parsing, it interferes with other flags, for example, disables flags(no-multi-line)

    store-raw-message: Save the original message as received from the client in the ${RAWMSG} macro. You can forward this raw message in its original form to another syslog-ng node using the syslog-ng() destination, or to a SIEM system, ensuring that the SIEM can process it. Available only in 7.0.9 and later.

  • If the original message is a syslog formatted message you don't need any of these options.
    The original message can be forwarded by using keep-hostname(yes) and keep-timestamp(yes) on the source.

    But, if the log is non-syslog formatted then you have the two options to forward the original message.

    1. no-parse
      The original message will be placed in the MESSAGE part of the outgoing log message. So the receiver can process the original message by working only on the MESSAGE part.
    2. store-raw-message
      This option does not modify the format of the log message but places the original log in the ${RAWMSG} macro.
      With that you can forward the original message to the remote side if needed with the template("${RAWMSG}") configured in the destination.
      Also, you can forward it via syslog() and put the original message in the SDATA.
Reply
  • If the original message is a syslog formatted message you don't need any of these options.
    The original message can be forwarded by using keep-hostname(yes) and keep-timestamp(yes) on the source.

    But, if the log is non-syslog formatted then you have the two options to forward the original message.

    1. no-parse
      The original message will be placed in the MESSAGE part of the outgoing log message. So the receiver can process the original message by working only on the MESSAGE part.
    2. store-raw-message
      This option does not modify the format of the log message but places the original log in the ${RAWMSG} macro.
      With that you can forward the original message to the remote side if needed with the template("${RAWMSG}") configured in the destination.
      Also, you can forward it via syslog() and put the original message in the SDATA.
Children
No Data