Hello folks:
I am sending my syslog-ng messages to kafka successfully. I am trying to figure out how I can set my macro names to use lowercase. I am talking about using "source" instead of "SOURCE" or "program" instead of "PROGRAM".
{"date":"1680889501","SOURCE":"s_net_kafka","PROGRAM":"1","PRIORITY":"info","MESSAGE":"2023-04-07T17:45:01.069987+00:00 pfsense.lab.com filterlog 49990 - - 4,,,1000000103,em1,match,block,in,4,0x0,,239,53158,0,none,17,udp,1383,203.121.204.46,10.10.15.193,6738,2055,1363","LEGACY_MSGHDR":"1 ","HOST_FROM":"10.10.10.1","HOST":"10.10.10.1","FACILITY":"local0","DATE":"Apr 7 17:45:01"}
Following is my config file:
@define kafka-implementation kafka-c
options {
keep-hostname(yes);
};
source s_net_kafka {
tcp(ip(0.0.0.0) port(514));
udp(ip(0.0.0.0) port(514) so_rcvbuf(26214400));
};
######################################################################################
destination d_kafka {
kafka(
bootstrap-servers("kafka:9092")
topic("syslog")
message("$(format-json --scope rfc5424 --scope nv-pairs --pair date=datetime($UNIXTIME))")
);
};
######################################################################################
log {
source(s_net_kafka);
destination(d_kafka);
};
I need timestamp fields represented as epoch time so I am inserting my own as part of the config file but I see that there is already a date field in the log message. Is there anyway to convert that field to epoch and avoid adding having another time field just to resolve formatting.
Thanks,
