Logging overview
mbox series (HSG, CMG, UA, HSA, mlog) have extensive support for user access and system audit logging, via Syslog. Syslogs are classified into different severity (Emergency, Alert, Critical, Error, Warning, Notice, Informational, debugging) and facilities, etc.
Many regulators, such as the Sarbanes-Oxley Act, PCI DSS, HIPAA, etc, require organizations to implement comprehensive security measures, which often include collecting and analyzing logs from various sources. Long time Syslog has become an industry standard for collecting logs from different sources. More details on Syslog can be searched online.
This document focuses on mbox Host’s support for security logging, either as a Syslog collector (eg. HSG/mlog) or a Syslog client. Depending on the deployment requirements, mbox can run in either (or both) of two modes (Syslog Collector/Syslog Client)
- Syslog collector. Syslog collector receives either self-originated or incoming logs from external hosts (log clients) via standard Syslog protocol, then the collector parses the received logs and inserts them into SQL database, making the binary logs reachable from intuitive GUI and ready for archival, etc.
- HSG by default has the ability to store user access logs locally, but with limited storage space. usually no more than 20GB is available for storing logs, which is generally enough to store archived user access logs for up to 90 days.
- We also have dedicated collector appliances to function as Syslog collectors (mlog series: LOG-500, LOG-1000, LOG-2000). mlog appliances are unique variants of mbox models with additional SSD/HDD storage capacity. mlog series are typically deployed as a central logging warehouse to consolidate logs from all devices within customer networks. Any devices supporting standard Syslog protocol can potentially export their logs to the mlog collector. NOTE, there’re 3rd-party opensource software converters to convert Windows Event Log into Syslog to export to mlog.
- HSG/mlog also comes with an intuitive log analyzer/GUI to display live incoming logs, with sophisticated searching functions for investigation and compliance reporting purposes.
- Syslog client. Syslog clients are basically devices generating messages in Syslog format and exporting the logs to an external Syslog server/collector.
NOTE
All mbox product families (CMG, HSG, HSA) can be configured as Syslog clients, track network packets, generate user access logs, and export as Syslog messages to local (in the case of HSG) or external Syslog server (mlog or other 3rd-party Syslog servers).
The below samples shows the different type of logs supported by mbox appliances.
- Firewall access logs. This is generated by the firewall module, by inspecting up to the transport layer (layer 4) of each packet. Below is a raw sample firewall log output.
Aug 30 13:45:31 CMG-ISP kernel: [5496992.470425] mboxfw-permit:IN=br0 OUT=eth0 PHYSIN=eth1 MAC=00:90:0b:34:b4:7f:00:90:0b:3e:05:0c:08:00 SRC=172.16.3.2 DST=49.128.58.66 LEN=52 TOS=0x00 PREC=0x00 TTL=60 ID=23565 DF PROTO=TCP SPT=58371 DPT=10051 WINDOW=29200 RES=0x00 SYN URGP=0
Aug 30 13:45:31 CMG-ISP kernel: [5496992.706739] mboxfw-permit:IN=br0 OUT=eth0 PHYSIN=eth2 MAC=00:90:0b:34:b4:7f:00:90:0b:3e:05:06:08:00 SRC=10.1.1.2 DST=49.128.58.66 LEN=52 TOS=0x00 PREC=0x00 TTL=63 ID=2490 DF PROTO=TCP SPT=49902 DPT=10051 WINDOW=29200 RES=0x00 SYN URGP=0
Aug 30 13:45:34 CMG-ISP kernel: [5496995.009301] mboxfw-permit:IN=br0 OUT=eth0 PHYSIN=eth1 MAC=00:90:0b:34:b4:7f:00:90:0b:3e:05:0c:08:00 SRC=172.16.3.2 DST=8.8.8.8 LEN=77 TOS=0x00 PREC=0x00 TTL=61 ID=17879 PROTO=UDP SPT=40809 DPT=53 LEN=57INFO:
The “MAC” option in the firewall log combines src/dst/type together.
For example below MAC value consists of 3 parts. Source MAC, Destination MAC, and frame type.
MAC=00:90:0b:34:b4:7f:00:90:0b:3e:05:0c:08:00
00:90:0b:34:b4:7f : Destination MAC=00:90:0b:34:b4:7f. This is the next-hop router MAC address.
00:90:0b:3e:05:0c : Source MAC=00:90:0b:34:b4:7f. This is the device (eg. user PC or phone) MAC address.
08:00 : Type=08:00 (ethernet frame carried an IPv4 datagram). This value is the same for all logs in an Ethernet environment.
- URL access logs. This is generated by web proxy by tracking each user browsing session, with the full URL path for each request.
This is applicable only for HTTP-based traffic. mbox proxy doesn’t intercept HTTPS traffic. As an alternative, you can consider DNS logging for tracking HTTPS requests, but unlike proxy logs, DNS logs don’t track the full URL path. Below is a raw sample URL log output:
04/May/2015:11:28:19 SGT 180 192.168.0.224 TCP_MISS/200 411 GET http://liveupdate.symantecliveupdate.com/minitri.flg - DIRECT/125.23.216.203 text/plain
04/May/2015:11:28:19 SGT 192.168.0.224 TCP_MISS/200 4083 GET http://liveupdate.symantecliveupdate.com/streaming/norton$202009$20streaming$20virus$20definitions_1.0_symalllanguages_livetri.zip - DIRECT/125.23.216.203 application/zip
04/May/2015:11:28:19 SGT 192.168.0.227 TCP_MISS/200 20670 GET http://www.youtube.com/watch? - DIRECT/209.85.231.136 text/html
04/May/2015:11:28:19 SGT 192.168.0.227 TCP_MISS/204 294 GET http://v15.lscache3.c.youtube.com/generate_204? - DIRECT/122.160.120.150 text/html- DNS access logs. This is enabled by default for HSG/CMG/HSA.
DNS log tracks all requests, for both http/https-based URL requests and all other applications (eg. even mobile apps requests), but not up to the full URL path. It’s a very effective method and is commonly used by many other products for user behavior analytics and URL filtering (eg. SafeDNS and OpenDNS). Below is a raw sample DNS log output
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.27.86 apple.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.27.86 p57-imap.mail.me.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.26.249 conn1.oppomobile.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.23.0 szextshort.weixin.qq.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.23.0 www.baidu.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.31.65 setup.icloud.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.23.0 www.youku.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.31.65 gspe35-ssl.ls.apple.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.22.220 43-courier.push.apple.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.21.254 encrypted-tbn0.gstatic.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.181.56.199 BCMLS2.glpals.com. A IN
Aug 30 13:54:02 mbox: [9906:0] info: 10.210.31.65 p50-ckdatabase.icloud.com. A IN- DHCP logs. DHCP logging is enabled by default. This log captures user device DHCP request and mbox offer/reply to the device, which is essential to track device NAME, MAC, and IP mapping.
Apr 8 13:07:42 HSG-DEMO dhcpd: DHCPREQUEST for 192.168.50.105 from 18:5e:0f:70:e2:02 (RandyRan) via vlan10
Apr 8 13:07:42 HSG-DEMO dhcpd: DHCPACK on 192.168.50.105 to 18:5e:0f:70:e2:02 (RandyRan) via vlan10
Apr 8 13:08:28 HSG-DEMO dhcpd: DHCPDISCOVER from 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:08:29 HSG-DEMO dhcpd: DHCPOFFER on 192.168.50.192 to 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:08:30 HSG-DEMO dhcpd: DHCPREQUEST for 192.168.50.192 (192.168.50.1) from 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:08:30 HSG-DEMO dhcpd: DHCPACK on 192.168.50.192 to 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:08:31 HSG-DEMO dhcpd: DHCPREQUEST for 192.168.50.154 from 50:c7:bf:90:2e:e0 (HS100) via vlan10
Apr 8 13:08:31 HSG-DEMO dhcpd: DHCPACK on 192.168.50.154 to 50:c7:bf:90:2e:e0 (HS100) via vlan10
Apr 8 13:11:06 HSG-DEMO dhcpd: DHCPDISCOVER from 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:11:07 HSG-DEMO dhcpd: DHCPOFFER on 192.168.50.192 to 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:11:08 HSG-DEMO dhcpd: DHCPREQUEST for 192.168.50.192 (192.168.50.1) from 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10
Apr 8 13:11:08 HSG-DEMO dhcpd: DHCPACK on 192.168.50.192 to 6c:4d:73:95:67:ae (seewees-iPhone) via vlan10- RADIUS session logs. RADIUS session logging is available for HSG only and it’s enabled by default. It captures user device authentication and connection requests on a per-connection basis.
Username User MAC IP Address Start Time Stop Time Total Time Download Upload
------------------------------------------------------------------------------------------------------------------------------------------------
demouser C4-9F-4C-F0-63-74 172.19.3.203 2020-08-13 18:13:49 2020-08-13 18:43:55 30min, 5sec 0.04 Mb 0.02 Mb
demouser F0-67-28-FE-AE-FB 172.19.2.74 2020-08-13 18:01:35 2020-08-13 18:35:36 34min, 1sec 9.07 Mb 0.36 Mb
demouser 8C-1A-BF-4A-6B-3E 172.19.2.18 2020-08-13 17:57:18 2020-08-13 19:21:14 1hrs, 23min, 81.28 Mb 3.3 Mb
demouser C4-06-83-A7-DF-39 172.19.3.202 2020-08-13 17:35:02 2020-08-13 18:08:13 33min, 11sec 0.23 Mb .12 Mb
demouser 62-62-C1-EA-11-47 172.19.2.182 2020-08-13 16:53:21 2020-08-13 17:28:10 34min, 48sec 4.98 Mb 0.26 Mb
demouser 04-D6-AA-2C-78-DB 172.19.3.239 2020-08-13 16:52:42 2020-08-13 17:26:54 34min, 11sec 0.38 Mb 0.14 Mb
demouser 20-F4-78-40-B2-2E 10.210.243.58 2020-08-13 16:46:25 2020-08-13 17:45:45 59min, 20sec 302.18 Mb 9.1 Mb
demouser 90-61-AE-54-1D-7A 10.210.242.92 2020-08-13 16:41:40 2020-08-13 18:13:54 1hrs, 32min, 167.38 Mb 23.89 Mb
demouser 80-AD-16-F8-EE-13 172.19.3.214 2020-08-13 16:09:25 2020-08-13 18:32:36 2hrs, 23min, 154.87 Mb 5.9 Mb
demouser 24-FB-65-6B-86-F3 172.19.3.247 2020-08-13 15:57:12 2020-08-13 17:35:54 1hrs, 38min, 197.52 Mb 13.99 Mb
demouser D4-A3-3D-2B-AD-BF 172.19.3.228 2020-08-13 15:52:14 2020-08-13 16:26:23 34min, 9sec 2.86 Mb 0.54 Mb
demouser B4-F6-1C-84-5C-48 10.210.243.220 2020-08-13 15:46:28 2020-08-13 16:09:52 23min, 24sec 2.07 Mb 1.52 Mb2. Configure log client (export logs. eg. HSG/CMG/HSA)
When a device is configured to export syslogs to an external Syslog server, we call it a Syslog client. HSG can function as both log server/collector and client.
NOTE
CMG and HSA work as log clients only
Different vendor products have their own syntax in tracking firewall access logs and enabling Syslog exports, please consult the respective product guides.
This section focuses on firewall access logging and how to export logs out for CMG/HSG/HSA.
Configuration steps for a logging client:
- Enable firewall access logging (CMG, HSG, HSA, UA).
NOTE
DNS logging, DHCP logging, and HSG RADIUS logging are enabled by default.
We use firewall-access rules to log each packet passing through mbox, eg.
User can use mfusion management and navigate to ‘ORCHESTRATOR > Configuration‘, click on the mbox host mac address and navigate to ‘Security Tab’ > ‘Access Link’ .
To add new rules click on the 
button to create a permit log rule.
Code to configure firewall-access rules to log
# firewall-access xx permit-log .......
or
# firewall-access xx deny-log .......It is important to know that HSG/HSA/UA maintains a separate set of firewall rules for each hotspot instance, so we enable logging using hotspot-access rules under each hotspot instance.
# hotspot-access xx permit-log
or
# hotspot-access xx deny-log- Configure Log-out Rules to Export Logs.
The log-output rule defines what type of logs to export out and to which servers (using the log-output xx command). If there’re multiple log-output rules, they work in top-down sequences.
log-output <acl> host <collector-ip> <filter>- <ACL Number> defines a sequence of output rules. It is like firewall rules, processed from top-down, once a log is matched with an upper rule, it will not be processed by lower rules. So it’s important to plan the rules sequence when we have many rules.
- <collector-ip> specifies the IP address of external Syslog collector (eg. LOG-500).
NOTE
If there’s a firewall in between, the firewall needs to open UDP/514 for the traffic to pass through.
- <filter> defines filtering rules based on Syslog fields to determine the matched logs to export. below is a list of available options:
- msg <text> filter by messages containing configured text
- fac <facility> filter by facility (eg. local1, local2, local3, local4…up to local7)
- prio filter by log priority/severity (eg. ALERT, NOTICE, INFO, etc), containing the configured priority.
- tag filter by Syslogtag, containing the configured text.
- all send all logs.
The log-output rule defines what type of logs to export out and to which servers (using the log-output xx command). If there’re multiple log-output rules, they work in top-down sequences.
log-output <acl> host <collector-ip> <filter>- <ACL Number> defines a sequence of output rules. It is like firewall rules, processed from top-down, once a log is matched with an upper rule, it will not be processed by lower rules. So it’s important to plan the rules sequence when we have many rules.
- <collector-ip> specifies the IP address of external Syslog collector (eg. LOG-500).
NOTE
If there’s a firewall in between, the firewall needs to open UDP/514 for the traffic to pass through.
- <filter> defines filtering rules based on Syslog fields to determine the matched logs to export. below is a list of available options:
- msg <text> filter by messages containing configured text
- fac <facility> filter by facility (eg. local1, local2, local3, local4…up to local7)
- prio filter by log priority/severity (eg. ALERT, NOTICE, INFO, etc), containing the configured priority.
- tag filter by Syslogtag, containing the configured text.
- all send all logs.
In real practice, if we are unsure which filter options to use, we use “all” first, then mbox will export out all the logs. After we study the logs from Syslog collector GUI and decide what field to use for filtering, then we can fine-tune the log-out rules for better control.