Also, the reason you may not have seen your test message come through when you listened on the OMS agent port (around 24:02) is that the tcpdump command by default listened to eth0 but the traffic to the OMS agent probably went through the loopback interface (lo) because I imagine the OMS agent is listening on localhost only :)
Your exactly right. This ensures the OMS agents don’t listen remotely to those ports so makes complete sense later on after I posted the video lol. Good catch! Also if your interested. You can also program additional ports on the OMS agent to send logs directly to a custom log table! Like 25227 to goto a named custom table and use Rsyslog to convert the message to match that new custom table schema similar to how we handle CEF on 25226. Ubiquiti connector does just this.
@@TeachJing that is super awesome to know! One thing that is bugging me is how long Sentinel takes to process a new source and see it as connected. Syslog via OMS has taken over an hour then magically appeared. Do you know how you can use the OMS config file in /etc/rsyslogd/ to send all messages except when they contain a phrase? I’ve seen your videos on exclusive includes but wondered about excludes. Currently I’ve put a file in /etc/rsyslod.d/10-exclusions.conf which has an if statement with a stop when the condition matches, however. I’m worried this will stop the system locally logging it, instead of just not sending it to the OMS agent. I think Sentinel could do with kind of configuration on the Syslog Workspace configuration instead of just syslog facility and priority level!
If your worried about local logging. Locate the entry that logs it locally which is typically the Rsyslog.conf. Move that over to your 10 file or put a file in front of it that logs it all and Rsyslog knows to handle it first. Then your 10 file excludes and then your OMS gets the leftover. So again 5 file to do local logging 10 file excludes OMS file handles the logs that have been filtered If you look at Rsyslog.conf you will notice the include Rsyslog.d gets brought it AFTER local logging so I think your ok, but try it out and come back to me with sample logger commands and let me know. I’m actually almost done building a fully containerized docker container with all kinds of scenarios with Logstash, kafka, file logging to mess with and see how it flows. Stay on the lookout for that in the near future.
@@TeachJing ha! I feel so stupid now because that makes perfect sense. Log to disk and then have a code block after with an if statement. Awesome, thanks for your help. I’ve come across your content in the past few days and you’ve helped my get Sentinel spun up with logging in my dev e5 tenant! Looking forward to more content from you!
@@nathanwebb2800 Yes! All this I learned bro through trial and error. If you really want to be a fancy, Add a few variables like host to that log path and boom, all your logs from each host goes into their own file into a auth folder. Like /var/log/auth/.log Then you don't have to tail auth but just tail the respective host file. Throw in some log rotation with wildcard and ensures those disks never get full but you can still log locally. That is also how I troubleshoot as if I don't see a file... then that host isn't coming in :D
Love the content! A lot was over my head, but I'll work on my Linux skills to get a better understanding of what you were doing. I don't know VI or many of the commands you used but will work on learning them. I'm also working on learning jupyter notebooks, so anything that incorporates the notebooks with Sentinel would be awesome. I am trying to become our SME for Sentinel but I am just getting started with SIEM's so anything covering Sentinel is a bonus. Your video has been the best coverage I have seen and I've seen a lot. I'm working through the Ninja course but what you covered is some much more useful then anything I've done in that course. Either way, continue the great work and I'm looking forward to future videos. I'm going to watch your P2 on CEF and your KQL course. Cheers!
Thank for this great Syslog tutorial. I have a question to Configuration of OMS agent. I am trying to configure 2 connectors, where one connector "Eset Security Management Center (Preview)" needs data in API format. My problem is that other connector "Cisco ASA" stops working if I update section "" and change value "type out_oms" to "type out_oms_api" in /etc/opt/microsoft/omsagent/{yourworkspaceid}/conf/omsagent.conf. Can you please give me some hint how to configure this two connectors to live together?🙂 Many thanks Martin
Amazing video Jing! My one question is how do you stop CEF logs from feeding into the Syslog table? I am currently working on that, and it's causing me some trouble.
@@RIYADMURAD docs.microsoft.com/en-us/azure/azure-monitor/agents/data-sources-custom-logs. This site explains a little, but you will want to first figure out how you can parse those logs. If they are syslog format where you can read it in Rsyslog, you can simply save logs to a new file and then setup a custom connector that will basically listen on another port but is directed to another table. Customlog --> specialOMSAgentPort (ex:25229), This special connector will point to your custom table you created and your done. Have rsyslog then forward traffic matching that message to this new port you setup. Or you can use logstash to read from a custom log file you have parsed out and upload that to sentinel.
Great video! I'm just starting to look into Sentinel as a possible SIEM for our company. Can the syslog/collector be setup on a Windows server? We're not a Linux shop, and thus have zero experience with Linux and would rather not bring in an OS we know little about. All new maintenance, updates, and security concerns. Thanks.
just wanted to add, i think you are running tcpdump on port 25224 which uses eth 0 default , it should be interface lo. That could be the reason why you didnt received the test msg.
This is all cool, but now Microsoft says that workspace agent configuration method is legacy. Would love to see if there is a solution to push this to Sentinel that is future proof beyond August 2024
If I have to configure log collector on cloud but my machines are on-prem (in my lab) . How do I tell my machines it has to forward lags to machine in cloud
man how did the log just show up on sentinel? when did you even do the config and set the sentinel ip, how does your logger know where is the sentinel ip? its like these videos just assume that people watching know most of the stuff lol
Hi TeachJing, I learned a lot on your videos. I'm new to KQL and Sentinel. I have a question by the way. I work in a small company any we have M365 and Azure Identity Protection. I noticed that Sentinel will display duplicate alerts coming from Azure Identity Protection and M365. What could be the reason for that?
Check if your actually receiving two events. If you are generating two alerts then you need to check if you are grouping similar alerts together so only one incident is generated. I’ll make a video they explains it next week that explains it in detail along with other things things
@@TeachJing Thank for the reply. If I may, can you also create a video about threat hunting in sentinel? Like hunting IOCs such as (Hash values, services running, backdoors, etc.). Thank you.
My home lab behind me most of the time, but sometimes it’s azure in some demos. I don’t use VMware too often, but do have a cluster I use sometimes. Any reason for that question I could help with?
@@TeachJing Yeah, I was just curious about sending logs of guest machines to azure. If you know then please make a video on that also because most of my attack defend setup is on VMWare.
hello, I have followed the steps mentioned in this webinar. But unfortunately all the PA FW and Cisco Meraki events are getting forwarded as normal syslog messages to Azure sentinel via the local oms agent running on syslog collector/RHEL server. Any idea what should be changes in the 95-omsagent.conf or rsyslog.conf file?
