Welcome to the Splunk Tutorials. The objective of these tutorials is to gain understanding of machine data /logsIt’s a powerful software/Engine which can be used to search,investigate, troubleshoot, monitor, visualize,alert, and report on everything that’s happening in your entire IT infrastructure from one location in real time.
In addition to these free Splunk Tutorials, we will also cover common Interview Questions, Issues of Splunk.
Splunk is an American company based in San Francisco, California. The company was founded in 2003 by Michael Baum, Rob Das, and Erik Swan with a mission to make it much easier to assemble and analyze the data needed to run and troubleshoot a datacenter. Splunk’s Basic Operation, in which we introduce basic data collection, processing, analysis, and visualization of results.
-You don’t need to login to multiple servers and dig for all logs for particular event. Splunk will do it for you in smarter way.
– You can even monitor your twitter feeds, gmail, mailbox etc using splunk.
-Its a data mining tool for Big Data. Built in to handle Big/large data without affecting performance
– Splunk do not require any database like Oracle or MS SQL to store its data.It stores it’s data in indexes.so no additional cost for DB
– It effectively reduces troubleshooting and resolving time by providing instant results.Splunk is your best friend for root cause analysis
– It can work as monitoring tool, SIEM, reporting tool,analysys tool….and much more…..
– Its very easy to setup and expand.
Features of Splunk
-it can index any type of data; however, it works best with data that contain timestamps.
-it provides powerful search, analysis and visualization capabilities to empower users of all types.
-it creates a central repository for searching data from many different sources.
-it offer hundreds of apps and add-ons that can enhance and extend the Splunk platform.
-it helps you gain valuable Operational Intelligence from your machine-generated data.
-Splunk is a high performance, scalable software server written in C/C++ and Python. It indexes and searches logs and other IT data in real time. Splunk works with data generated by any application, server or device.
-The Splunk Developer API is accessible via REST, SOAP or the command line.
-After downloading, installing and starting Splunk, you’ll find two Splunk Server processes running on your host, splunkd and splunkweb.
-Splunkd is a distributed C/C++ server that accesses, processes and indexes streaming IT data and also handles search requests. splunkd processes and indexes your data by streaming it through a series of pipelines, each made up of a series of processors.
-Pipelines are single threads inside the splunkd process, each configured with a single snippet of XML.
-Processors are individual, reusable C/C++ or Python functions that act on the stream of IT data passing through a pipeline. Pipelines can pass data to one another via queues. splunkd supports a command line interface for searching and viewing results.
-Splunkweb is a Python-based application server providing the Splunk Web user interface. It allows users to search and navigate IT data stored by Splunk servers and to manage your Splunk deployment through the browser interface. splunkweb communicates with your web browser via REST and communicates with splunkd via SOAP.
-Splunk’s Data Store manages the original raw data in compressed format as well as the indexes into the data. Data can be deleted or archived based on retention period or maximum data store size.
-Splunk Servers can communicate with one another via Splunk-2-Splunk, a TCP-based protocol, to forward data from one server to another and to distribute searches across multiple servers.
-Bundles are files that contain configuration settings including, user accounts, Splunks, Live Splunks, Data Inputs and Processing Properties to easily create specific Splunk environments.
-Modules are files that add new functionality to Splunk by adding to or modifying existing processors and pipelines.
IT operations with Splunk
Proxy logs = these logs are good for C2 analysis of files, domains, downloads of DLL/EXE files
Anti‐virus logs = these logs are good for analysis of malware, vulnerabilities of hosts, laptops, servers, monitor for suspicious file paths
Server Operating System logs = these logs are good for analysis of server activities such as users, runaway services, security logs,
Firewall logs = logs for network traffic of source/destination ip addresses, ports, protocols
Mail logs = logs for inbound/outbound mail for malicious links, targeted recipients, unauthorized file out bound, data loss, bad attachments
Custom apps logs = logs could be analyze for possible buffer overflow, code injection, SQL injection analysis
Intrusion Prevention System logs = capture these logs to alert on signatures firing off, COTS signatures, threat analysis of bad network packets
Intrusion Detection System logs = capture logs to alert on signatures firing off, custom signatures, bad network packets,
Database logs = capture these logs for authorized access to critical data tables, authorized logons, op ports, admin accounts
Virtual Private Network(VPN) logs = capture logs to analyze users coming into network for situational awareness, monitored foreign ip subnets, compliance monitoring of browsers/apps of connected hosts
Authentication logs = authentication logs to monitor authorized/unauthorized users, times of day of connection, how often, logons/logoffs, BIOS analysis,
Vulnerability Scan Data = import data about assets, vulnerabilities, patch data, etc
Web Application logs = external facing logs to monitor suspicious SQL keywords, text patterns, REGEX for threats coming in through browser
DNS logs = to correlate ip’s going to what domain at a client level
DHCP logs = monitor what systems are being assign what ip address and how long, how often
Active Directory/Domain Controller logs = monitor user accounts for AD admins, privilege accounts, remote access, multiple admins across the domain, new account creation, event ID’s
Badge Access logs = logs to capture to correlate insider threat, situational awareness, correlate data with authentication logs
Router/Switch data (net-‐flow) = capture this critical data source for APT monitoring, network monitoring, data exfiltration, flow analysis, this is a very important data source
Packet Capture logs(PCAP) = capture this very critical data source for APT, data exfiltration awareness, packet analysis, deep packet inspection, malware analysis, etc
FW + AV = will help detect and respond to viruses, worm propagation
IPS + AV + FW = detect/alert on network based attacks such as buffer overflow, reconnaissance scans, code injection
PROXY = monitor majority of web based/application layer attacks such as: cross-site scripting, session hacking, browse redirects
AV + PROXY = monitor/detect/respond to download of bad files, remote code execution…web-based attacks
FW + PROXY = detect outbound data exfiltration, detect potentially misconfig fw rules,
IPS + FW = monitor all network packet signature threats
AD Server = monitor all user/group modifications, deletes, updates for administrators
AD + PROXY = monitor/detect/alert on post compromise analysis, lateral movement
Splunk configuration files
-Identify and resolve issues upto 70% faster.Reduces costly escalations by upto 90%
-SPlunk converts complex logs to visual graphs and reports resulting simplified analysis,reporting and troubleshooting, No separate database requirements like oracle or SQL as splunk stores all data in its index supports any format and any amount of data -enables centralized log management.
-Simple to implement and scale. Continually index all of your IT data in real time.
-Automatically discover useful information embedded in your data, so you don’t have to identify it yourself.
-Search your physical and virtual IT infrastructure for literally anything of interest and get results in seconds, Save searches and tag useful information, to make your system smarter.
-Set up alerts to automate the monitoring of your system for specific recurring events.
-Generate analytical reports with interactive charts, graphs, and tables and share them with others, Share saved searches and reports with fellow Splunk users, and distribute their results to team members and project stakeholders via email.
-Proactively review your IT systems to head off server downtimes and security incidents before they arise.
-Design specialized, information-rich views and dashboards that fit the wide-ranging needs of your enterprise.
-Trusted by wide customers over the globe Not limited to IT-can be used where big data is involved -only limitation is your mind.
For indepth understading on Splunk Tutorials click on
These core tutorials that helps you to learn the fundamentals of the Splunk platform. For in-depth knowledge and practical experience explore Online Splunk Training.