Splunk 7 "What's New"
Splunk Enterprise 7 new release marks “the end of meh-trics,” and delivers improvements in machine learning, as well as speed and scale improvements for analytics, monitoring and alerting.
Covered will be:
- Metrics - which are sets of numerical, time series data are now treated as a first class data type bringing massive performance improvements such as up to 200x faster queries
- Event Annotation seamlessly unifies logs and metrics by overlaying multiple searches in a single time chart or graph
- Chart Enhancements expand the selection of visual styles and chart options geared toward improving the visualization of metrics and multi-series monitoring use cases
- Faster Data Model Acceleration through core search technology tweaks
- Self-Service App Management in the cloud has been updated to allow the installation of your organization’s own internal apps
- The latest Machine Learning Toolkit improves extensibility, scalability and ease of use through several new enhancements
Metrics are sets of numbers describing a particular process or activity, measured over time. Some common examples of metrics that you may be familiar with are: time series data; system metrics such as CPU, memory or disk; infrastructure metrics such as AWS CloudWatch; and IoT devices (temperature readings).
A metric consists of the following:
In Splunk Enterprise 7, there is a 20X speed improvement against accelerated log data (tstats), and up to 200X speed improvement against non-accelerated log or event data when querying metrics. Additionally, real-time metrics queries will use substantially fewer resources.
All of the Splunk platform benefits apply to metrics–visualizations and alerting, role-based access controls, data on boarding, clustering, scaling and alerting; and, importantly for new use cases, the ability to leverage open source data collection daemons such as statsD and collectD.
How Metrics are configured
First you will need to create a new index that is specifically tuned for metrics data. This index will use our Metrics Store which provides the ability to ingest and store metric measurements at scale.
Next, you’ll need to configure a data input. There are out-of-the-box sourcetypes and native support for both statsD and collectD. Alternatively, you can configure any other data source with props.conf and transforms.conf to fit the metrics structure.
To query and retrieve your metrics data, you will use a new Splunk Search Processing Language (SPL) command called “mstats.”
Mstats is the tstats equivalent to query time series from metrics indexes and can be used for both historical and real-time searches. Below is an example:
Can help decipher what is and is not actionable from disparate data sources.
Event Annotation allows you to see your metrics results and your other searches in one dashboard.
You can get event context for any time chart (line, column, area), and event annotation markers and labels can be pulled from sources such as log data, lookup files, or external sources. All together. In one view.
How to use and configure Event Annotations? It’s a few simple lines in the dashboards XML.
The “annotation_label” can be a Splunk field or any text you wish. The same goes for the “annotation_category” which helps define the annotation colour. Now the dashboards are getting even more context.
The lines look differently in this last screenshot. That’s because Chart Enhancements in Splunk Enterprise 7.0 have been added, that expand the selection of visual styles and chart options geared toward improving the visualization of metrics and multi-series monitoring use cases. Line width, line style, and a new series comparison option in the legend are included in these enhancements and editable by SimpleXML.
Faster Data Model Searching
Optimisations to the core search technology decrease the time and resources required to run Data Model Accelerations (DMA) and accelerated searches giving us Faster Search & DMA Performance.
Self-service App Management
Splunk Enterprise Security users will immediately see up to a 3X speed improvement on the Data Model Acceleration time and a reduction in summarization lag. While this feature should help Splunk Enterprise Security users right away, it affects all DMA.
In 6.6 the Self-service App Management interface for Splunk Cloud customers was introduced, that allowed the installation and management of Splunk Certified Apps. Now in 7.0 you will have the ability to install your own private or internally built Apps using our new auto-vetting process. This will reduce the time it takes to get an App vetted and installed from weeks to minutes.
Machine Learning Toolkit
Lastly, we are excited to announce several key updates to the Splunk Machine Learning Toolkit over the past year. These enhancements to the toolkit include an improved API, new data prep algorithms, role-based access controls for machine learning models and new out-of-the-box algorithms to make it even easier for you to predict future IT, security, and business outcomes.
The advancements toolkit includes:
- Machine Learning Model Access Controls: machine learning models are now fully integrated with Splunk's role-based access controls (RBAC)
- New Data Prep options for using pre-processing algorithms save you time preparing datasets for machine learning models
- ARIMA Forecasting: the Autoregressive Integrated Moving Average (ARIMA) algorithm has been added to the available options for forecasting time series data. This includes new visualizations for inspecting properties unique to the ARIMA algorithm.
- Extensible Machine Learning: We’ve opened up the API to allow partner and customer app developers to import custom algorithms and build their own, and then share with the community via Splunkbase.
Want to take a further look at Splunk? Then contact us today for a demo!