Security analytics and insight: mapping the road to success
When something new is invented, it is often hailed as the next greatest thing. SIEM (security information and event management) systems were introduced more than a decade ago, touted as systems that would take much of the manual labour and time-intensive processes of sifting through information related to security events and incidents to provide actionable insight for organisations on which to better base decisions. But they were not all that they promised to be. They were expensive, required vast amounts of customisation and left organisations drowning in an ocean of alerts, with no means of knowing where to look first.
As organisations look to transform their models and infrastructure to take advantage of the possibilities opened up by the increasing digitisation of apparently just about everything—something that 100% of respondents to a recent poll conducted by CIO Watercooler said was of interest to them, with 72% saying they were very interested in digital transformation—technology vendors in the SIEM and related spaces have been stepping up their innovations in this space to cater to the reality that businesses are facing today.
The world of digital transformation
As an analogy as to how digital transformation is changing things, albeit on a much longer timescale, I offer up maps. Maps have long been an essential guide to the lay of the land, showing the interspatial relationships between objects. They help to guide us through areas with which we are not familiar so that we can prioritise our routes based on particular circumstances, such as taking a highway when pressed for time or a byroad when time is less of an object. Whilst basic at first, technology has added to their usefulness. Once just a snapshot in time, they now are aids in effective decision making.
The oldest maps, with some dating back to Babylonian clay tablets from 2300 BC, were really just artistic impressions of the world as it was known at the time.
In around 1500 AD, Ptolemy is credited as the first person to apply mathematical rules to the composition of maps in terms of longitude and latitude. This created a grid that spanned the globe, based on a common set of coordinates. Although later scholars found his measurements to not be entirely accurate, the principle was set.
With the invention of the printing press in 1440 and the subsequent growth of major publishing houses, maps and other documents could be more widely disseminated to a broader audience. Over the years, road networks came into being that were traversable by all, with highways being built from the early 20th Century and a rapid rise in car ownership occurring from the 1950s onwards. Whilst maps from this era had enhanced functionality, adding places of interest along routes and showing locations of transport links such as airports, they were still static, fixed to paper. But, the addition of such contextual information made for more efficient decision making regarding how routes were planned.
This has mirrors in the way that computer networks continue to expand to incorporate an ever greater range of information sources, increasing their usefulness. With the right technologies, context can be derived from events recorded from these information sources, such as showing relationships among various events that could trigger a security incident. However, traditional SIEM systems fell down in this aspect as they lacked sufficient reasoning power.
Mathematical reasoning and contextual decisions
From the late 19th Century onwards, there was a surge in mathematical reasoning and measurement technology, vastly increasing the accuracy of maps. This expanded their usefulness, but maps still remained static representations until late in the 20th Century, when the use of modern satellite systems and surveying techniques became commonplace. The GPS system was opened up for commercial use in the 1980s, enabling enhanced data gathering and analysis to vastly improve the information presented to users. Today, connected systems have spawned manifold advanced capabilities, enabling route planning and advice services through telematics. Today, dynamic GPS-based navigational systems are commonplace, especially through apps on mobile devices that are available for all and that include advanced capability. Static, paper-based maps have become all but obsolete for those planning journeys.
In the world of network computing, the world has also become an ever more connected place. Applications are no longer solely deployed within an organisation’s network, but have been pushed out for use by a wider population, available via mobile devices and cloud-based services that put information at everyone’s fingertips. Unable to adequately cater for these new advances and ingest the rich sources of information that they contain, traditional SIEM systems could only provide a limited picture of the workings of the extended network.
The analogy of maps shows how technological advances have enabled static informational sources to be transformed into dynamic aids that allow for much more effective decision making. Instead of just showing how a road traverses the land from one point to another, the integration of contextual information such as traffic jams and accidents can guide users around roadblocks to create more efficient journeys.
From reactive to proactive security
The same evolution is occurring in the delivery of network security and operations. Complementary technologies that integrate directly into foundational SIEM systems are expanding their usefulness considerably. Once seen primarily as a tool for providing evidence for audit and compliance purposes, next-generation SIEM systems are now capable of providing organisations with a proactive security stance, able to ingest a much wider range of information from a vast array of source to make sense of what is occurring on the network.
This has been made possible by a range of advances, not least of which is the ability to ingest data from a wider range of sources. Advanced statistical analysis capabilities that can handle extremely large data sets enable information sources related to events to be more quickly and effectively analysed, identifying patterns of activity so that informed response decisions can be made. Through use of these analytic capabilities, patterns of related activity emerge that guide practitioners in their quest to make sense of what is happening. Those patterns can be tracked over time, recording events as they unfurl and even identifying patterns of behaviour exhibited by exploits that have not previously been seen, enabling detection of even advanced threats that look to defeat existing security defences.
Advanced analytics capabilities can work with both unstructured as well as structured info—extending their use beyond databases to investigation of user-generated information from a variety of devices or sources.
The addition of contextual information means that risk scores can be defined and applied to events to enable prioritisation, ending the alert fatigue that has been a bugbear with traditional SIEMs. Such context can include information such as what hosts have been impacted, where data comes from and how it has been used, including by whom, enabling the criticality of an event to be gauged. SIEMs fell down in this regard as security practitioners needed to know what they were looking for upfront, rather than being guided by evidence from events as they unfold.
Increasingly, machine learning features are being added, making systems able to learn from patterns of behaviour to make them capable of discerning even the latest, most advanced threats as well as information related to events generated by insiders to the organisation. This is another area where traditional SIEMs performed poorly. They do this by comparing patterns of behaviour against behavioural baselines that have been set.
Another advance is the inclusion of threat intelligence feeds, whereby data regarding the latest threats and vulnerabilities seen can be fed automatically into security controls to increase their effectiveness against emerging threats and to increase detection capabilities, providing early warning signs of an impending attack and thus helping to speed up incident response by providing context regarding the tools and techniques being used by attackers. The use of threat intelligence enables threat hunting, whereby security practitioners can actively search to look for evidence of threats that can evade existing security controls.
As endpoints proliferate and become more powerful, they are increasingly being used as threat vectors by adversaries. By incorporating data from endpoints, a wealth of information is provided that was not available to traditional SIEMs that ingested data primarily from sources internal to the network. Without the ability to ingest information from endpoints only half the picture can be seen.
Automation is the holy grail of technology systems, reducing the required manual labour, especially where events seen are fairly commonplace and not critical to the running of systems. New technologies being added to security intelligence platforms aim to increase the use of automation so that operations run more efficiently. For more complex, and especially for critical events, the application of human knowledge is still be essential to ensure that actions taken are suited to the criticality of the event observed. But, with the current worldwide shortage of skilled security professionals, any task that can be efficiently automated is a bonus.
With these technologies, SIEM systems have taken another large step in terms of their evolution to security analytics and insight platforms.
Among the complementary technologies that are improving the capabilities of SIEMs, those showing the most promise are:
- Endpoint detection and response–Endpoint detection and response (EDR) provides visibility into activity occurring on the network and endpoints by continuously monitoring activity for behavioural patterns that appear to be suspicious or anomalous. Data captured provides rich contextual information related to a threat to enable more efficient, prioritised remediation. EDR technologies not only collect data, but use preconfigured detection models to enable teams to proactively hunt for threats before they can impact the organisation.
- User and entity behavioural analytics—User and entity behaviour analytics (UEBA) technologies focus on identifying patterns of user and device activity that are outside of the normal patterns of expected behaviour in order to identify activity that could be malicious.
- Security response orchestration—Security response orchestration helps organisations to respond effectively to data related to security events that has been gathered, correlated and analysed from multiple sources throughout the network. Building on automation capabilities, the emphasis of orchestration is on utilising the knowledge of humans where an outcome is unsure to make the entire response process more efficient and effective.
When these tools are added to a core SIEM system—built into it, integrated through APIs, purchased as a subscription service via a cloud provider or MSP—the SIEM system evolves into one integrated platform, combining multiple technologies that vastly increase its usefulness. It enables security to be turned on its head, from playing a game of catch up by trying to protect against the inevitable by preventing threats from entering the network, to a more resilient stance of proactive detection of threats that have already gained a foothold and using the tools available for effecting a more efficient response.
Going back to the example of maps, static, limited information sources based on immutable paper records have morphed into demand-driven rich information sources, enabled by taking advantage of new technology-driven possibilities.
With the evolution in security analytics and insight capabilities, the lay of a network can be more clearly seen, including interdependencies and patterns of related activity. Many vendors even provide visualisation of the results in the form of topology maps. With these capabilities, organisations are better able to understand what events are occurring on the network in order to more efficiently remediate incidents, focusing on the most critical first, without the blindspots and hindrances that plagued early systems.
