In this blog post, we will offer an overview on how to deal with Security information and event management/log management (SIEM/LM) data overflow. Let’s see how Linkurious’ advanced graph visualisation solution helps easily identify and investigate cyber-security threats.
Switching to a data lake architecture is often a required first step for analysts who wish to use graph data visualisation solutions such as Linkurious to start visualising their SIEM/LM data. Linkurious enables analysts to deal with SIEM/LM data overflow and perform precise real-time and/or post-attack forensics analysis. In the second part, we will demonstrate the extent of Linkurious’ possibilities using a real life SIEM/LM data-set use case and perform a forensics analysis example.
Dealing with SIEM/LM data overflow: putting security analysts back in control
SIEM/LM solutions have evolved continuously over the last 15 years to match the ever changing landscape of cyber-security threats. SIEM/LM solutions aim to provide analysts all the necessary information and context they need to determine the nature of an attack, its degree of sophistication and of proliferation inside the network. To efficiently contain security breach damages and react efficiently, analysts need the right information at the right time.
Today, it still remains a considerable challenge for organisations of all sizes to meet their necessary operational, audit and security needs. As networks become more and more complex, the number of devices to monitor has significantly increased. Analysts are literally overflowed with data. Because of that, aggregating these different SIEM/LM data sources together has become a challenge in itself. These significant framework limitations disable analysts. They have too much data, but not enough information. There is a real need to reduce the scale and complexity of the analysis to a more intelligible level in order for analysts to come up with appropriate solutions to improve overall security. Advanced data visualisation solutions enable just that.
But for the moment, SIEM/LM solutions still rarely include data visualisation tools. Even if they do, they are not efficient at treating such big amounts of data and do not offer real-time pattern detection and exploration possibilities. Right now most companies relying on SIEM/LM data visualisation solutions only use them for illustrations purposes rather than for analysis. They often have to rely on external services to carry out post-attack forensics as these operations require a lot of skill and time.
Using graph data visualisation tackles this problem and makes SIEM/LM data operational again
Today, the trend in the cyber-security world to resolve these issues is to switch from the traditional data warehouse framework to more flexible and scalable data backends. This enables the use of new tools such as graph data visualisation analytics solutions. Typically these new backends take the form of data lake frameworks: often Hadoop combined with other services such as graph databases and other analytics tools. Data Lakes have many advantages compared to data warehouses when it comes to managing terabytes of security logs: centralisation, flexibility, operationality, and high scalability. Companies who are serious about using new analytics applications such as Linkurious for their SIEM/LM data will have to make the switch sooner or later. One might also add that depending on the company’s needs, the switch can be fairly non-intrusive for the overall existing system architecture.
How Linkurious empowers security analysts
Once the SIEM/LM data is centralised into the data lake, using a graph data visualisation solution like Linkurious to explore and investigate the data provides analysts with a real added value for their everyday operations. They are operational in real time, can visualise the data instantly and can carry out precise post-attack forensics analysis in much simpler ways than ever before. The detection of suspicious activity patterns can be largely automated using pattern recognition algorithms. That way, analysts can focus on investigating suspicious activity visually.
Visualisation is empowering for analysts as it resolves to a great extent the problem of having large amounts of data to interpret. Visualisation considerably reduces the scale and complexity of the analysis. It also allows companies to carry out most of their forensics analysis internally. With Linkurious’ advanced collaboration and security features, analysts are able to work together, share visualisations between them, and administer user access rights to the data. Finally, the advanced customisation possibilities that Linkurious offers allows its integration into internal security systems.
Next, we will demonstrate Linkurious’ possibilities using a real-life SIEM/LM dataset to see the advantages of graph visualisation technology to monitor networks in real-time and perform advanced forensics analysis.
Using Linkurious for cyber-security: a real-life use case
This dataset was created using a real life log archive of an enterprise network. Courtesy of the University of Victoria who created and made public the dataset for general research purposes. The dataset is the combination of several existing publicly available malicious and non-malicious SIEM/LM log datasets. The dataset reproduces the day to day usage of an enterprise network. More information on the dataset here.
The PCAP files were generated with Wireshark and we converted it into a CSV file. We then generated several CSV files to model the dataset and import it into Neo4j.
We used the following model for the Neo4j database:
neo4j-import –USING PERIODIC COMMIT 1000 –skip-bad-relationships –C:\Users\linkurious\Downloads\neo4j-community-3.0.0-RC1-windows\neo4j-community-3.0.0-RC1\bin –nodes nodeip.src.csv –nodes nodeport.csv –relationships Relationshipdst.portip.dst.csv –relationships RelationshipIP.srcdst.port.csv –into C:\
The connections were aggregated together with the start date and end date to reduce the number of edges. Creating an edge for each transmitted packet would create super nodes and make the graph very difficult to read. The model we use is very simple, but the modeling can be made to fit very specific use cases depending on what the analyst is looking for.
Using Linkurious to identify a UMTP storm botnet
Linkurious enables analysts to visualise data that is otherwise seemingly difficult to conceptualise. Experienced analysts know what “normal” behaviours looks like on the network they manage. This makes it possible for them to set pattern detection algorithms that will pull up abnormal behaviours from the database. For example, the following visualisation shows a “normal” interaction in the network. IP’s interact with a wide variety of different service ports of 18.104.22.168.
On the other hand, here is an abnormal behaviour pattern. Most of the IPs that connect to “172.16.0.11”use port 25 (SMTP Port) and don’t generate any other traffic than that on any other services. This is suspicious in itself. But the large number of IP’s doing the same operation at the same time seem to indicate a botnet network carrying out a UDP storm attack. These attacks are basically a denial of service attack (DoS).
If a geolocation service fetches the GPS coordinates of the IP addresses, it is possible to visualise them directly on a map. In one click, using Linkurious geospatial visualisation feature, we can see that most of the IPs that are part of the botnet network are in the same region. Most of them come from around Odessa in Ukraine.
Most of the toxic traffic comes from Ukraine around Odessa
We can then explore the activity of specific IP addresses and see which services were affected by their activity. For example, the address “22.214.171.124” has other links that we haven’t examined yet. Let’s examine it separately and expand it to see all its connections. That way, we see it links to another IP on our network: “172.16.0.12”.
If we expand the IP address “172.16.0.12” to see its connections, we find it is connected to another attack. This means the two are probably linked together and that the network was maybe compromised several times. The attack follows the same pattern as the first SMTP storm attack we just saw.
Linkurious: graph data visualisation for cyber-security threats analysis
This simple use case shows the great potential graph visualisation technology has for cyber-security analysts. Analysts can now start to make sense of their connected data and investigate any suspicious behaviours on their network. Graph Visualisation offers a high level of precision for analysts to quickly understand any kind of security event. Assessing the degree of sophistication of an attack and reacting accordingly becomes easier than ever before.
Once the company’s data framework ready for graph data visualisation Linkurious will become a solid ally for all security analysts. The multiple possibilities that solutions like Linkurious offer enable analysts to overcome the overflow of SIEM/LM data and extract the information they need. Graph visualisation has the potential to reduce the complexity of their analysis, making SIEM/LM data operational. Forensic analysis also becomes less expensive as it is now possible to conduct it internally more often.
Graph technology enables the automation of a large part of the detection process. That way, analysts can focus on investigating the security alerts on the network. Linkurious’ collaboration features also enable them to work together more efficiently and rapidly. Linkurious meets all security standards for such sensible data and provides all the necessary tools to administrate user rights access. Using a graph-based approach also offers many advantages when working with non-technical users and other departments inside the company because of its inherent simplicity. Who doesn’t understand nodes and edges?