Reassessment Of SIEM Solutions On The Market

It has been so long since the last time I have written about SIEM solutions in this blog! How long exactly? Well, since the last blog, I have changed 2 companies, 4 functions and relocated in another country type of long.

SIEM and Log Management topics are always very dear to me, although my focus on IT and Information Security has widened, to the point to include Information Risk Management as well.

I am going to resume the landscape in SIEM area in the last 3 years with the main actors involved.

HP ArcSight became Microfocus ArcSight with HP’s spin-off from the software business. The decision seems to be taken a long time ago already. The ambiguity in the way merger/selling operations were dealt between HP and Microfocus pushed the customers to look for alternatives. The acquisition of HP Software division including ArcSight product family by MicroFocus was announced on the 7^th of September 2016 and was completed officially on the 1^st of September 2017. But people close to the subject know that long before the acquisition announcement, product development activities were severely slowed down if not completely stopped.

HP ArcSight was already having some difficulties addressing long term storage of data on the platform itself other than lacking advanced features proposed by competitors. Running even simple queries on the old-fashioned Logger was taking ages, even when tried on the command line with scripts.

The problems were since addressed ArcSight Data Platform (I will later provide a dedicated post on that) and some advanced features are presented to the customers such as User Entity and Behavior Analysis (UEBA) but the damage I think is done. ArcSight has lost an important part of its customers other than the big accounts which really invested too much on the solution to leave it.

The arch-rival of ArcSight was IBM QRadar at the time we left. QRadar was somehow less customizable comparing to ArcSight but was a strong competitor in regards to the integrations it had such as Network Packet Flow Analysis (QFlow) being the most important. Other than this, platform was and still is, capable of indexing all log fields comparing to limited indexing capability of ArcSight, which can be considered as a huge advantage.

Moreover, architecture-wise, QRadar supported scaling out (increasing the performance/capacity by adding new devices), therefore allowing a much better retention of logs online, without sending logs to external storage while since recently ArcSight’s Logger only supported scaling up (increasing the performance/capacity by increasing system resources).

Relative simplicity of IBM QRadar also helped the solution’s overall stability, which on ArcSight’s side required a separate management appliance (ArcSight Management Center) and sometimes some 3^rd party appliances for connector health management.

Another advantage of QRadar is its integrated additional capabilities, such as the User Behavior Analysis module which is not as efficient as a full blown UEBA solution from Exabeam for example to compare with but still does the essentials for enriching the bare log data. When talking about data enrichment ability to consume Vulnerability Management data also should not be left behind.

IBM QRadar seems, in my humble opinion, the best option for large environments and for on premise use.

McAfee ESM used to be the 3^rd major actor behind ArcSight and Q-Radar. McAfee had the advantage of being simpler to configure and to license solution within a vendor-controlled package. Not much has changed since other than providing a big-data approach to McAfee ESM’s architecture, transforming the front-end to HTML5. McAfee’s SIEM solution was and is one of the least appealing SIEM solutions for me as it never got the attention it deserved from the organization, always lagging behind McAfee’s flagship products.

It can be advised to small-to-medium size organizations having with strong relations to McAfee.

SIEM Planning - Capacity Planning and Sizing

SIEM projects are well-known to be demanding and greedy when it comes to the resources and your CIOs/CISOs would like to hear about your direct (software licensing, server investment, etc.) and indirect ( archiving storage, support) costs for at least following 3 years other than the benefits the Project will provide.

In this article, I will of course give the basic formulas about sizing. However the most important information to be provided is the real life experience of a live operation concerning the values and comparison to those values to the benchmarks.

First unit, which constitutes the base for all our calculations, is the Event per Second (EPS) value that each source system generates. EPS value greatly depends on 2 factors, audit policy or rules applied on the source system and business of the system. A server with “Object Access” audit rules enabled and Web Server functionality configured of course would not generate the same number of logs with a standard server. Windows family of servers also tends to generate much greater number of logs than Linux and UNIX servers, all with standard default configurations.

EPS estimations can be reached using links 1, 2 and 3.

Having calculated the number of EPS for each source asset group the next step to do is calculating the Event per Day (EPD) value.

EPD = ∑ EPS X 86400

Once EPD value is calculated, we have to decide an average log message size to know how much storage we will need each day. Log sources generate logs starting from 200 bytes range on network and infrastructure devices to 10 kilobytes or more on application and database side.  Syslog standard (RFC 5424) sets the maximum size of the content field of a log message to 2 kilobytes. In light of this information, it is wise and advisable to assume a raw log message size as 500 bytes.

Average raw log message size being set to 500 bytes, the amount of Daily log messages in GB is calculated as follows:

Daily Raw Log Size = EPD * 500 / (1024)³

Log management appliances do some changes on the log messages to make them understandable and meaningful. This operation is called “Normalization” and it increases the log size depending on the solution you use. In my personal experience with HP ArcSight, normalization increased the log size about 90% to 100%. Some other people have seen up to 200% of increase in their experiences.  As a result we obtain the below given formula for daily normalized log size:

Daily Normalized Log Size = Daily Raw Log Size * 2

The calculated value does not really represent the daily storage value for log management systems. Many vendors came up with proprietary compression solutions and claim they compress logs 10 times (10:1) which is quite idealistic. It is however, safe to consider a ratio of 8:1 for calculations. So the formula becomes:

Daily Storage Requirement = Daily Normalized Log Size / 8

The annual storage need would basically be 365 times the Daily Storage Requirement, if you want your calculations to be on the safe side. Nevertheless, EPS numbers seriously fall during weekends and vacations. Watch how much your average EPS numbers decrease in such periods and do your own calculations for your annual needs.

Annual Storage Requirement = Daily Storage Requirement * 365

The last important point is the retention period when you plan your storage investments for future. 2 factors are decisive in the definition of retention period, Compliance Requirements and Security Requirements.

Compliance Requirements only concern Log Management systems, in HP’s case it is Logger and there is not much to decide really, whatever the legislation obliges, you have to configure.

For security needs which are addressed by HP’s ESM system, the decision is yours. I have seen many decision makers trying to keep themselves on the very safe side and take retention periods unnecessarily long.  According to Mandiant, the median number of days attackers were present on a victim network before they were discovered was 205 days in 2014, down from 229 days in 2013 and 243 days in 2012. This brings me to the conclusion that retention period for security alert creation, monitoring, trending and forensics should be at least 1 year and not longer than 3 years. According to the same study of Mandiant, “The longest time an attacker was present before being detected in 2013 was six years and three months.”. Last but not the least, the retention period of course depends on the sector of activity, defense being the longest and the strictest followed by financial institutions.

A rough estimation about Storage IOPS values can be calculated with the following formulas:

Storage IOPS Needed (Direct Attached Storage) = EPS * 1.2

Storage IOPS Needed (SAN) = EPS * 2.5

Considerations About a Successful SIEM and Log Management Project

You may spend a lot of effort to build up an infrastructure of SIEM and Log Management composed of source systems, connectors, loggers, rule creation and correlation engines and management systems and then see very little of valuable output….if you do not pay attention to the first thing you actually do in the first place: Define a proper auditing policy.

A recent and very successful approach for building Log Management and SIEM capabilities actually consists of an inverse installation process. Some Security Service Providers come and collaborate with your teams to apply defined risk scenarii on your infrastructure to see if your infrastructure components generate the log messages which should alert you that something unusual and odd is going on. It is at this very step that your SIEM team learns which type of events they should be collecting among a big pile of others, which in many cases constitute most of your log storage without actually providing a value.

Such an approach may create huge differences in outputs and may trigger changes in your infrastructure. I know companies which changed some of their components just because they do not provide essential log information which would allow security alert generation.

A very important thing to keep in mind when deploying SIEM solutions is the involvement of all infrastructure and application teams. No matter how qualified IT Security guys responsible for SIEM deployment, they do not master Operating Systems as much as Windows, Linux and UNIX administrators do, also considering different versions of OSes that can be in place. To my experience, I know that 3 generations of Windows Servers coexist in majority of companies without counting R2 versions. It goes the same way for databases and applications. A SIEM project would probably fail or underperform if all IT teams do not collaborate with SIEM project team and stay isolated in their silos.

Another way of dealing with this issue would be to create a security team composed of security masters in each domain. First difficulty in that approach is to bring together such talents which is very costly and the second challenge is to keep them in the company and provide consistency because such people are highly in demand. This option seems applicable only in very large structures such as multinationals, especially in finance sector where there really are some things to be at stake, money and also reputation.

There are of course lots and lots to say about other aspects of SIEM and Log Management projects. But maybe the most important things to know about them are to set the expectations correctly (Benefits, Aim: “Security?, Compliance?, Both?”, Scope, Schedule and Budget), be patient, provide continuous support and monitor the output closely. The technology in this market is rapidly evolving and it still has much more to offer in years to come.

Introduction to Security Event and Log Management

Starting from this article, I will share my experiences and insight on Security Event Management and Log Monitoring. I hope you will find some useful information for yourself in them.

To start with: maybe the most important point to know at the start of a SIEM project is to correctly set the expectations from such project. As very important sums of money and effort are put to these projects, the expectations of several units of your company (not only the IT by the way) may be very high.

So, at the kick-off it is important that you know and make sure that others understand:

• that a SIEM solution is not a big box where different parts of IT throw all their logs until the day they will need it. The importance of “Security” word in SIEM must be underlined. Even though by the very nature of SIEM systems, they are considered as Big Data systems but so far they are only “Big Data like” and maybe they should stay so. Peer Teams in IT should take their time to think which category or type of alerts they have to keep, to find out an anomaly in their own systems or which can be meaningful when it comes together with other piece of information.
•  that sizing is one of the critical steps of SIEM project’s success. A strict sizing in such projects is of course impossible but EPS numbers should be guessed with up to 20% accuracy. After that comes the decision of retention period. Most admins would enjoy seeing what website your colleague in the next cubicle visited a year ago but for an effective forensic analysis, a period from 3 to 6 months is advised to be aimed. A failure in not correctly predicting the storage size may cost you having your critical log data overwritten when you most need it. Not to forget, these numbers are also important for the financials, software licenses and appliances sizes and sometimes even the solution architectures change according to the number of logs to be processed.
• that SIEM projects may be huge and should not be tried to be swallowed at once, your company choke while doing so. A wise approach would be to start with Security and  Network Security systems including but not limited to Firewalls, UTMs, IPSs, VPN boxes, Access Control Systems (Radius, Tacacs+), Web Application Firewalls, DOS protection devices, NACs, Load Balancers, Wireless Controllers and Endpoint Security Systems. After these should follow the Infrastructure components, namely OSs and Middlewares of your critical systems. At the last step, you will finally be ready for integrating your applications, which obviously is the most laborious and time-consuming part as several teams will have to come together and discuss.
• that you should document your project very well. SIEM systems are like those puzzles with thousands of pieces and almost every company cooks that soup to its own taste. The way you collect the logs, reduce them and correlate them largely differs. Without proper documentation, you may have serious problems in roll-outs, troubleshooting and handing it over to somebody else or even outsourcing.
• that even to everybody is into it and the technology is very promising, it is still a reactive solution. It will help your company to discover important breaches, learn from important misconfigurations and mistakes and improve your incident response skills and practices but that is all. Everything still depends on your agile, well-documented and respected incident response procedures… Unless you deploy SIEM as a compliance box or to show off your C-level executives.

      After taking into consideration these important facts, you are ready to start choosing your SIEM product. I will mention about SIEM product selection criteria in my next article, before diving into more detail on implementation.