The Lacework agent: A technical deep dive

Security agents in the cloud have historically carried a bad reputation, often associated with crashed servers and operational disruptions. However, the perception of agents has changed because of well-designed solutions like the Lacework agent, built specifically to accurately and easily capture critical data in the cloud. 

Lacework derives actionable insights from a diverse collection of data. Using agents to observe runtime data across all workloads, agentless data scanning to provide additional coverage, and asynchronous analysis of cloud deployments, the Lacework Polygraph® Data Platform provides all-encompassing visibility. This deep dive explores the Lacework agent’s unique capabilities, how and why we created it, and why it’s so effective today.

 

 

The agent is efficient, flexible, and seamlessly adds runtime context to the platform. It improves visibility and insights from directly inside of the most critical cloud workloads. The agent supports: 

• Every major Linux distribution
• Windows servers
• Every major container runtime
• Every major container orchestrator
• Serverless runtimes like Fargate and Cloud Run

Building a reliable and lightweight agent that can analyze huge amounts of data efficiently requires careful consideration and expertise. The Lacework agent intelligently observes, filters, aggregates, de-duplicates, and compresses data that feeds into the unified knowledge graph, which in turn powers runtime alerting and enriches context across various findings and alerts, including vulnerability management. Achieving this level of performance at runtime, particularly in performance-sensitive workloads, requires substantial innovation, expertise, and hardening.

The Lacework agent was designed from the ground up with four goals in mind: 

1. Gain comprehensive signals on all aspects of cloud compute 
2. Respect performance-sensitive workloads
3. Capture crucial data accurately
4. Deploy easily across entire cloud environments 

Simple, flexible deployment

The Lacework agent has a simple and flexible deployment model and can be deployed on a host, as a container, or inside an application container. It can be delivered directly via cloud provider APIs, packaged within an Amazon Machine Image (AMI), deployed through helm (a Kubernetes package manager), or through custom means like your preferred package manager. This is possible because the Lacework agent is self-contained, meaning it requires very little from the environment around it and operates with minimal dependencies. 

 

 

To eliminate the need for customers to navigate complex package choices or configurations when deploying the solution, the intelligence required to operate efficiently in different distributions, versions, and environments is embedded directly into the Lacework agent. The agent automatically recognizes the environment it is deployed into and captures relevant information. 

For example, when the Lacework agent is installed on a host with running containers, it collects data about both the host environment and container-level activity. Whether it is Windows, Linux, Kubernetes, or virtual machines, the Lacework agent is equipped with the intelligence to perform accurately and optimally at runtime.

Massive data, tiny overhead

Lacework finds needles in haystacks: it efficiently highlights the runtime behaviors that need urgent attention and promptly alerts teams about malicious, suspicious, and other anomalous behavior. Finding the relevant pieces of information in massive amounts of data requires intelligence, sophistication, and scalability on the data processing side. Over the years, we have meticulously fine-tuned the Lacework agent to ensure it operates seamlessly alongside critical, high-performance cloud workloads while maintaining an impressively low overhead. With the Lacework agent, organizations have the required intelligence, efficiency, and scalability to confidently address security concerns without compromising operational efficiency.

The Lacework agent examines network packets sent and received from the host and attributes this network communication to the related artifacts like processes and containers. It knows how to de-duplicate redundant information and efficiently package that information for lightweight transmission to the Polygraph Data Platform. 

 

 

The Lacework agent automatically — without any additional configuration — determines the best way to collect data using a variety of hooks into the underlying operating system. For example, if it is running on newer Linux kernels that support eBPF, the agent utilizes that infrastructure for minimally invasive collection of process activity. If the Linux kernel is older, it uses other techniques to collect the data from user mode. These techniques are tuned to efficiently use as little CPU and memory as possible while keeping the information fidelity high.

Adds rich context and captures unknown threats

The Lacework agent examines software packages installed on the host and determines which packages are active and which are dormant. This information can then be leveraged to prioritize fixing active, vulnerable packages first and ensure that security teams address real issues rather than non-exploitable vulnerabilities.

The agent collects security signals necessary to detect breaches or compromises with runtime alerting. The platform then uses this data to understand your cloud, find unknown threats, and behavioral concerns using machine learning techniques.

Automatic upgrades, easy maintenance

Once the Lacework agent is installed, it collects internal operation metrics that continuously adjust the agent for improved efficacy and reduced footprint without customer configuration. Meanwhile, the Lacework backend detects any abnormal behaviors, even for the agent itself. The Lacework agent is automatically upgraded as needed so customers don’t need to maintain the software manually. This automatic upgrade is implemented so there is zero downtime monitoring the host during the upgrade — even the update process is fault-tolerant.

Richly customizable and highly efficient

We tailored the agent to automatically determine the optimal behaviors and configurations so the agent is easy to deploy and manage. But we also recognize that every workload and organization is different, and many have specific needs that can’t be automatically anticipated. For those reasons, we designed the agent to be extremely customizable. 

Customers can instruct the Lacework agent to collect specific data sources custom to specific workloads. Internally, each new data source is recognized as a new class of events to monitor. Customers can configure event types for their own needs. When a Lacework agent observes that there’s a new type of event it needs to monitor, it knows how to fetch the recipe from the Lacework library that helps it get the correct data. 

Further, the Lacework agent can filter these custom event types based on myriad properties. Customers can instruct agents on how to group and aggregate these events for efficient processing. 

For example, if someone needs to collect every process that runs in a host or a container, they only need to configure the Lacework agent to do so. Typically when security teams attempt to collect so much data on servers, the noise levels become very high and they start looking for ways to reduce the data set. In those cases, the security teams can configure the agent to see only processes that matched a filter on process name, executable path, parent chain information, or command line, etc. That is all configurable and the agent can be updated without even restarting.

 

 

Continuous file monitoring is an example of these techniques coming together. The Lacework agent can easily be configured to monitor file and directory changes in near real time, but to be valuable without drowning the team in noise, the agent can be directed to monitor only specific locations and specific actions. The power of the Lacework platform means that agent observations can inform runtime security monitoring, integrate with compliance, and more. These file change events include not only the file/directory that is changed, but who changed it, which process changed it, in addition to other attributes — a very rich data set for analysis. The agent can even do recursive directory monitoring. Through the techniques identified above, the Lacework agent automatically selects the most efficient techniques to get total coverage with the lightest CPU and memory overhead. 

Prevent incidents and reduce impacts

As businesses prioritize taking proactive measures to minimize risks and using deep insights into workloads to respond to incidents faster, a combination of agentless and agent-based approaches is required. To prevent security incidents, agents help identify and monitor which packages are loaded into memory, enabling more accurate vulnerability risk scoring. Agentless solutions also support incident prevention by more broadly scanning cloud environments, ensuring comprehensive coverage of all systems, applications, and data. By combining the two approaches, organizations can understand the origin of threats, track attacker’s paths, and prioritize risks based on that information, ultimately improving their ability to contain and react to security events faster. Read our blog to learn more about why true end-to-end security requires both an agent and agentless approach.