Malware analysis for beginners (step-by-step)

This post is based on the Hack The Box (HTB) Academy module: Introduction to Malware Analysis. This module offers an exploration into malware analysis with a particular focus on Windows-based threats. 

You can learn more by browsing the catalog of free or advanced cybersecurity courses on the HTB Academy! 

What is malware?

Malware, short for malicious software, is a term for various types of software designed to infiltrate, exploit, or damage computer systems, networks, and data.

Although all malware is used for malicious purposes, the specific objectives of malware can vary among different threat actors. These objectives commonly fall into several categories:

• Disrupting host system operations.

• Stealing critical information, including personal and financial data.

• Gaining unauthorized access to systems.

• Conducting espionage activities.

• Sending spam messages.

• Using the victim's system for Distributed Denial of Service (DDoS) attacks.

• Locking up the victim's files on their host and demanding ransom (also called Ransomware).

Intro To Malware Analysis

This HTB module guides you into the world of malware analysis with a particular focus on Windows-based threats. Leveraging industry-standard tools and methodologies, it provides hands-on experience in identifying, understanding, and detecting malware.

Take the malware analysis course

The different types of malware

In the fast-paced world of cyber threats, we find ourselves up against a broad spectrum of complex and varied malware forms. 

In recent years, governments, judicial systems, hospitals, schools, and companies have been attacked by malware and ransomware, such as Stuxnet and WannaCry. 

Let's peel back the layers of some commonly seen types of malware:

Viruses

These notorious forms of malware are designed to infiltrate and multiply within host files, transitioning from one system to another. They latch onto credible programs, springing into action when the infected files are triggered. 

Their destructive powers can range from corrupting or altering data to disrupting system functions, and even spreading through networks, inflicting widespread havoc.

Worms

Worms are autonomous malware capable of multiplying across networks without needing human intervention. They exploit network weaknesses to infiltrate other systems without permission. 

Once inside, they can either deliver damaging payloads or keep multiplying to other vulnerable devices. Worms can initiate swift and escalating infections, resulting in enormous disruption and even potential denial of service (DoS) attacks.

Trojans

Also known as Trojan Horses, these are disguised as genuine software to trick users into running them. After entering a system, these programs craft backdoors, allowing attackers to gain unauthorized control remotely. 

Trojans can be weaponized to pilfer sensitive data, such as passwords or financial information, and to orchestrate other harmful activities on the compromised system.

Ransomware

This type of malware encrypts files on the target's system, making them unreachable. Attackers then demand a ransom in return for the decryption key, effectively holding the victim's data to ransom. 

The impacts of ransomware attacks can debilitate organizations and individuals alike, leading to severe financial and reputational harm.

Spyware

This type of malware stealthily gathers sensitive data and user activities without the user’s consent. It can track online browsing habits, record keystrokes, and capture login credentials, posing a severe risk to privacy and security. The pilfered data is often sent to remote servers for further attacks.

Adware

Though not as destructive, adware can still be an annoyance and a security threat. It shows uninvited and invasive advertisements on infected systems, often resulting in a poor user experience. Adware can also be used to track user behavior and collect data for targeted advertising.

Botnets

These are networks of compromised devices, often referred to as bots or zombies, controlled by a central command-and-control (C2) server. 

Botnets can be exploited for a variety of harmful activities, including launching DDoS attacks, spreading spam, or disseminating other malware.

Rootkits

These are stealthy forms of malware designed to gain unauthorized access and control over the fundamental components (the "Root") of an operating system (OS). 

They alter system functions to conceal their presence, making them extremely challenging to spot and eliminate. Attackers can utilize rootkits to maintain prolonged access and dodge security protocols.

Backdoors and RATs (Remote Access Trojans)

Backdoors and RATs are crafted to offer unauthorized access and control over compromised systems from remote locations. Attackers can leverage them to retain prolonged control, extract data, or conduct additional attacks.

Droppers

These are a kind of malware used to transport and install extra malicious payloads onto infected systems. They serve as a conduit for other malware, ensuring the covert installation and execution of more sophisticated threats.

Information stealers

These are tailored to target and extract sensitive data, like login credentials, personal information, or intellectual property, for harmful purposes. This includes identity theft or selling the data on the dark web.

Types of malware analysis 

Note: This blog post will cover the basics of static and dynamic malware analysis, but you can learn more about the other types in the Introduction to Malware Analysis module.

Static malware analysis

Static malware analysis is an approach to scrutinizing malware code without executing it. 

An analyst will examine the file structure, identify strings, search for known signatures, and study metadata to gain preliminary insights into the malware's characteristics.

Dynamic malware analysis

Dynamic analysis entails executing the malware within a controlled environment, such as a sandbox or virtual machine, to observe its behavior and capture its runtime activities. 

This includes monitoring network traffic, system calls, file system modifications, and other interactions.

Malware code analysis

Code analysis (which includes reverse engineering) involves disassembling or decompiling the malware's code to understand its logic, functions, algorithms, and employed techniques. 

This helps in identifying concealed functionalities, exploitation methods, encryption methods, details about the C2 infrastructure, and techniques used for obfuscation and evasion. 

💡Note: Code analysis can also help analysts infer ways to discover potential indicators of compromise (IOC).

Memory analysis

Analyzing the malware's interactions with system memory helps identify injected code, hooks, or other runtime manipulations. 

This can be instrumental in detecting rootkits, analyzing anti-analysis techniques, or identifying malicious payloads.

Malware unpacking

This technique refers to the process of extracting and isolating the hidden malicious code within a piece of malware that uses packing techniques to evade detection. 

Malware authors may use packers to compress, encrypt, or obfuscate their malicious code, making it harder for antivirus software and other SOC analyst tools to identify the threat. 

Unpacking involves reverse-engineering these packing techniques to reveal the original, unobfuscated code for further analysis. 

This can allow researchers to understand the malware's functionality, behavior, and potential impact.

Malware analysis steps 

Now, we’ll look at the different steps involved in static malware analysis. 

How to conduct static malware analysis

In malware analysis, we exercise a method called static analysis to study malware without necessitating its execution. This involves the meticulous investigation of malware's code, data, and structural components, serving as a vital precursor for further, more detailed analysis.

This helps us extract important information like:

• File type.

• File hash.

• Strings.

• Embedded elements.

• Packer information.

• Imports.

• Exports.

• Assembly code.

Step 1: Identifying the file type

Our first port of call in this stage is to understand the basic information about the malware specimen to lay the groundwork for our investigation. 

Given that file extensions can be manipulated and changed, our task is to find a way to identify the actual file type we are encountering. 

Establishing the file type plays an integral role in static analysis, because it ensures that the procedures we apply are appropriate and the results obtained are accurate.

Let's use a Windows-based malware named Ransomware.wannacry.exe residing in the /home/htb-student/Samples/MalwareAnalysis directory of this module’s target as an illustration.

The command for checking the file type of this malware would be the following:

/home/htb-student/Samples/MalwareAnalysis/Ransomware.wannacry.exe

/home/htb-student/Samples/MalwareAnalysis/Ransomware.wannacry.exe: PE32 executable (GUI) Intel 80386, for MS Windows

From this, we would learn that it is a Portable Executable32 file:

Step 2: Malware fingerprinting

In this stage, our mission is to create a unique identifier for the malware sample. This typically takes the form of a cryptographic hash—MD5, SHA1, or SHA256. 

To do this, we will employ Fingerprinting techniques. Fingerprinting allows us to perform a variety of tasks, including:

• Identification and tracking of malware samples.

• Scanning an entire system for the presence of identical malware.

• Confirmation of previous encounters and analyses of the same malware.

• Sharing with stakeholders as IoC (Indicators of Compromise) or as part of threat intelligence reports.

To check the MD5 file hash of our example malware, we use the following command, which returns the following results:

md5sum /home/htb-student/Samples/MalwareAnalysis/Ransomware.wannacry.exe

db349b97c37d22f5ea1d1841e3c89eb4  /home/htb-student/Samples/MalwareAnalysis/Ransomware.wannacry.exe

Algorithm       Hash                                                                   Path

---------       ----                                                                   ----

MD5             DB349B97C37D22F5EA1D1841E3C89EB4                                       C:\Samples\MalwareAnalysis\Ra...

To check the SHA256 file hash of the abovementioned malware, we use the following command and receive the following results:

PS C:\Users\htb-student> Get-FileHash -Algorithm SHA256 C:\Samples\MalwareAnalysis\Ransomware.wannacry.exe

Algorithm       Hash                                                                   Path

---------       ----                                                                   ----

SHA256          24D004A104D4D54034DBCFFC2A4B19A11F39008A575AA614EA04703480B1022C       C:\Samples\MalwareAnalysis\Ra..

Step 3: File hash lookup

This next step involves checking the file hash produced in the prior step against online malware scanners and sandboxes, like VirusTotal. 

It’s an online malware scanning engine that collaborates with various antivirus vendors and allows us to search for the file hash. This step aids us in comparing our results with existing knowledge about the malware sample.

The following image shows the results from VirusTotal after the SHA256 file hash of the aforementioned malware we submitted.

Even though a file hash like MD5, SHA1, or SHA256 is valuable for identifying identical samples with disparate names, it doesn’t help us as much with identifying similar malware samples. 

This is primarily because a malware author can alter the file hash value by making minor modifications to the code and recompiling it.

Step 4: String analysis

In this phase, our objective is to extract ASCII & Unicode strings from a binary. 

Strings can provide us with clues and valuable insight into the functionality of the malware. 

Occasionally, we can unearth unique embedded strings in a malware sample, such as:

• Embedded filenames (e.g., dropped files).

• IP addresses or domain names.

• Registry paths or keys.

• Windows API functions.

• Command-line arguments.

• Unique information that might hint at a particular threat actor.

The following command will reveal strings for a ransomware sample named dharma_sample.exe residing in the /home/htb-student/Samples/MalwareAnalysis directory of this module’s target.

strings -n 15 /home/htb-student/Samples/MalwareAnalysis/dharma_sample.exe

!This program cannot be run in DOS mode.

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@>@@@?456789:;<=@@@@@@@

!"#$%&'()*+,-./0123@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/

WaitForSingleObject

InitializeCriticalSectionAndSpinCount

LeaveCriticalSection

EnterCriticalSection

C:\crysis\Release\PDB\payload.pdb

0123456789ABCDEF

Note: The -n flag specifies to print a sequence of at least the number specified—in our case, 15.

Occasionally, string analysis can facilitate the linkage of a malware sample to a specific threat group if significant similarities are identified.

For example, we can use a string containing a PDB path to link the malware sample to the Dharma/Crysis family of ransomware.

The use cases for malware analysis in cybersecurity

Malware analysis is the process of understanding the behavior and inner workings of malware. It is a crucial aspect of cybersecurity that aids in grasping the threat posed by malicious software and devising effective countermeasures.

Malware analysis serves several use cases:

• Detection and classification: Through analyzing malware, we can identify and categorize different types of threats based on their unique characteristics, signatures, or patterns. This helps us develop detection rules and understand the nature of the malware we encounter.

• Reverse engineering: Malware analysis often involves the intricate process of reverse engineering the malware's code to discern its underlying operations and employed techniques. This can unveil concealed functionalities, encryption methods, details about the command-and-control (C2) infrastructure, and techniques used for obfuscation and evasion.

• Behavioral analysis: Studying the behavior of malware during execution gives us insights into its actions. Such as modifications to the file system, network communications, changes to the system registry, and attempts to exploit vulnerabilities. This information about the impact of the malware on infected systems and assists in devising potential countermeasures.

• Threat intelligence: Through malware analysis, threat researchers can amass critical intelligence about attackers, their tactics, techniques, and procedures (TTPs), and the malware's origins. This valuable intelligence can be shared with the wider security community to enhance detection, prevention, and response capabilities.

Hone your defensive security skills with HTB

Malware analysis is an essential part of blue team training and should be a part of your foundational training. 

At HTB, we’ve got you covered with our comprehensive Academy Module: Introduction to Malware Analysis. You’ll learn the basics of malware analysis, and how to conduct your own tests. 

If you prefer practicing your skills in a realistic environment, try our free Sherlocks labs for blue teamers. 

Explore HTB Academy

Play HTB Sherlocks