Generally speaking, AV detection can be classified into three main approaches:
- Static Detection
- Dynamic Detection
- Heuristic and Behavioral Detection
Static Detection
A static detection technique is the simplest type of Antivirus detection, which is based on predefined signatures of malicious files. Simply, it uses pattern-matching techniques in the detection, such as finding a unique string, CRC (Checksums), sequence of bytecode/Hex values, and Cryptographic hashes (MD5, SHA1, etc.).
It then performs a set of comparisons between existing files within the operating system and a database of signatures. If the signature exists in the database, then it is considered malicious. This method is effective against static malware.
In this task, we will be using a signature-based detection method to see how antivirus products detect malicious files. It is important to note that this technique works against known malicious files only with pre-generated signatures in a database. Thus, the database needs to be updated from time to time.
We will use the ClamAV antivirus software to demonstrate how signature-based detection identifies malicious files. The ClamAV software is pre-installed in the provided VM, and we can access it in the following path: c:\Program Files\ClamAV\clamscan.exe. We will also scan a couple of malware samples, which can be found on the desktop. The Malware samples folder contains the following files:
- EICAR is a test file containing ASCII strings used to test AV software's effectiveness instead of real malware that could damage your machine. For more information, you may visit the official EICAR website here.
- Backdoor 1 is a C# program that uses a well-known technique to establish a reverse connection, including creating a process and executing a Metasploit Framework shellcode.h
- Backdoor 2 is a C# program that uses process injection and encryption to establish a reverse connection, including injecting a Metasploit shellcode into an existing and running process.
- AV-Check is a C# program that enumerates AV software in a lab machine. Note that this file is not malicious. We will discuss this tool in more detail in task 6.
- notes.txt is a text file that contains a command line. Note that this file is not malicious.
ClamAV comes with its database, and during the installation, we need to download the recently updated version. Let's try to scan the Malware sample folder using the clamscan.exe binary and check how ClamAV performs against these samples.
Command Prompt
c:\>"c:\Program Files\ClamAV\clamscan.exe" c:\Users\thm\Desktop\Samples
Loading: 22s, ETA: 0s [========================>] 8.61M/8.61M sigs
Compiling: 4s, ETA: 0s [========================>] 41/41 tasks
C:\Users\thm\Desktop\Samples\AV-Check.exe: OK
C:\Users\thm\Desktop\Samples\backdoor1.exe: Win.Malware.Swrort-9872015-0 FOUND
C:\Users\thm\Desktop\Samples\backdoor2.exe: OK
C:\Users\thm\Desktop\Samples\eicar.com: Win.Test.EICAR_HDB-1 FOUND
C:\Users\thm\Desktop\Samples\notes.txt: OK
The above output shows that ClamAV software correctly analyzed and flagged two of our tested files (EICAR, backdoor1, AV-Check, and notes.txt) as malicious. However, it incorrectly identified the backdoor2 as non-malicious while it does.
You can run clamscan.exe --debug <file_to_scan>, and you will see all modules loaded and used during the scanning. For example, it uses the unpacking method to split the files and look for a predefined malicious sequence of bytecode values, and that is how it was able to detect the C# backdoor 1. The bytecode value of the Metasploit shellcode used in backdoor 1 was previously identified and added to ClamAV's database.
However, backdoor 2 uses an encryption technique (XOR) for the Metasploit shellcode, resulting in different sequences of bytecode values that it doesn't find in the ClamAV database.
While the ClamAV was able to detect the EICAR.COM test file as malicious using the md5 signature-based technique. To confirm this, we can re-scan the EICAR.COM test file again in debug mode (--debug). At some point in the output, you will see the following message:
LibClamAV debug: FP SIGNATURE: 44d88612fea8a8f36de82e1278abb02f:68:Win.Test.EICAR_HDB-1 # Name: eicar.com, Type: CL_TYPE_TEXT_ASCII
Now let's generate the md5 value of the EICAR.COM if it matches what we see in the previous message from the output. We will be using the sigtool for that:
Command Prompt
c:\>"c:\Program Files\ClamAV\sigtool.exe" --md5 c:\Users\thm\Desktop\Samples\eicar.com
44d88612fea8a8f36de82e1278abb02f:68:eicar.com
If you closely check the generated MD5 value, 44d88612fea8a8f36de82e1278abb02f, it matches.
Create Your Own Signature Database
One of ClamAV's features is creating your own database, allowing you to include items not found in the official ClamAV database. Let's try to create a signature for Backdoor 2, which ClamAV already missed, and add it to a database. The following are the required steps:
- Generate an MD5 signature for the file.
- Add the generated signature into a database with the extension ".hdb".
- Re-scan the ClamAV against the file using our new database.
First, we will be using the sigtool tool, which is included in the ClamAV suite, to generate an MD5 hash of backdoor2.exe using the --md5 argument.
Generate an MD5 hash
C:\Users\thm\Desktop\Samples>"c:\Program Files\ClamAV\sigtool.exe" --md5 backdoor2.exe
75047189991b1d119fdb477fef333ceb:6144:backdoor2.exe
As shown in the output, the generated hash string contains the following structure: Hash:Size-in-byte:FileName. Note that ClamAV uses the generated value in the comparison during the scan.
Now that we have the MD5 hash, now let's create our own database. We will use the sigtool tool and save the output into a file using the > thm.hdb as follows,
Generate our new database
C:\Users\thm\Desktop\Samples>"c:\Program Files\ClamAV\sigtool.exe" --md5 backdoor2.exe > thm.hdb
As a result, a thm.hdb file will be created in the current directory that executes the command.
We already know that ClamAV did not detect the backdoor2.exe using the official database! Now, let's re-scan it using the database we created, thm.hdb, and see the result!
Re-scanning backdoor2.exe using the new database!
C:\Users\thm\Desktop\Samples>"c:\Program Files\ClamAV\clamscan.exe" -d thm.hdb backdoor2.exe
Loading: 0s, ETA: 0s [========================>] 1/1 sigs
Compiling: 0s, ETA: 0s [========================>] 10/10 tasks
C:\Users\thm\Desktop\Samples\backdoor2.exe: backdoor2.exe.UNOFFICIAL FOUND
As we expected, the ClamAV tool flagged the backdoor2.exe binary as malicious based on the database we provided. As a practice, add the AV-Check.exe's MD5 signature into the same database we already created, then check whether ClamAV can flag AV-Check.exe as malicious.
Yara Rules for Static Detection
One of the tools that help in static detection is Yara. Yara is a tool that allows malware engineers to classify and detect malware. Yara uses rule-based detection, so in order to detect new malware, we need to create a new rule. ClamAV can also deal with Yara rules to detect malicious files. The rule will be the same as in our database in the previous section.
To create a rule, we need to examine and analyze the malware; based on the findings, we write a rule. Let's take AV-Check.exe as an example and write a rule for it.
First, let's analyze the file and list all human-readable strings in the binary using the strings tool. As a result, we will see all functions, variables, and nonsense strings. But, if you look closely, we can use some of the unique strings in our rules to detect this file in the future. The AV-Check uses a program database (.pdb), which contains a type and symbolic debugging information of the program during the compiling.
Command Prompt
C:\Users\thm\Desktop\Samples>strings AV-Check.exe | findstr pdb
C:\Users\thm\source\repos\AV-Check\AV-Check\obj\Debug\AV-Check.pdb
We will use the path in the previous command's output as our unique string example in the Yara rule that we will create. The signature could be something else in the real world, such as Registry keys, commands, etc. If you are not familiar with Yara, then we suggest checking the Yara THM room. The following is Yara's rule that we will use in our detection:
rule thm_demo_rule {
meta:
author = "THM: Intro-to-AV-Room"
description = "Look at how the Yara rule works with ClamAV"
strings:
$a = "C:\\Users\\thm\\source\\repos\\AV-Check\\AV-Check\\obj\\Debug\\AV-Check.pdb"
condition:
$a
}
Let's explain this Yara's rule a bit more.
- The rule starts with rule thm_demo_rule, which is the name of our rule. ClamAV uses this name if a rule matches.
- The metadata section, which is general information, contains the author and description, which the user can fill.
- The strings section contains the strings or bytecode that we are looking for. We are using the C# program's database path in this case. Notice that we add an extra \ in that path to escape the special character, so it does not break the rule.
- In the condition section, we specify if the defined string is found in the string section, then flag the file.
Note that Yara rules must store in a .yara extension file for ClamAV to deal with it. Let's re-scan the c:\Users\thm\Desktop\Samples folder again using the Yara rule we created. You can find a copy of the Yara rule on the desktop at c:\Users\thm\Desktop\Files\thm-demo-1.yara.
Scanning using the Yara rule
C:\Users\thm>"c:\Program Files\ClamAV\clamscan.exe" -d Desktop\Files\thm-demo-1.yara Desktop\Samples
Loading: 0s, ETA: 0s [========================>] 1/1 sigs
Compiling: 0s, ETA: 0s [========================>] 40/40 tasks
C:\Users\thm\Desktop\Samples\AV-Check.exe: YARA.thm_demo_rule.UNOFFICIAL FOUND
C:\Users\thm\Desktop\Samples\backdoor1.exe: OK
C:\Users\thm\Desktop\Samples\backdoor2.exe: OK
C:\Users\thm\Desktop\Samples\eicar.com: OK
C:\Users\thm\Desktop\Samples\notes.txt: YARA.thm_demo_rule.UNOFFICIAL FOUND
As a result, ClamAV can detect the AV-Check.exe binary as malicious based on the Yara rule we provide. However, ClamAV gave a false-positive result where it flagged the notes.txt file as malicious. If we open the notes.txt file, we can see that the text contains the same path we specified in the rule.
Let's improve our Yara rule to reduce the false-positive result. We will be specifying the file type in our rule. Often, the types of a file can be identified using magic numbers, which are the first two bytes of the binary. For example, executable files (.exe) always start with the ASCII "MZ" value or "4D 5A" in hex.
To confirm this, let's use the HxD application, which is a freeware Hex Editor, to examine the AV-Check.exe binary and see the first two bytes. Note that the HxD is already available in the provided VM.
Knowing this will help improve the detection, let's include this in our Yara rule to flag only the .exe files that contain our signature string as malicious. The following is the improved Yara rule:
rule thm_demo_rule {
meta:
author = "THM: Intro-to-AV-Room"
description = "Look at how the Yara rule works with ClamAV"
strings:
$a = "C:\\Users\\thm\\source\\repos\\AV-Check\\AV-Check\\obj\\Debug\\AV-Check.pdb"
$b = "MZ"
condition:
$b at 0 and $a
}
In the new Yara rule, we defined a unique string ($b) equal to the MZ as an identifier for the .exe file type. We also updated the condition section, which now includes the following conditions:
- If the string "MZ" is found at the 0 location, the file's beginning.
- If the unique string (the path) occurs within the binary.
- In the condition section, we used the AND operator for both definitions in 1 and 2 are found, then we have a match.
You can find the updated rule in Desktop\Files\thm-demo-2.yara. Now that we have our updated Yara rule, now let's try it again.
Scanning using the Yara rule
C:\Users\thm>"c:\Program Files\ClamAV\clamscan.exe" -d Desktop\Files\thm-demo-2.yara Desktop\Samples
Loading: 0s, ETA: 0s [========================>] 1/1 sigs
Compiling: 0s, ETA: 0s [========================>] 40/40 tasks
C:\Users\thm\Desktop\Samples\AV-Check.exe: YARA.thm_demo_rule.UNOFFICIAL FOUND
C:\Users\thm\Desktop\Samples\backdoor1.exe: OK
C:\Users\thm\Desktop\Samples\backdoor2.exe: OK
C:\Users\thm\Desktop\Samples\eicar.com: OK
C:\Users\thm\Desktop\Samples\notes.txt: OK
The output shows we improved our Yara rule to reduce the false-positive results. That was a simple example of how AV software works. Thus, AV software vendors work hard to fight against malware and improve their products and database to enhance the performance and accuracy of results.
The drawback of the signature-based detection is that files will have a different hash value if the binary is modified. Therefore, it is easy for someone to bypass signature-based detection techniques if they know what AV software looks for and how to analyze binaries, as shown in later rooms.