AV-TEST: Cylance Advanced Threat Prevention Test
Date: Feb 20, 2017
Cylance recently commissioned AV-TEST* to perform an advanced threat prevention test of enterprise endpoint protection software.
The testing methodology was developed jointly to provide additive testing to the commodity Anti-Virus protection tests currently produced by AV-TEST.
CylancePROTECT® was tested against 5 competitor endpoint products from Kaspersky, McAfee, Sophos, Symantec and Trend Micro. The test was performed in December 2016 and January 2017.
The report contains the results of four test cases. The primary goal was to show the detection and prevention capabilities of new and unknown malicious executables.
Test Case 1 – The Holiday Test: The purpose of this test is to reproduce a real world scenario whereby an end user goes on holiday for a given period. Upon returning from the holiday the end user returns to their endpoint and gets infected prior to the endpoint being able to update their protection measures. This is also a scenario encountered by organisations who may ultimately be forced to go to extreme measures such as delaying signature updates by a quarter due to performance impact to the user. This scenario is also a fairly easy way to test for “zero-day” detection and prevention capabilities. The test itself freezes the product at Day 0 and then removes it from being online. We then wait 7 days and collect new malware (executables) that are considered newly discovered on day 7 and start testing. We bring up the frozen product without connectivity to the internet so that the protection measures are essentially 7 days old. Testing is performed completely offline with the outdated protection measures. Products were unable to update themselves or query the cloud. We then ran the newly discovered malware against the products for detection and prevention efficacy against what essentially would be unknowns to the security solution being tested.
Test Case 2 – Simulated Attacks: The second test simulated a targeted attack where an attacker was able to introduce an executable file on the system. These executables were created by AV-TEST to simulate certain types of attacks that had to be detected and blocked by the products. These executables are based upon common advanced attacks seen today. The new zero-days are executed on systems first in offline mode to validate endpoint security solutions ability to detect true unknown attacks without connectivity to the cloud. And then online to show the impact of cloud queries.
Test Case 3 – Malware distributed by Websites: This test looked at malware executables delivered via websites. The URL itself is not malicious, it’s the content of the website that is malicious. This test turns off URL Filtering of all products under test to determine if they can truly detect malicious nature of visited website.
Test Case 4 – False Positives: Every protection test should be verified by a false positive test to make sure, good detection rates are not sacrificed for usability and false positives. In order to test this we downloaded, installed and actually used 38 different, common applications including Adobe reader, Google Chrome, Java JDK or Skype. Any warning messages or blockings of the tested protection product was noted.
In all test cases CylancePROTECT® showed extremely high efficacy prevention rates. They have a very reliable approach that works offline, without the need for regular updates even before execution of the malware. It also shows the dependency for the other products on regular updates, cloud queries or dynamic analysis.
The tests have shown that CylancePROTECT® is able to detect unknown attacks, while most of the other tested vendors could not demonstrate this ability.
Request a copy the full report here >>
The tests have shown that CylancePROTECT® is able to detect and prevent unknown attacks, while the other vendors have more problems with new attacks.
*AV-TEST GmbH is an independent supplier of services in the fields of IT Security and Antivirus Research, focusing on the detection and analysis of the latest malicious software and its use in comprehensive comparative testing of security products.