How to test Datto EDR, Datto AV, and Ransomware Detection with third party security products

Prospective and existing customers evaluating Datto EDR, Datto AV, and ransomware detection capabilities against other vendors' tools should follow the guidelines below. Following them produces consistent, comparable results across products, keeps testing safe for your environment, and ensures our team has what it needs if a genuine detection gap is found.

Isolate real-time agents from one another

Running multiple real-time security agents side by side on the same machine doesn't just create an unfair test, it actively changes what each product can see and do. A result produced this way reflects the conflict between the two products, not either product's real detection capability.

Antivirus and EDR agents hook deep into the operating system, kernel drivers, ETW, AMSI, and filesystem minifilters. Two products with overlapping real-time protection should never share a single machine:

  • Each product can quarantine or interfere with the other's process, driver, or files; to a competing AV, an EDR agent's own behavior often looks like malware.
  • One agent can silently block the other's telemetry source, degrading its detection with no indication that this happened.
  • A "miss" observed in this configuration usually reflects the conflict between the two agents, not either product's actual detection capability.

IMPORTANT  Dedicate a separate machine (or a clean VM snapshot) to each product under evaluation. One machine should run only Datto AV/EDR; a second, separate machine should run only the competing product. Both products should never be installed on one host at the same time.

Recognize that different layers catch different things

Antivirus, EDR, and Ransomware Detection are not interchangeable. They are designed to catch different things at different points in an attack, and testing should account for this rather than treating all three as equivalent.

Layer What it catches When it fires
Antivirus (signature/heuristic) Known-bad files, hashes, and static patterns On execution, at or before the point of file access
EDR (behavioral) Suspicious process behavior, living-off-the-land techniques (where hackers misuse legitimate, pre-installed tools and processes), and post-exploitation activity During or after execution, sometimes with a short analysis delay
Ransomware detection Mass file modification/encryption patterns After some file activity has already occurred by design. It is a behavioral tripwire, not a preventive block

A test that only measures whether a file was stopped from landing on disk will make behavioral and ransomware-detection layers look like they "missed," when in fact they are designed to catch a different stage of an attack. Each layer should be evaluated against material appropriate to it.

Use safe, standardized test material — never live malware

Test material should match the layer being validated and should always remain contained:

  • For antivirus signature testing, use the EICAR anti-malware testfile, an industry-standard, benign string that every AV vendor detects. It is safe to use in production environments.
  • For EDR and behavioral testing, use a recognized adversary emulation framework (for example, MITRE ATT&CK Evaluations or Atomic Red Team) rather than live malware samples.
  • Datto EDR provides the Datto EDR Behavioral Attack Simulator, a PowerShell script that can emulate adversary activity on a device. The Readme.md file describes how to use it.
  • Live, uncontained malware should never be detonated on a production endpoint or on a machine connected to the production network. Testing of this kind belongs on an isolated VM or snapshot that can be rolled back afterward.

Treat a suspected miss as a case for review, not a conclusion

A product should not be considered to have "missed" a threat based on a single, isolated test run. Detection sometimes occurs later in the kill chain, or through a different mechanism than expected. Any suspected gap should be submitted to Datto EDR Support so the detection team can verify what actually happened. A well-formed submission includes:

  • The agent ID/hostname and timestamp of the test.
  • The sample file used, or the exact simulation tool and technique (for example, an Atomic Red Team test ID).
  • Whether any other antivirus, EDR, or security agent was present on the same machine during the test.
  • Agent logs from the device covering the time of the test. Logs can be gathered as an agent action uploaded to the Reports tab in the platform, from Organizations > Devices, using the ellipses menu on the applicable device and selecting Fetch Logs to pull current device logs into the Reports section. For more information, see Accessing agent log files.

Our team reviews each case submitted and confirms whether it represents a genuine detection gap or expected behavior for that layer.

 

Revision Date
Initial release. 7/10/26