Red Team vs Red Canary: How Sparring with Customers Improves Security

Suzanne Moore

Share this

The Red Canary team is a competitive bunch, especially when it comes to identifying emerging adversary tradecraft. We highly encourage our customers to run red team tests to validate our detection and response capabilities. The tougher, the better! Testing the newest and most advanced techniques keeps our Cyber Incident Response Team (CIRT) on its A-game and helps us all improve.

Meet one customer who regularly takes us up on this challenge: Eric, an information security engineer at a popular fast food chain. Eric is part of a mature security team charged with protecting approximately 450 macOS machines and 3,500 Windows machines across the restaurant’s corporate headquarters. Big believers in testing, his team consistently runs Red Canary’s CIRT through its paces.

Eric explained, “It’s been fun to see how the sparring between our team and Red Canary has manifested itself. I’ve improved some of my red team tactics from being caught by Red Canary so many times. From a detection perspective, I feel good when I can do some initial testing and fly under the radar for six or eight hours, versus getting an alert in twenty minutes.”

Detecting Emerging Techniques

This kind of testing is mutually beneficial. Our CIRT loves to create new detectors and detection processes based on red team engagements.

“We’ve always tried to be really open with the fact that we’re not ever going to be perfect,” commented Adam Mathis, a Red Canary incident handler. “The way we view it is: if we miss something in a test, good. It’s an opportunity to improve our defenses against an actual attack.”

Of course, we also love it when we catch something particularly tricky.

Eric recounted some fun he had sparring with the Red Canary team to test detection of an emerging technique: “I was watching a live stream of Matt Graeber from SpecterOps giving a talk at DerbyCon about subverting trust by modifying the behavior of Windows built-in code signing validation. I could see the sample code in his slides well enough to type the code and try it out. Sure enough, Red Canary detected it. Matt was still explaining how the technique worked and when I tried to emulate it, I got a Red Canary detection. I was blown away.”

As it turned out, one of our detection engineers was in the crowd at DerbyCon, developing a method of detecting Graeber’s technique just as Eric was creating a test for it.

Atomic Red Team Tests & Adversary Emulation

Another way Eric and his team spar with Red Canary is by running adversary emulation tests in his environment, both with Atomic Red Team tests and real-world post-exploit tooling. The screenshots below show an example of a macOS threat Red Canary detected when Eric was running some tests.

First, Red Canary detected Python executing and checking for firewalls.

Red Canary threat detection: Python executing and attempting to initiate outbound connections

Red Canary’s CIRT then detected Python attempting to initiate outbound connections—a common indicator of the Empire post-exploit framework on macOS. Our detection engineers provided context on observed reconnaissance and privilege escalation techniques.

Red Canary threat detection: reconnaissance and privilege escalation techniques

Eric commented, “I don’t think I’ve done anything Red Canary hasn’t ultimately caught. As soon as I start doing something an attacker would do, the detection team is all over it. They notify me of what’s happening and correlate it back to suspicious activity and the point of entry. Our testing is valuable to Red Canary as well; it’s an opportunity for both of us to improve. The better Red Canary gets, the better we get.”

Key Takeaways

We love all of our customers here at Red Canary, but we’re especially appreciative of those who deftly test our detection and response capabilities. Not because it demonstrates where we are strong, but because it shows us where we need to improve. Through quality testing and friendly competition, we can collaborate as a community and drive ongoing improvements.

Eric gives similar advice: “For other teams like ours, I’d recommend: Look at what you have today in terms of visibility. Go download Atomic Red Team and run it to see what you detect and what you don’t. If you’re lacking visibility and failing to detect a good portion of techniques in the ATT&CK matrix, you would benefit from looking at Red Canary.”

To learn more about Eric and his team’s security program, read the full case study: Finding Adversaries on MacOS Devices: How an Advanced Security Team Covers Gaps Across Its Fleet