Matt Swann is a principal software engineer for Office 365 and Vijay Kumar is a senior product manager for Office 365.
In our last post, From Inside the Cloud: What does Microsoft do to prepare for emerging security threats to Office 365?, we looked at the Red team’s activities and what goes into our “Assume Breach” approach for preparing for emerging security threats to Office 365.
The rigor and discipline required to testing ourselves continuously to keep your data safe within the service, is by nature an operation of tremendous scale especially when you consider the terabytes (TB) of data flowing daily through the service.
And it’s the Blue team’s job, to literally find that potential needle in the haystack of activity that may signify anomalous behavior and to then take action. Which is what we explain this week’s episode of From Inside the Cloud.
To achieve this scale, we use the Cloud itself through a big data system that holds petabytes of data on events within the system—including security, process starts, network activity, Microsoft engineer activity like logons and much more—and classify activity types across servers globally in Office 365.
At the highest level, the framework for classification that we use determines three sets of possible activity. The ‘Known Good’ processes that we have identified as part of the day-to-day running of the service. The ‘Known Bad’ processes where we work with our Red team of penetration testers to determine what real world adversaries would do and then zero in on activities that may meet this description.
And as we evolve our knowledge within these categories, what we are really looking for is the activity that does not fit in the first two buckets—the ‘Unknown Unknowns’ where we are proactively seeking to unearth anomalous behavior. This is where we deploy machine learning to model behavior and detect patterns in activity that require further forensics and investigation and possible action by our security team.
The refinement and focus to increase our detection and response times is ongoing. Our Red team in particular keeps us on our toes daily—in fact, from the Blue team’s perspective, we never know if we are dealing with a real or simulated attack until it is revealed to us at the end.
Please continue to let us know if you have any further questions that we can answer in this blog and in this series.
As always you can find other recent episodes and additional resources on our Office 365 Trust Center.
You may also want to check out our recent Garage Series Under the Hood: Continually Safeguarding your Data in the Office 365 Service for a more in depth look into the intrusion detection process.
— Matt Swann and Vijay Kumar