We build a lot of hands-on-labs for events like Microsoft Ignite. These labs sometimes help evaluate new technologies or simply provide much needed training. Some are easy to build, while others aren’t. This article is about one of those that took an incredible amount of stubbornness to get right. (We specialize in stubbornness.) I’m talking about our Credential Guard hands-on lab, which took several iterations to nail.
What is Credential Guard?
When you sign in to a Windows device, it authenticates your user name and password to create a derived credential. In this case, that’s an NTLM hash, which is basically a long string of characters that represent your authenticated identity on the network. This hash allows you to access tools and data without typing your password every time.
It’s a real timesaver. However, this system has a fundamental flaw: If someone steals the hash that represents your authenticated identity, that person can effectively become you on the network. The attacker doesn’t need your user name or password to impersonate you. Ouch. Turns out, stealing your derived credentials by using tools that are freely available on the internet is super easy to do.
However, Windows 10 has a solution: Credential Guard. It stores important secrets like derived credentials in an encrypted virtualized container so they’re not directly readable from memory. Credential Guard acts like a security gate between the credential and the operating system.
It’s kind of like storing treasures in a castle that’s protected by an imposing mote and gatehouse. Pretty cool, and this is what we wanted to demonstrate in our Credential Guard hands-on lab.
Credential Guard hands-on lab goals
We think that Credential Guard is important security technology (so does our client, Microsoft). Therefore, we wanted to demonstrate it from beginning to end. These were our goals:
- Successfully attack a user’s credentials to demonstrate how easy they are to steal.
- Walk participants through a Credential Guard deployment by using Group Policy and manual configuration to demonstrate how easy it is to set up.
- Demonstrate how to verify that Credential Guard is configured correctly.
- Attack a user’s credentials again—unsuccessfully this time. This step demonstrates that Credential Guard has immediate value after very little deployment pain.
- Use advanced Credential Guard capabilities in the hands-on lab, such as certificate storage. For example, we wanted to cover advanced scenarios like using certificates bound to Credential Guard in Active Directory Federation Services authentication policies. This practice helps ensure administrative users sign in to only those machines on which Credential Guard protects their credentials.
Credential Guard hands-on lab challenges
Most of the hands-on labs we develop require trial and error. We invariably make several passes to configure the scenarios before we get them just right. In fact, we often work with the Microsoft product groups or program managers to troubleshoot issues because the features we’re demonstrating aren’t generally available to the public, yet.
It took five passes at Credential Guard before we got it to hands-on lab perfection. Needless to say, we’re very proud of this lab, partly because of the amount of work we put into it but also because of how well we think it demonstrates Credential Guard. The following timeline describes each pass:
Version 1
When we first started this journey, the Credential Guard feature was brand new and available only in preview versions of Windows 10. We spent weeks creating this lab but couldn’t get it to work. We ended up abandoning this first version because the feature itself wasn’t ready for prime time. Sigh.
Version 2
Because Credential Guard stores derived credentials in a virtualized container, demonstrating it in a hands-on lab environment requires nested virtualization. However, Microsoft Hyper-V didn’t provide this capability during our first iteration. So, we could show how to enable Credential Guard but couldn’t verify that it was working.
This limitation made it futile to show the credentials available in memory. Because, we wouldn’t be able to show Credential Guard protecting them. To show the certificate storage portion, we simulated the effect of binding the certificates to Credential Guard simply by using machine certificates. It was a simulation, but it was conceptually accurate.
This version of the Credential Guard hands-on lab checked off a few of our goals but still fell short. Nonetheless, we published this lab at Microsoft Ignite.
Version 3
So close, yet so far. Nested virtualization was now available in Hyper-V. However, our hands-on lab platform still didn’t support it in the data centers where we host labs. We still couldn’t show Credential Guard working in a hands-on lab environment, but we continued to support the limited lab online and at shows like Microsoft Ready. All we did this time around was refresh the lab guide and test.
Version 4
Finally. Phew. Our hands-on lab platform supported nested virtualization in the data centers. Now, we could achieve the perfect Credential Guard hands-on lab.
Not so fast. We had the required virtualization capabilities and could nail this lab, but the most recent Windows 10 update introduced a bug that blocked us. Certutil.exe enables Credential Guard certificate binding, and the most recent update broke it.
We still made progress, though. We could check off the first four goals, including:
- Demonstrate an attack.
- Enable Credential Guard.
- Test the Credential Guard configuration.
- Verify Credential Guard protection.
Version 5
After Microsoft patched the bug that broke Certutil.exe, we updated the hands-on lab to include the last goal—that is, we added an exercise to bind the machine certificate to Credential Guard.
After maintaining the lab for almost 2 years, we had finally nailed it. It took a lot of time and patience, but we’ve published what we feel like is the perfect Credential Guard hands-on lab. Happiness.
Final thoughts about hands-on labs
We feel strongly that hands-on labs should go deep.
Just demonstrating how to configure Credential Guard isn’t enough. Learners should attack the computer’s derived credentials, enable Credential Guard, and try again. Not only that, but a good Credential Guard hands-on lab should demonstrate advanced use cases.
What do you think makes a great hands-on lab? Is it depth? Is it breadth? What’s the best hands-on lab you’ve completed? Leave your comments below. We love to talk about hands-on labs, so reach out and say hello to start a conversation. We look forward to hearing from you.
