NETID Domain in Azure


A common scenario is for an Azure hosted virtual machine (VM) to communicate with the netid.washington.edu Active Directory (AD) domain.  Microsoft Infrastructure (MI) has established an AD site in Azure and has established key technologies to support a hub and spoke architecture to accommodate this scenario.  Requesting VNet peering to MI's hub VNet will allow domain communications to be established and remain in Azure, which includes domain joining and AD authentication for servers and applications which require AD authentication. Depending upon customer needs, a combination of VNet Peering, DNS, and ExpressRoute may be utilized to establish AD connectivity and optionally extend connectivity to UW networks.  In order to utilize the AD Azure site and VNet peering, a customer must meet the prerequisites for VNet Peering with the MI hub VNet.

Finding the right fit - is VNet peering right for me?

I want domain joined VMs in Azure AND:

Related Actions

The following prerequisites must be met in order to request a VNet Peering connection to Microsoft Infrastructure's hub Vnet.
  1. Request an Azure subscription within the UW's protected and discounted contract
  2. Request an address block from Network Operations for use in Azure
  3. Create a VNet using the above address space.
  4. Grant the group u_msinf_service_serviceteam_sadm the Network Contributor role to that VNet. The role may be assign and inherited from the subscription or resource group if desired.
To request VNet peering, send a request to help@uw.edu with the following details:
  • Address Space
  • Desired VNet link name
  • VNet Resource ID (Available on the VNet properties page)
  • Whether the use of the UW-IT Shared Express Route connection is needed
Note: Network connection properties, such as DNS server IPs, should not be edited directly within VMs; instead these settings should be specified at the VNet level.
s]
Azure services that are bound to a Azure public IP address may suffer from connectivity issues due to asymmetric routing. Services such as a VM, app service or load balancer with a public IP are examples. If a campus resource attempts to connect to an Azure public IP, it does so over the public internet. ExpressRoute advertises campus address space to the VNet hosting the Azure resource which attempts to return to the original source of traffic over ExpressRoute which prevents establishing a connection. A common solution is to:
  1. Create new subnet in the Azure VNet
  2. Create a new route table and set Propagate Gateway Routes to 'No'
  3. Associate the new route table to the new subnet create in 1

Understanding the Key Technologies

The NetID domain has an AD site defined for activity in the Azure public cloud. UW-IT network engineering has established 10.4.0.0/17 dedicated to VNets in Azure. The NetID domain has defined an AD site which encompasses this same address space. Replication and traffic between the two sites is maintained via an Azure ExpressRoute connection provided by UW-IT. This ExpressRoute connection is called the UW-IT Shared ExpressRoute. Domain joined computers which support the DC locator process (all Windows clients do) will prefer a domain controller which is in the same AD site as they are in, as described at https://docs.microsoft.com/en-us/windows-server/identity/ad-ds/plan/site-functions#client-affinity. These domain joined computers first find a random domain controller from any site. The domain controller will inform the client machine which site it belongs to based on it's source IP address. For a machine with an IP address in 10.4.0.0/17, the domain controller will refer the client to domain controllers in Azure for subsequent operations. NetID AD Site - Azure From Azure networks without a dedicated ExpressRoute of their own, two scenarios are possible with respect to traffic to NETID domain controllers:
  1. If the client machine is in a VNet that has a peering relationship with the MI hub VNet, but no access to the UW-IT Shared ExpressRoute, all traffic will remain in Azure.
  2. If the client machine is in a VNet that has a peering relationship with the MI hub VNet and the client can utilize the UW-IT Shared ExpressRoute, the client may first connect to a domain controller on the Seattle campus or Azure, then all subsequent traffic will remain in Azure.
From Azure networks with their own dedicated ExpressRoute, with respect to traffic to NETID domain controllers, another scenario is possible:
  1. If a peering relationship does not exist with the MI hub VNet, the client has its own path to UW networks via the customer dedicated ExpressRoute. That client may first connect to a domain controller on the Seattle campus or Azure (from the UW side of the UW-IT Shared ExpressRoute), but then all subsequent traffic will be to the Azure domain controllers traversing both the customer dedicated ExpressRoute and then the UW-IT Shared Express Route.
VNet Peering allows two Vnets to be connected together and appear as a single network. The peering relationship establishes a network connection between peered VNets. Routing between VNets and subnets are maintained by the Azure fabric. A VNet may be divided into one or more subnets. Azure services such as hosted VMs, Azure SQL, and App Services may be connected to the subnets. With VNet peering, services and VMs on those subnets may communicate with one another within Azure. An example peering relationship between two VNets which are both subdivided into subnets is shown in figure 1. Figure 1, VNet Peering Both VM1 and VM2, AzSQL and the AppSvc shown in figure 1 will be able to communicate over the established peering link.
A central, hub VNet is a recommended design pattern for hosting common infrastructure resources that are required by one or more spoke VNets. UW-IT's Microsoft Infrastructure (MI) has implemented a central hub VNet to enable customers to connect to the NETID Active Directory (AD) domain controllers and campus network resources. A VNet is the fundamental security boundary in Azure and is partly defined by an IP Address space. A VNet address space is divided into one or more subnets for use by customer resources. UW-IT has extended the campus address space into Azure by reserving 10.4.0.0/17 for use in Azure. A customer may establish a VNet and request that it connect to the MI hub VNet, via VNet Peering. The peering relationship may or may not utilize the UW-IT Shared Express Route connection. Multiple customer may establish a similar peering relationship to the MI hub VNet as shown in figure 1. Vnet Peer Hub and Spoke Figure 1 A customer must first request a reserved address space from UW-IT network operations to avoid IP address collisions elsewhere on UW networks in Seattle, Bothell and Tacoma. UW-IT will not peer to VNets that are determined to use IP addresses allocated in other address ranges. UW-IT may terminate peering if address spaces are discovered other than those reserved by UW-IT network engineering for use in Azure. The customer is responsible for managing their own Azure assets and resources including
  • Azure subscriptions, VNets, subnets and other resources
  • Troubleshooting their network connectivity between resources, e.g. asymmetric routing
  • Peering relationships with VNets other than MIs hub VNet
A pair of forwarding DNS servers have been establish to resolve DNS for hosts located on both UW and Azure networks. These forwarding DNS servers are called the UW-IT Azure DNS bridge solution. This is necessary because both UW and Azure operate a split-horizon DNS solution. This means UW-IT's external DNS view is available via Azure DNS but not UW-IT's internal DNS view. And likewise, Azure public DNS records are available via the UW DNS, but not DNS records in the Azure private view. Without a DNS bridge solution, you can only resolve private DNS records in one or the other--depending on which DNS server you choose as the resolver. The default Azure DNS configuration for VNets is to use Azure DNS. This configuration allows dynamic registration and name resolution of VMs and Azure resources within the VNet. This may be sufficient in some cases, but will not enable the use of UW-IT's private DNS view which includes use of Active Directory (AD) and other campus resources in private address spaces. This scenario, depicted in figure 1, allows VM1 and VM2 to resolve one another, but not SRV1 or SRV2. VM1 and VM2 dynamically register with Azure DNS, therefore Azure can resolve those names. Azure DNS is unaware of SRV1 and SRV2 in UW-IT's private view and is therefore unable to resolve those names. Figure 1, Azure DNS Alternatively, VNet's may be configured to use UW-IT's DNS servers over Express Route (or another site-to-site VPN solution) which enables name resolution for AD and other UW hosts in the private DNS view. This configuration, while enabling campus resources, prevents name resolution of Azure hosted VMs and resources. Figure 2 depicts a VNet configured to use UW-IT's campus DNS servers over Express Route. VM1 and VM2 will be able to resolve SRV1 and SRV2 (as well as VM1 and VM2 if they are registered in a campus DNS zone), but will be unable to resolve Azure SQL, App Service or other Azure services on private networks. Figure 2, Campus DNS In order to allow Azure hosted VMs and resources to resolve all 4 DNS views, the DNS servers in the UW-IT Azure DNS bridge solution use conditional forwarding to send all DNS requests in the uw.edu or washington.edu zone to UW-IT's DNS servers, and forward all other DNS requests to Azure's DNS server, 168.63.129.63. For example, figure 3 shows a DNS query by VM1 for vm2.my.uw.edu. The UW-IT Azure DNS bridge solution will forward this query to campus DNS servers for name resolution. On the other hand, a DNS query for portal.azure.com would result in the UW-IT Azure DNS bridge solution forwarding the query to the Azure DNS server for resolution. Figure 3, DNS Bridge From a campus technology perspective, the Azure hosted VMs and resources can be resolved using the standard campus DNS servers. To enable this, the customer continues to manage DNS records using the common UW-IT DNS management portal. Registering, for example, vm1.mydomain.uw.edu with an A record to the assigned private IP address in Azure. If a spoke VNet is utilizing private DNS, additional configuration with MI will allow a Azure private name resolution to take place through the UW-IT Azure DNS bridge solution.
An ExpressRoute circuit has been established to offer a persistent, shared, bidirectional communication between Azure and UW Networks. ExpressRoute is site-to-site VPN connection. ExpressRoute can enable customers to access Azure hosted VMs and resources without exposing resources to the internet with public IP addresses. It also can allow Azure VMs to access campus resources, e.g. SQL servers. An additional benefit is enabling communication between spoke VNets without additional network virtual appliances. If a hub and spoke customer needs to access resources, other than Active Directory, on the UW networks, ExpressRoute gateway transit will be enabled on the VNet peering relationship to enable traffic to flow between the UW network and the spoke VNet. The UW-IT Shared ExpressRoute connection is a shared resource. UW-IT expects customers to be good neighbors by not abusing the limited bandwidth available to the ExpressRoute circuit. It is not intended for:
  • Backups
  • Device and sensor streams
  • Frequent database replication and migration
  • VM replicas
  • Other large data transfers
UW-IT reserves the right to disable gateway transit, which will prevent connectivity with campus, but allow continued use of AD.

Common Issues and Solutions

Azure services that are bound to a Azure public IP address may suffer from connectivity issues due to asymmetric routing. Services such as a VM, app service or load balancer with a public IP are examples. If a campus resource attempts to connect to an Azure public IP, it does so over the public internet. ExpressRoute advertises campus address space to the VNet hosting the Azure resource which attempts to return to the original source of traffic over ExpressRoute which prevents establishing a connection. A common solution is to:
  1. Create new subnet in the Azure VNet
  2. Create a new route table and set Propagate Gateway Routes to 'No'
  3. Associate the new route table to the new subnet create in 1
Azure services in spoke VNets that depend upon Azure private DNS for name registration and name resolution may not work with Azure MI DNS servers. A common solution is to request MI add a virtual network link to the hub Private DNS zone.
A known limitation of the Azure MI DNS servers conditional forwarding configuration for uw.edu and washington.edu. If the ExpressRoute link is unavailable, name resolution for campus resources will continue for the value of the record's TTL. DNS resolution will resume immediately when the link is reestablished. The Azure MI DNS servers contain a the netid.washington.edu zone, so AD authentication will continue uninterrupted.

Regardless of what path you use, you'll want to carefully design to take into account:

The following picture is complex but does cover all of these issues, with dept1, dept3, and dept4 in broken configurations, and only dept2 in a fully functional configuration. There are other approaches which are fully functional--this diagram is mostly for illustrative purposes to show what not to do. oneuwit