We recently deployed a remote office in China, where we were tunneling all traffic back to a central location to be filtered by a proxy. It didn’t take long for complaints of slowness to start coming in, and understandably so. At the time we had no way of filtering the traffic through a local proxy, but this later changed and we were able to take advantage of Websense in the cloud (look for a future article on this). We ended up sending Internet traffic out locally from the site, filtered by a Websense agent and expected all of the complaints to disappear. But they didn’t. After looking into the issue one of our engineers found that we didn’t think about how DNS traffic would flow in this setup. While Internet traffic itself was leaving the site locally, DNS traffic was still coming back to the US for resolution. This presents two main problems:
- The DNS request/response had to come back to the US, which takes about 300 ms RTT. That’s 300 ms extra time added on to however long it takes to load your site.
- Since the DNS request was taking place in the US, any website that uses a service like Akamai or some other geographical load balancing was serving up the sites closest to the US. So you wind up with Internet traffic leaving the office in China locally, and then still coming all the way back to the ‘best’ server closest to where the DNS resolution was done in the US. Really no better then you were to start.
This presented a problem. How do we perform DNS lookups for all of the Internet sites using some local Chinese ISP DNS, while still sending DNS requests for internal sites and websites to our own internal corporate servers. A quick google turned up a Cisco feature called split DNS which did exactly this.
The concept behind split DNS is pretty straightforward. Cisco allows you to setup multiple DNS views, each with a different DNS server, that directs traffic based on certain parameters that you pick. To make this work you also end up changing your clients to point to the router interfaces themselves for DNS servers, and the router then forwards on the request to the appropriate DNS server depending on the criteria you set. In our case we were using the Cisco router for DHCP as well, so we modified the Cisco DHCP scope to include the router itself as the DNS server.
ip dns server ! enable the DNS server on the Cisco routerip dns view corporate-internal ! Define a view called ‘corporate-internal’ which contains corporate DNS Server IP addressesdns forwarder <your_internal_DNS_server_IP>dns forwarder <your_backup_internal_DNS_server_IP>ip dns view default ! Define a view called ‘default’ which contains non corporate, public DNS Server IP Addressesdns forwarder 18.104.22.168 ! China public DNS servers, similar to Google’s 22.214.171.124dns forwarder 126.96.36.199ip dns view-list dnsview !This view-list assigns a priority to each of the DNS views from above, and also links the ‘corporate internal’ view servers to the name-group ‘1’ . Name-group 1 refers to the ip dns name-list 1 lines in the next section, similar to an ACL, but DNSview corporate-internal 10 !bind the view called ‘corporate-internal’ to the group called name-grouprestrict name-group 1 ! The ‘1’ here refers to the DNS name-list in the section belowview default 99ip dns name-list 1 permit 10\IN-ADDR !match reverse DNS records for the 10/8 netip dns name-list 1 permit .*.yourdomain.com !match anything ending in your domain.comip dns server view-group dnsview !Apply the DNS ‘view-list’ to the Cisco DNS Server
Once this config is in there, it works like this. Anytime a DNS query from a client comes in to the router, it will look to see what site you are trying to go to. If it matches the ‘name-list’ in our example, so say intranet.yourdomain.com, it will then know to use the ‘corporate-internal’ view and forward that DNS request to the appropriate internal servers. If the domain name you are trying to reach does not match the name-list, for example google.com, then you will fall down to the DNS view called ‘default’ in the view-list, which will forward your DNS request to a different set of DNS servers.
After making these changes we saw pretty impressive improvements. We got rid of the 300+ ms RTT for the DNS request itself, and in addition we were now getting geographically appropriate results for the DNS queries which means that the servers returned to us were usually much closer, and therefore quicker. There are a ton of other options and complexity you can add to this feature. If you are interested in learning more check out this Cisco page to get started.