Why do certain web sites load slowly from our office?
May 15, 2015 2:27 PM Subscribe
There are certain web sites (e.g. www.amd.com) which load incredibly slowly from our office. On the same laptops from anywhere else they load fine. What gives?
Problem: Most (80/90%?) web sites load fine. Some, such as amd.com and this themonthly.com.au link from the blue, will load eventually but take 2 or 3 minutes! (I've tried IE, Chrome & Firefox.)
Background: Our office is small (6 people) and the problem occurs even if I'm the only one in the office. Internet provider is Comcast -- whom I've contacted and cannot see a problem -- with a fairly fast connection. Speedtest.net just showed 66Mbps / 23Mbps (down/up).
DNS: I don't think it's a DNS issue since ping, tracert, nslookup tend to return the same IP whether I'm in the office, at home or elsewhere. (I can also ping etc. from the Comcast gateway's utility and it has the same results.) Also, we have a couple of web pages we load using an IP rather than name and these are slow too.
Firewall: We have a Sonicwall installed. I don't want to poke around in this -- although I could. Our remote IT department administer this. They are currently at a lost too.
Likely relevant: I noticed two scenarios where we don't see this issue:
1. I run Windows 7 and we are all on the same domain. If I run an old XP, logged on locally since it is no longer on our corporate domain -- it does not have the problem.
2. Phones or tablets on the same wifi network do not suffer the same issue.
Both of these make me think that there is some issue with our domain somehow? But now I'm getting out of my knowledge comfort zone. (Other offices on our WAN don't have the issue. E.g. I can open a 2X desktop and access all these sites with ease.)
Helps!
Problem: Most (80/90%?) web sites load fine. Some, such as amd.com and this themonthly.com.au link from the blue, will load eventually but take 2 or 3 minutes! (I've tried IE, Chrome & Firefox.)
Background: Our office is small (6 people) and the problem occurs even if I'm the only one in the office. Internet provider is Comcast -- whom I've contacted and cannot see a problem -- with a fairly fast connection. Speedtest.net just showed 66Mbps / 23Mbps (down/up).
DNS: I don't think it's a DNS issue since ping, tracert, nslookup tend to return the same IP whether I'm in the office, at home or elsewhere. (I can also ping etc. from the Comcast gateway's utility and it has the same results.) Also, we have a couple of web pages we load using an IP rather than name and these are slow too.
Firewall: We have a Sonicwall installed. I don't want to poke around in this -- although I could. Our remote IT department administer this. They are currently at a lost too.
Likely relevant: I noticed two scenarios where we don't see this issue:
1. I run Windows 7 and we are all on the same domain. If I run an old XP, logged on locally since it is no longer on our corporate domain -- it does not have the problem.
2. Phones or tablets on the same wifi network do not suffer the same issue.
Both of these make me think that there is some issue with our domain somehow? But now I'm getting out of my knowledge comfort zone. (Other offices on our WAN don't have the issue. E.g. I can open a 2X desktop and access all these sites with ease.)
Helps!
It could still be a DNS issue, if the problem sites are loading resources from another domain.
If it was happening to me I'd look at the load times of the problem sites in the web inspector. In Chrome, open Developer Tools (right-click anywhere, "Inspect Element") and click on the Network tab, then tick "Disable cache" and reload the page. You'll see the loading times for each element on the page, which might give a hint where exactly the problem is.
posted by neckro23 at 3:33 PM on May 15, 2015
If it was happening to me I'd look at the load times of the problem sites in the web inspector. In Chrome, open Developer Tools (right-click anywhere, "Inspect Element") and click on the Network tab, then tick "Disable cache" and reload the page. You'll see the loading times for each element on the page, which might give a hint where exactly the problem is.
posted by neckro23 at 3:33 PM on May 15, 2015
DNS would be my first thought. Even if running ping and the like returns the right IP/host, doesn't stop it from being slow. Try experimenting with the Google DNS servers as a simple starting point?
posted by jaffacakerhubarb at 3:35 PM on May 15, 2015
posted by jaffacakerhubarb at 3:35 PM on May 15, 2015
Response by poster: I've switched my DNS explicitly to Google's. (I've tried this before however. I'd also tried setting explicitly to Comcast's DNS IPs.)
Wow. That Chrome Network page is brilliant! I can see all the js elements. E.g. cookie-prompt.js took 52.56s to get a status of 200.
In the network page, I had around 8 rows, the remote address was either correct (23.4.152.20:80 in this case) or blank. After about 3 min the web page suddenly loaded and I had 50 request rows appear.
I'm trying again and now I have 35 request rows most of which are pending.... It'll get there I know... but why each one is at pending for so long I havge no idea.
posted by NailsTheCat at 4:13 PM on May 15, 2015
Wow. That Chrome Network page is brilliant! I can see all the js elements. E.g. cookie-prompt.js took 52.56s to get a status of 200.
In the network page, I had around 8 rows, the remote address was either correct (23.4.152.20:80 in this case) or blank. After about 3 min the web page suddenly loaded and I had 50 request rows appear.
I'm trying again and now I have 35 request rows most of which are pending.... It'll get there I know... but why each one is at pending for so long I havge no idea.
posted by NailsTheCat at 4:13 PM on May 15, 2015
When you go to a website, it likely causes dozens and sometimes hundreds of DNS lookups to other domains as part of advertising, social media buttons, analytics, off-server fonts, images, JavaScript files, etc.
It may be that your Active Directory DNS server(s) are slow or non-responsive in querying the root domain servers for new domain names, if it doesn't have DNS forwarders enabled. If the AD DNS server has forwarders enabled, it may be that the forwarders you're using are slow or out of date. In addition, my experience is that AD DNS gets cranky if it only has one server. This isn't wrong, but I've had to periodically restart DNS services in those cases.
Like jaffa said, change the DNS servers on your local PC to be Google's and try again. If that works, have your consultants restart the DNS service on your AD server. They should also check the Sonicwall to make sure it's not getting in the way of DNS queries.
posted by cnc at 4:15 PM on May 15, 2015
It may be that your Active Directory DNS server(s) are slow or non-responsive in querying the root domain servers for new domain names, if it doesn't have DNS forwarders enabled. If the AD DNS server has forwarders enabled, it may be that the forwarders you're using are slow or out of date. In addition, my experience is that AD DNS gets cranky if it only has one server. This isn't wrong, but I've had to periodically restart DNS services in those cases.
Like jaffa said, change the DNS servers on your local PC to be Google's and try again. If that works, have your consultants restart the DNS service on your AD server. They should also check the Sonicwall to make sure it's not getting in the way of DNS queries.
posted by cnc at 4:15 PM on May 15, 2015
OK - saw what you wrote. Confirm that your Windows 7 machines have the same DNS server, subnet mask, a similar IP address and the same default gateway as your XP machine and tablets/phones.
I wonder if the AD machines are configured differently or are using a separate path from the other devices on the network. If the AD machines are taking a different network path, it might explain the differences.
Also, if you're using VPN inside the office, either on your PC or between servers/devices/sites to connect to your home office, mention that as well.
posted by cnc at 4:23 PM on May 15, 2015
I wonder if the AD machines are configured differently or are using a separate path from the other devices on the network. If the AD machines are taking a different network path, it might explain the differences.
Also, if you're using VPN inside the office, either on your PC or between servers/devices/sites to connect to your home office, mention that as well.
posted by cnc at 4:23 PM on May 15, 2015
SonicWall OS 5.8.1.15 is solid but 5.8.4.x and 5.9.x.x cause all sorts of browsing slowdowns.
posted by ridogi at 5:59 PM on May 15, 2015
posted by ridogi at 5:59 PM on May 15, 2015
Response by poster: At home now. amd.com loads in a second. I get different DNS results now.
C:\nslookup www.amd.com
OFFICE
Server: google-public-dns-a.google.com
Address: 8.8.8.8
Non-authoritative answer:
Name: e2897.b.akamaiedge.net
Address: 23.4.152.20
Aliases: www.amd.com
dsa-www.amd.com.edgekey.net
dsa-www.amd.com.edgekey.net.globalredir.akadns.net
HOME
Server: router.asus.com
Address: 192.168.1.1
Non-authoritative answer:
Name: e2897.b.akamaiedge.net
Address: 184.30.11.218
Aliases: www.amd.com
dsa-www.amd.com.edgekey.net
dsa-www.amd.com.edgekey.net.globalredir.akadns.net
So DNS is perhaps the culprit as you've indicated.
posted by NailsTheCat at 6:15 PM on May 15, 2015
C:\nslookup www.amd.com
OFFICE
Server: google-public-dns-a.google.com
Address: 8.8.8.8
Non-authoritative answer:
Name: e2897.b.akamaiedge.net
Address: 23.4.152.20
Aliases: www.amd.com
dsa-www.amd.com.edgekey.net
dsa-www.amd.com.edgekey.net.globalredir.akadns.net
HOME
Server: router.asus.com
Address: 192.168.1.1
Non-authoritative answer:
Name: e2897.b.akamaiedge.net
Address: 184.30.11.218
Aliases: www.amd.com
dsa-www.amd.com.edgekey.net
dsa-www.amd.com.edgekey.net.globalredir.akadns.net
So DNS is perhaps the culprit as you've indicated.
posted by NailsTheCat at 6:15 PM on May 15, 2015
Response by poster: Re. Tell me about your Java updates...
Could it be this if it's the blazes at home and elsewhere -- just slow elsewhere?
Re. SonicWall OS 5.8.1.15 is solid but 5.8.4.x and 5.9.x.x cause all sorts of browsing slowdowns.
My last tests were patched round the Sonicwall. Also, tablets, cellphones non XP laptops are fine. I'll definitely check it anyway when back in the office.
posted by NailsTheCat at 6:18 PM on May 15, 2015
Could it be this if it's the blazes at home and elsewhere -- just slow elsewhere?
Re. SonicWall OS 5.8.1.15 is solid but 5.8.4.x and 5.9.x.x cause all sorts of browsing slowdowns.
My last tests were patched round the Sonicwall. Also, tablets, cellphones non XP laptops are fine. I'll definitely check it anyway when back in the office.
posted by NailsTheCat at 6:18 PM on May 15, 2015
Best answer: Other things to check on the SonicWall are Security Services (Content Filter, Gateway AV, App Control, etc) or firewall rules (with QoS) applied to just some devices instead of all devices on the network.
Getting different addresses from nslookup doesn't necessarily indicate a problem, although it means you connected to a different instance / server. Both of those IPs (2 comments up) resolve to Akamai which is a CDN, so getting different IPs is to be expected based on your location (as determined by your IP and DNS generally). Simply connecting at a different time would connect you to a different IP as they are obviously using a CDN which effectively is a load balancer (so the site is hosted at multiple IPs).
Traceroute would be a better tool, as this could be a problem with the ISP. If you can identify a domain on amd.com for ads, scripts, images etc that is causing the slowdown a traceroute of that may indicate the problem.
posted by ridogi at 6:57 PM on May 15, 2015
Getting different addresses from nslookup doesn't necessarily indicate a problem, although it means you connected to a different instance / server. Both of those IPs (2 comments up) resolve to Akamai which is a CDN, so getting different IPs is to be expected based on your location (as determined by your IP and DNS generally). Simply connecting at a different time would connect you to a different IP as they are obviously using a CDN which effectively is a load balancer (so the site is hosted at multiple IPs).
Traceroute would be a better tool, as this could be a problem with the ISP. If you can identify a domain on amd.com for ads, scripts, images etc that is causing the slowdown a traceroute of that may indicate the problem.
posted by ridogi at 6:57 PM on May 15, 2015
Best answer: More Windows DNS problems than seems credible are fixed by flushing its DNS caches, both on the affected workstations and on any DNS servers (often the domain controllers) inside your LAN. If you change the DNS server setting on a workstation and don't flush its DNS cache, and the cache still holds a stale entry that it's unwilling for whatever inscrutable Windows reason to let go of, the change of setting won't make any difference to a lookup for the cached address.
Easiest way to flush the DNS cache is to stop and then start the Windows DNS Client service. You can do that either from the Services control panel GUI, or from an administrative console window using
(c) When I hover my mouse over system tray wireless icon, it shows 'companyname.com Internet Access'. My gut feel is that also means 'you're in crappy web page load mode' somehow.
No, all that means is that Windows has noticed that its domain controller is accessible via that network connection and named it accordingly, and that Windows has detected the reachability of one of Microsoft's test servers over that connection.
posted by flabdablet at 4:46 AM on May 16, 2015
Easiest way to flush the DNS cache is to stop and then start the Windows DNS Client service. You can do that either from the Services control panel GUI, or from an administrative console window using
net stop dnscache
net start dnscache
(c) When I hover my mouse over system tray wireless icon, it shows 'companyname.com Internet Access'. My gut feel is that also means 'you're in crappy web page load mode' somehow.
No, all that means is that Windows has noticed that its domain controller is accessible via that network connection and named it accordingly, and that Windows has detected the reachability of one of Microsoft's test servers over that connection.
posted by flabdablet at 4:46 AM on May 16, 2015
You've found the Chrome Network profile tool. But not sure if you've found all the info that's there. The "Timeline" column has a bunch of colored bars in it that show exactly what's happening on the network in that time. If you hover over it it will show you various categories of network activity, including DNS. (Screenshot). If DNS is taking anything over a few hundred milliseconds something is probably wrong. Look to the earliest requests. If you could post a screenshot of what it looks like on a bad page it'd help us figure it out.
This advice is probably off-base, but if it's Google properties that are causing the delays (including Google Analytics and Fonts) it may be the problem is QUIC, Google's new UDP experimental protocol. But that's only implemented in Chrome so far, so wouldn't explain Firefox and IE problems.
posted by Nelson at 10:44 AM on May 16, 2015
This advice is probably off-base, but if it's Google properties that are causing the delays (including Google Analytics and Fonts) it may be the problem is QUIC, Google's new UDP experimental protocol. But that's only implemented in Chrome so far, so wouldn't explain Firefox and IE problems.
posted by Nelson at 10:44 AM on May 16, 2015
Response by poster: If you hover over it it will show you various categories of network activity, including DNS.
Fantastic. Thanks for that screenshot. Yet another level of detail. I will examine that on Monday.
posted by NailsTheCat at 7:03 AM on May 17, 2015
Fantastic. Thanks for that screenshot. Yet another level of detail. I will examine that on Monday.
posted by NailsTheCat at 7:03 AM on May 17, 2015
Response by poster: Here's the screenshot of the timeline pop-up.
So it seems to be the 'Waiting (TTFB)' segment that is the hold up: "Time spent waiting for the initial response, also known as the Time To First Byte. This time captures the latency of a round trip to the server in addition to the time spent waiting for the server to deliver the response."
I've tried stopping and starting DNS locally. I think what I'll try next is eliminating the domain controller from the network briefly.
posted by NailsTheCat at 8:13 AM on May 18, 2015
So it seems to be the 'Waiting (TTFB)' segment that is the hold up: "Time spent waiting for the initial response, also known as the Time To First Byte. This time captures the latency of a round trip to the server in addition to the time spent waiting for the server to deliver the response."
I've tried stopping and starting DNS locally. I think what I'll try next is eliminating the domain controller from the network briefly.
posted by NailsTheCat at 8:13 AM on May 18, 2015
Best answer: Thanks for the screenshot. Neither of those popups are showing a DNS request; by the time you get to loading an asset like jQuery the DNS has been cached already. You should look at the very first request in the page, in a new Incognito Window in Chrome. That will show a DNS request.
But the fact it's stalled on TTFB suggests it's probably not DNS, but something else. Sometimes that's an indicator the web server is overloaded / slow, but you say this happens with multiple sites that normally work well so that seems unlikely. FWIW that IP address for the AMD server is an Akamai content server. Odds are good it's not the problem.
TTFB could also mean a problem with the network stack making HTTP requests. Assuming HTTP 1.1 and multiple connections / pipelining, all sorts of things can go wrong. Particularly if some network device between you and the web server is meddling with HTTP. I'd blame your network. Are you absolutely sure you bypassed the Sonicwall? Is there anything else intercepting HTTP requests? Those are questions only a network administrator can really answer. If I were doing that job, I'd be asking you for packet captures next.
Does this problem only happen with https:// sites, using SSL? Or does it happen on nonsecured HTTP too?
posted by Nelson at 9:03 AM on May 18, 2015
But the fact it's stalled on TTFB suggests it's probably not DNS, but something else. Sometimes that's an indicator the web server is overloaded / slow, but you say this happens with multiple sites that normally work well so that seems unlikely. FWIW that IP address for the AMD server is an Akamai content server. Odds are good it's not the problem.
TTFB could also mean a problem with the network stack making HTTP requests. Assuming HTTP 1.1 and multiple connections / pipelining, all sorts of things can go wrong. Particularly if some network device between you and the web server is meddling with HTTP. I'd blame your network. Are you absolutely sure you bypassed the Sonicwall? Is there anything else intercepting HTTP requests? Those are questions only a network administrator can really answer. If I were doing that job, I'd be asking you for packet captures next.
Does this problem only happen with https:// sites, using SSL? Or does it happen on nonsecured HTTP too?
posted by Nelson at 9:03 AM on May 18, 2015
Response by poster: Nelson, great questions & additional information -- thank you! Our IT team and I spent some time on this morning, we think we've now resolved it:
(1) SonicWall is 5.9.x. (Re. ridogi's concern -- so yes, we're on one of the later, more problematic releases.)
(2) On the SonicWall, Interface MTU was set to fragment non-VPN packets large than 1404. We don't know why, now cleared.
(3) Also on SonicWall, we saw that various items that should be updated (antivirus, intrusion prevention etc.) are showing as last updated in 2014 -- even when we click to update and it appeared to be successful. Very strange.
(4) Are you absolutely sure you bypassed the Sonicwall? Yes but no. I patched straight into the back of the gateway. But we just realized that I'm still -- somehow -- going through it. We have two gateways, something to do with the video conferencing and there must be some shenanigans there. I don't know what exactly yet, but clearly I wasn't. Sorry for that red herring.
Conclusion: IT just managed to get the SonicWall anti-virus to properly, properly update. And it's FIXED. I can load amd.com (for whatever reason) in <1>
Thanks everyone for all your kind help. Thanks for helping me learn about Chrome's tools and more besides. Other than helping my occasional browsing you've made my colleague who uses a remote app very, very happy. Things have been so difficult for her and they're now fixed.
posted by NailsTheCat at 11:49 AM on May 18, 2015 [2 favorites]
(1) SonicWall is 5.9.x. (Re. ridogi's concern -- so yes, we're on one of the later, more problematic releases.)
(2) On the SonicWall, Interface MTU was set to fragment non-VPN packets large than 1404. We don't know why, now cleared.
(3) Also on SonicWall, we saw that various items that should be updated (antivirus, intrusion prevention etc.) are showing as last updated in 2014 -- even when we click to update and it appeared to be successful. Very strange.
(4) Are you absolutely sure you bypassed the Sonicwall? Yes but no. I patched straight into the back of the gateway. But we just realized that I'm still -- somehow -- going through it. We have two gateways, something to do with the video conferencing and there must be some shenanigans there. I don't know what exactly yet, but clearly I wasn't. Sorry for that red herring.
Conclusion: IT just managed to get the SonicWall anti-virus to properly, properly update. And it's FIXED. I can load amd.com (for whatever reason) in <1>
Thanks everyone for all your kind help. Thanks for helping me learn about Chrome's tools and more besides. Other than helping my occasional browsing you've made my colleague who uses a remote app very, very happy. Things have been so difficult for her and they're now fixed.
posted by NailsTheCat at 11:49 AM on May 18, 2015 [2 favorites]
Two rules of thumb for network problems:
posted by Nelson at 12:18 PM on May 18, 2015 [2 favorites]
- Blame DNS
- Blame the firewall
posted by Nelson at 12:18 PM on May 18, 2015 [2 favorites]
Response by poster: From IT: "had to unregister the SonicWall device and then re-register it. Its licensing was in a weird state. Once it was able to communicate mysonicwall.com, the licensing and signature dates all appeared normal again."
posted by NailsTheCat at 12:31 PM on May 18, 2015
posted by NailsTheCat at 12:31 PM on May 18, 2015
red herring
Par for the course. Don't think I've ever had to diagnose a network issue that wasn't liberally provided with them.
There are just so many ways for networks to do things other than what you assume they're doing, which is why I'm so grateful for tools like tcpdump and wireshark. And even drawing conclusions based on those requires extreme care, especially if you've been using any form of capture filtering.
Networking is hard. I've never even once seen an antivirus product make it easier.
posted by flabdablet at 10:10 PM on May 18, 2015 [1 favorite]
Par for the course. Don't think I've ever had to diagnose a network issue that wasn't liberally provided with them.
There are just so many ways for networks to do things other than what you assume they're doing, which is why I'm so grateful for tools like tcpdump and wireshark. And even drawing conclusions based on those requires extreme care, especially if you've been using any form of capture filtering.
Networking is hard. I've never even once seen an antivirus product make it easier.
posted by flabdablet at 10:10 PM on May 18, 2015 [1 favorite]
This thread is closed to new comments.
(a) I patched a spare wifi router directly into the gateway bypassing the Sonicwall, problem still exists.
(b) That themonthly URL seems fine now. www.esri.com is another example which struggles however.
(c) When I hover my mouse over system tray wireless icon, it shows 'companyname.com Internet Access'. My gut feel is that also means 'you're in crappy web page load mode' somehow.
posted by NailsTheCat at 2:39 PM on May 15, 2015