Why won't my web page reflect changes in linked data files?
November 6, 2010 1:59 PM Subscribe
My supplier is providing data at the promised frequency. I'm fetching it, and it's changing as expected, but the rendering of the data on my website does not reflect the changes. Any ideas what's going on? Details inside...
posted by Fred Mars to computers & internet (5 answers total)
My organization is working with a data supplier who updates a feed for us every 15 seconds. They gave us a standard URL-based interface that looks like this:
When I enter the URL in my browser's address bar and hit refresh a bunch of times, I can see that the information is changing at the desired frequency. Everything appears to be working properly on the supplier's end. Now we want to display this data on our website, but some part of the system I have set up is not working the way I expected it to.
There are two computers involved on our end. Let's call them Data and Web. Data, an Ubuntu Linux box, makes the requests by issuing curl commands (using the above URL) scheduled using cron. It stores the data returned (an HTML fragment) to a file in a directory that is then shared via samba. Web is our public web server, Windows Server 2008 with IIS7. I created a virtual directory pointing to the shared directory on Data. On a test page, I used a server-side include to render the HTML fragment in our layout.
To some extent this works. I am definitely getting data from the supplier and it shows up on the page. I have verified that the data on Data is changing every 15 seconds. But when it comes time to render it, that 15 second frequency goes out the window. The weird thing is that eventually it does change. I'm not sure what the actual frequency is, but it is on the order of minutes (at least five, for sure) instead of seconds. You can hit refresh all you want, but it stays the same, even though the source HTML fragment on Data has changed....until eventually for some reason the web display changes too.
I have no idea what is going on. Is something getting cached somewhere automatically? I tried adding a no-cache meta tag on the test web page. I tried disabling output caching in IIS7. Neither of those worked. Could it be something about SSI that I don't understand?