Apache returns 404 for an existing file

October 31, 2007 10:29 AM

Apache web server is insisting that the page we are trying to access is missing, but we know it isn't so.

We have an installation of Oracle Application Server (actually, more than one, but different servers). This product uses the Apache web server to serve its web pages. We are accessing apache directly without going through any of the Oracle services, so none of the Oracle parts should be affecting this scenario.

One of our servers will allow access to files in a certain directory. The others will not. We cannot find any .htaccess file or any statement in httpd.conf that seems to restrict the directory. Apache returns a 404 error. The access_log indicates a 404 error, there is no corresponding entry in the error log.

To make it more confusing, only two specific directory names are at issue. It is not a permissions problem on the files; if I take a directory that can be read and simply rename it to the offending directory name, the file under it will no longer be readable and return 404. Renaming it back makes it work again. The parent directory of the offending directories can be accesses without problems.

I am unsure as to how to proceed. Is there a way to make Apache reveal where on the filesystem it is looking for the file? Some way to test whether or not Apache would allow access to a specific directory based on its config, or a way to trace so that it explains why it thinks the file is 404? Inquiring minds want to know.

posted by splice to Technology (19 answers total)

URLs are case-sensitive. Are you using an upper case character instead of a lower case one, or vice versa?
posted by Steven C. Den Beste at 10:34 AM on October 31, 2007

Response by poster: No, nothing to do with case.
posted by splice at 10:39 AM on October 31, 2007

Is there anything noteworthy about the directory names that don't work? What platform is this on?
posted by dreadpiratesully at 10:41 AM on October 31, 2007

Yes, there is a way to make Apache reveal where it's looking for the file: error_log. Check your ErrorLog directive in httpd.conf for the location of this file.
posted by rhizome at 10:45 AM on October 31, 2007

Check the other config files for directory directives. In Ubuntu, look at /etc/apache2/sites-enabled for links to the sites-available directory. These config files get parsed after the httpd.conf
posted by advicepig at 10:56 AM on October 31, 2007

First typ the error log.

Then turn up the log level.
posted by cmiller at 11:14 AM on October 31, 2007

Do you have a Rewrite directive in your Apache configuration file for that item or its parent directories, which might be pointed at the wrong location?
posted by Blazecock Pileon at 11:15 AM on October 31, 2007

We are accessing apache directly without going through any of the Oracle services, so none of the Oracle parts should be affecting this scenario.

Not sure what you mean by that since the Oracle services are in the back-end in the form of Apache mods.

What version is this? Check your mod_xxx.conf files to make sure there's not a conflict between your directory name and a mount point in the routing table for some application.
posted by vacapinta at 11:21 AM on October 31, 2007

Response by poster: cmiller, thanks for the idea of turning up the log level. I wasn't even aware I could do that.

It seems the error is being returned by an oracle component after all. At least the error log leads me to believe that. I will have to investigate that and hopefully track down the offending statement, likely in an oracle config file.
posted by splice at 11:47 AM on October 31, 2007

Response by poster: vacapinta, it was a clumsy way of saying we are bypassing the web cache and are going directly to the http server port.

The directory specified is a mount point for OC4J_BI_Forms, which is servicing the request. Of course the config on the working server is the exact same, so there must be something else telling the component to fall through and just process the files if it's not a report or jsp...
posted by splice at 11:48 AM on October 31, 2007

Response by poster: OK, some more details from the Oracle side:

Directory /mydir is set as a mount point for OC4J_BI_Forms. It is used to test reports.

I can access /mydir/testfile.html from the web server.

There is a directory for jsp files under this, /mydir/jsp.

I can access the jsp reports under that, no problem. However, if I create /mydir/jsp/testfile.html and try to access it via the web, 404 error. The file in question is world readable.

Que?
posted by splice at 11:53 AM on October 31, 2007

I would expect files in the jsp directory to be given a special handler of some sort (like your servlet engine). Check for a 'Location /jsp ' stanza or something like that in the httpd.conf.
Yeah, on preview...angle brackets just aren't cool.
posted by dreadpiratesully at 12:03 PM on October 31, 2007

Or, like you just said and I failed to read properly, something on the oracle side is set up to handle regular HTML files on the working server, and not on the broken ones. But it you can't find that, you might get away with narrowing down what gets passed off to oracle in the first place by giving the Location tag something like /*.jsp instead of the entire directory.
posted by dreadpiratesully at 12:09 PM on October 31, 2007

Response by poster: Yeah, the problem is that I can't find a location handler for the jsp subdirectory, but the parent is handled by the OC4J_BI_Forms so I imagine it is handled by that as well.

If anyone has a clue as to this behaviour from the Oracle side, I'd appreciate any help.
posted by splice at 12:21 PM on October 31, 2007

I get 404's when apache doesn't have read access to the file. Just another thought....
posted by cdmwebs at 1:14 PM on October 31, 2007

If the box is running selinux, try disabling it. The default security policy imposes some restrictions on what Apache can see, leading to symptoms such as you describe.
posted by dws at 6:30 PM on October 31, 2007

Are you sure that the file on the file system can be read by the process that's running apache?

How are you creating the file (or your test files)? With the same user that is the apache user, or another one (like root)?

Also make sure that if it's in a subdirectory that the directory has the appropriate access rights for the apache user.
posted by freshgroundpepper at 8:45 PM on October 31, 2007

Response by poster: It is not a permission issue. From my problem description in the post:

It is not a permissions problem on the files; if I take a directory that can be read and simply rename it to the offending directory name, the file under it will no longer be readable and return 404.

I appreciate any attempt to help, really, but I am no newbie to Apache, Linux or Oracle Application Server. I have checked the obvious things.

dws, this is not selinux.

I have a case open with Oracle support. Hopefully they will have a clue about what's going on.
posted by splice at 8:02 AM on November 1, 2007

Best answer: As long as we "solved" this I may as well post a follow-up.

Oracle AS 10g2 release 2 (10.1.2.2) included new security "features" in the OC4J components and will no longer read files over symbolic links. A lot of the confusion stemmed from the fact that if the server saw the file in a normal directory and you later removed that directory and replaced it with a link, the server would load the file fine until it was restarted. At that point it saw that the file was in a symbolically-linked folder and refused to load the file.

No workaround provided by Oracle and so we moved the files out of that directory structure.
posted by splice at 11:50 AM on June 4, 2008