How can I see a website's file tree?

September 23, 2008 7:10 PM

How can I browse a website like how it would be if there weren't a "index.html" or similar file? That is, see a directory-type listing of the files in that directory?

You know, that type of listing where there are directories, a link to move up one folder, the files in that directory, etc.

posted by demagogue to computers & internet (7 answers total)

You can't. If the web server has been configured to serve a default document, and that document is found in the folder, that's what you'll get. You can't make a request that says "show me your directory tree."
posted by justkevin at 7:19 PM on September 23, 2008

Uh, I don't think this works like you think it does. Permissions.

That said, if you view images on a site, you can usually get a folder full of content, which doesn't always have the means in place to prevent snooping.
posted by shownomercy at 7:21 PM on September 23, 2008

that would have to be set at the server side as an apache directive, specifically <Directory>.
posted by Mach5 at 7:23 PM on September 23, 2008

You can use Xenu link sleuth to map a website's links the way googlebot would.

It only gives you a site map sort of list of files and pages as they are linked. So, if there's a file in there not linked-in, you won't see it.
posted by metajc at 8:00 PM on September 23, 2008

Allowing users to browse would be a security risk, so anything professionally coded won't let you do this. Many sites deliberately dispense with the robots.txt file because it could tell attackers where things like scripts are located. (Many other sites just forget to create one, of course.)
posted by dhartung at 11:25 PM on September 23, 2008

Just to chime in: this has nothing to do with Linux file permissions and everything to do with the way web servers work. When you make an HTTP request, all you're getting is a document that corresponds to some arbitrary URL string. There's no requirement for the server to provide any way to enumerate those URLs, or even for addresses to correspond in any way to files on a disk.

In other words, if I stick a file called "u8rf4mjaw24wqf2.html" on a web server and turn off directory listings, then there's no way for anyone to get the server to disclose that information short of guessing the filename. Of course, if I link to the file from somewhere public then all bets are off, because it won't be long before Google et al. pick it up.
posted by teraflop at 11:30 PM on September 23, 2008

More specifically, the directory listing that you see is something that Apache generates on the fly when there is no index file, thus the presence of an index file means it's impossible to get a directory listing.
posted by Rhomboid at 8:29 PM on September 24, 2008