C++ filter: How do I get a filename from a FILE * pointer?

April 12, 2012 8:39 AM

Is there a way to find out the filename associated with a specific FILE * pointer? (OS: Red Hat EL 5)

I'm running into a strange bug where we verify that a file exists and then about two or three lines later, when we try to rename the file, we get an error that says there is no such file.

The code is something like the following...

...

FILE *fp;
char temp_filename[MAX_NAME_LENGTH];
char perm_filename[MAX_NAME_LENGTH];

...

fp = fopen(temp_filename, "w");

...
// done writing to temp file, move it to permanent file
if(fp)
{
fclose(fp);
if (rename(temp_filename, perm_filename))
{
cout << "Error: can't rename file: " << strerror(errno) << endl;
}
}

The strerror command shows "No such file or directory," which is a head-scratcher because fp is open and we are in the if block of code to begin with.

I'm trying to find out if the file has been moved or deleted out from under me, i.e. if temp_filename is still the filename associated with fp. For debugging, I've tried to do a stat on temp_filename and an fstat on fp, and then cout the inode numbers (st_ino) to see if they are the same, but I can't get it to compile since the first argument to fstat(int, struct stat *) needs to be an int, not a FILE pointer.

The target operating system, if it matters, is RHEL 5, x86_64.

Help, please.

posted by tckma to computers & internet (28 answers total)

Devil's advocate: have you checked that perm_filename is a valid path and filename?
posted by bfranklin at 8:42 AM on April 12, 2012

It should be, as it's in the same directory and with the same permissions.
posted by tckma at 8:52 AM on April 12, 2012

the only C++ is the cout statement /nitpick

You can get the fd from the FILE* for your fstat.
http://linux.die.net/man/3/fileno

What's the real error value (errno) ? And what does the man page say about the meaning of the error ?
http://linux.die.net/man/2/rename

Any goofy things in the path ? (spurious dots, slashes, etc) Are they relative paths or full paths ?
posted by k5.user at 9:01 AM on April 12, 2012 [1 favorite]

Does the permanent file already exist before you do the rename? Perhaps it is refusing to overwrite the existing one.
posted by procrastination at 9:12 AM on April 12, 2012

Thanks for the fileno command!

errno == 2

buf_fstat->st_ino == 32785
buf_stat->st_ino == 0

Where do I get the mappings between errno and the enum values shown on the rename man page? This is why I used the strerror function in the error message.
posted by tckma at 9:14 AM on April 12, 2012

buf_fstat is from fstat(fileno(temp_filename), buf_fstat);
buf_stat is from stat(temp_filename, buf_stat);

When all goes well, these inode numbers are the same.
When I get the error, buf_stat->st_ino == 0.
posted by tckma at 9:18 AM on April 12, 2012

typo above.

buf_fstat is from fstat(fileno(fp), buf_fstat);
posted by tckma at 9:26 AM on April 12, 2012

dumb question:

Why are you writing to a temp file (that you create, or wipe and create if it already exists) and then at the end try to rename the file ?

(ie is there a ton of error checks that bail and whack the temp file ? Do you really expect to fail to write to the file ? Or do you bail if anything you need to write out isn't there, etc)

If that's the case, why not write to a stream/buffer type class (ie something in memory), and at the end then write the buffer out to the final file name ?
posted by k5.user at 9:32 AM on April 12, 2012

We want a complete data file at the end in the permanent file name. Another executable further down the stream picks up the permanent file name, and it would be bad if it that process were to work on an incomplete file.
posted by tckma at 9:35 AM on April 12, 2012

When the operation fails, print both filenames (with delimiters so we can see leading/trailing spaces) in the error message, then provide that to us, please.
posted by introp at 9:46 AM on April 12, 2012 [1 favorite]

I'm not sure I can do that. The filenames may be company proprietary info.
posted by tckma at 9:51 AM on April 12, 2012

Right, and none of that prevents you from doing everything in memory via a stream/buffer, and only doing the write out for the consuming process (to the perm_filename) when you're done, no ?

Now, that's not answering your question which sounds like your tempfile is not there on disk (for some reason), even after closing the file pointer.

Does adding a flush to the FILE* help any ?

Are you writing to some area that gets whacked repeatedly ? Are you using mktemp() viz some roll-your-own temp file name ?

http://linux.die.net/man/3/mktemp

(introp: based on the fstat() answer, it *appears* the problem is all on the temp file side, the file pointer is mostly legit, but the underlying file/path isn't. I'm taking OP on faith that the paths are legit and permissions are OK for now. )
posted by k5.user at 9:57 AM on April 12, 2012 [1 favorite]

You don't have test case files on which you can run this?

If you're absolutely confident the filenames are valid, my only remaining advice is to add a huge sleep between the fclose and the rename, then while it is spinning issue a "sync" from the command line in a shell. When you fclose, the filename is not guaranteed to exist at that (or, technically any) point: you may, in fact, be renaming a file that doesn't yet exist on disk (you need to fsync on the containing directory's handle in order to have that guarantee).
posted by introp at 9:58 AM on April 12, 2012 [2 favorites]

I'm pretty sure introp is on the right path, though you should be able to make the fsync call in your program instead of the sleep() spin.
posted by mysterpigg at 10:11 AM on April 12, 2012

Yes, but my sleep vote was just to do a trivial test. Since you have to fsync the directory handle, it's a non-trivial amount of code to add. Simple test first; if it works, fix it in code.
posted by introp at 10:12 AM on April 12, 2012

good point. Well, if the sleep test works, it shouldn't be too difficult to implement a general purpose solution to get the directory name, though it's possible that the directory name is known right off.
posted by mysterpigg at 10:31 AM on April 12, 2012

At the successful (!) call of fclose(), your filename is written by the OS (it might not be in iron oxide yet, but you can be sure the OS knows about it).

I'd play with link()ing the named file to other names, in this directory, in the parent directory, and over in /tmp, and seeing if those exist at various places.


Also, break all that code out into its own program and make sure it works or doesn't, outside the rest of your program.
posted by cmiller at 10:52 AM on April 12, 2012

The temp filename is a dot (hidden) file that is the same as the permanent filename except for the leading dot that hides the file. The filenames are generated (not using mktemp) having information in the filename that is used by the downstream executable such as the CPU identifier and seconds-since-1970 of file creation, among other useful info. I suppose we could use mktemp to generate a unique ID, but that would be superfluous and would likely require approval above my pay grade (and much more code changes in downstream processes).

Adding an fflush immediately before the fclose seems to have stopped the problem from occurring, or at least being reliably reproducible. I'll try to recreate the problem with the new code changes.
posted by tckma at 10:57 AM on April 12, 2012

1. man fileno stat link unlink mkstemp and fdopen
2. perr and strerror are your friend
3. You do not need fsync. Avoid voodoo programming.
4. Don't use mktemp, use mkstemp
posted by rr at 10:58 AM on April 12, 2012 [1 favorite]

The directory name is indeed known right off. It is user configurable by means of a configuration file. We make sure the configured directory exists and is writable at the very start of the program, if not the program gracefully exits with an appropriate error message written to the log file.
posted by tckma at 11:00 AM on April 12, 2012

Btw, opening a file to check xxx (existence and especially permissions) and then closing it and doing something with some file that has the same name is classically wrong.
posted by rr at 11:01 AM on April 12, 2012 [1 favorite]

fclose calls fflush. (If it didn't imply fflush, LOTS of things would break.) If adding it fixes things, it's just a side-effect. (Increased execution time, etc.)
posted by introp at 11:35 AM on April 12, 2012

Is this multi-threaded or multi-process ? (ie are multiple instances of this code running ?) Because unless there's more "uniqueness" in the file name, you can easily get a name collision is you are using the time_t as a "unique" value etc.. Now if this is single threaded, only one process, than you can't be yanking the rug out from yourself, but if not, that may be the another place to look at.
posted by k5.user at 11:38 AM on April 12, 2012

Is it possible that your downstream executable is finding it even though it starts with a dot, noticing that it's empty, and deleting it before you get a chance to close it?
posted by aubilenon at 12:39 PM on April 12, 2012

No, we have code specifically telling it to leave dot files alone.
posted by tckma at 1:11 PM on April 12, 2012

Could there be a bug in that code? Everything you've said so far points to the file being deleted:

• The rename(2) manpage describes the errno 2 (ENOENT) condition beginning with The link named by oldpath does not exist
• fstat() with the open fd succeeds, but stat() with the file name fails.

To answer your original question, yes, it's possible to get the filename by fd (and therefore by extension of a FILE * via fileno()) on Linux: the file /proc/self/fd/nnn will be a link that you can read. If the file was actually deleted, this will contain the previous filename with "(deleted)" after. Here's a testcase:

#define _GNU_SOURCE
#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <assert.h>

int main()
{
    char temp_file_name[] = "/tmp/testcase.XXXXXX",
         procfn[64], rlbuf[256];
    int fd;
    ssize_t len;

    assert((fd = mkstemp(temp_file_name)) >= 0);
    assert(write(fd, "foo\n", 4) == 4);
    assert(unlink(temp_file_name) != -1);
    sprintf(procfn, "/proc/self/fd/%d", fd);
    assert((len = readlink(procfn, rlbuf, sizeof(rlbuf))) != -1);
    printf("fd %d has filename '%.*s'\n", fd, (int)len, rlbuf);
    assert(close(fd) != -1);
    return 0;
}

...which prints something like this:

$ ./a.out
fd 3 has filename '/tmp/testcase.f6XGTG (deleted)'

posted by Rhomboid at 4:02 PM on April 12, 2012

Also inotify could help you watch what's going on in your directory. Find some tools that use it. Here in debian-ish land, "inotify-tools" and "iwatch" packages do a lot.
posted by cmiller at 9:15 AM on April 13, 2012

If I adapt Rhomboid's code, I get output like:

fd 4 has filename ''
Error: Cannot rename file /some/dir/.blah to /some/dir/blah -- exiting...
posted by tckma at 10:52 AM on April 13, 2012