Join 3,365 readers in helping fund MetaFilter (Hide)


Datamining of my local filesystem
January 5, 2014 2:34 AM   Subscribe

I would like to harvest file-system activity from my operating system on an ongoing basis so that in the future I can look back on the data to see what files I was working on at a given time. I'm using windows 7. Basically something like the web history in a browser, but for my local files and applications.

Ideally, I would like to have the following capabilities:
-As exhaustive as possible, ideally, I'd like to have a log of all file-system activity including creation date, modified date, last access date etc. I want this logged every time these things change.

-Some way of filtering the data so that I can differentiate between the times when I actually accessed the file vs when an automated service accessed the file ie antivirus or a cloud sync app.

-An easy interface to view the info in a searchable / filterable fashion. It would be nice if I could have some kind of calendar view where I can look back at a period and see something like the top 5 most frequently accessed MSOffice documents on a given day. This is just an example though, I'd like to have a bunch of options that control how I drill-down. Another example would be if I could see which files I created/downloaded on a given day

-The ability to reach back into existing system logs and file system data to reconstruct this info from the period before I implement whatever solution we come up with in this thread. I'm not sure how deep or far back this could go, but I figure I should at least be able to harvest all the NTFS dates as they currently are.

-It would be good if this solution could also pull my web history into the searchable database

-Minimal coding required, ideally, it would be an application that runs in the background like drive indexing apps such as "Google Desktop" or "Everything". It seems that the indexing operations of these programs are already collecting all the data I want, but there is no way to access this info in a historical way.

-Google Desktop comes pretty close to what I am looking for, but it is no longer supported and seems to only show the last time the file was modified, not the entire history of a given file. It also doesn't have a very flexible UI and no easy way to access the raw data.

Thanks!
posted by Popcorn to Computers & Internet (5 answers total)
 
You probably want to turn on NTFS auditing.
posted by flabdablet at 2:37 AM on January 5


Why? Should it be for time tracking, there software that does automated time tracking based on files (programs) being opened.
posted by oxit at 5:59 AM on January 5


Microsoft Outlook Journal can record what Microsoft Office files you have open and for how long, and you can also write notes in the entries to note what you were doing at each time.
posted by ambrosen at 1:45 PM on January 5


How would I use NTFS auditing once it is turned on?

Outlook Journal is limited only to office files? No way to get it to cover other filetypes? And same question as with auditing, how do you view the information after it has been collected?

So far these things don't seem to be what I'm looking for. Perhaps there is a way to just get a big csv file of ntfs fields like filename, path, date accessed/created/modified, etc?

Thanks
posted by Popcorn at 5:28 PM on January 5


Once you have auditing turned on, the file access events you've told it to audit will appear in the event log. The event log is a binary blob of limited size and the GUI query tools are complete crap, so your best bet would be to grab copies of the events you care about using a periodically executed script. Something based on this Dump Windows Event Log To CSV Text Files script would probably do.
posted by flabdablet at 1:29 AM on January 6


« Older For cosplay purposes, I'd like...   |  We're driving in snowy weather... Newer »

You are not logged in, either login or create an account to post comments