Work process vs technology

February 28, 2017 7:01 PM

I need to come up a solution to both protect my data in various locations and make it easily available without using the cloud.

Part 1 - secure file access

I work at a number of different sites (usually universities) for a number of different people doing a variety of work using mostly the MS Office and Adobe Creative suites. Recently one of my universities (where I have a permanent office) upgraded to Windows 10 and as a result, 1000s of random files on my portable backup drive (which has >65k files) had access permissions changed which made copying the files elsewhere impossible without going into security settings. (I'm now going through and fixing this shit and it is fucking annoying and has cost me about a thousand dollars in lost time and expert help).

So, I don't want to connect my hard drive to my work PC ever again. For various reasons (don't argue, they are good reasons) I can't store this stuff in the cloud. So apart from lugging my 5 kilogram laptop with me when I work on campus and connecting the hard drive to that, and then emailing myself the files I want to work on, and then emailing them back to save them (clunky! open to forgetfulness and failure!), how do I manage it so I have access to all my files no matter where I am?

Part 2 reusing elements
Seeing as now I have to spend many hours in the bowels of my directories, it's time, I think, that I finally organise reusable elements of my work. I have done a lot of design work over the last 10 years which resulted in developing symbols, icons, diagrams and so on, which is reusable (in a way that doesn't impact copyright) in other projects. This work is in many (thousands) of Illustrator and PowerPoint files with names that refer to the project or the caption, so not easily searchable. I want to create an index for myself so I don't have to constantly reinvent the wheel but how? Do I make a massive illustrator file, pop all the symbols in there? Likewise PPT? I suspect files would be unworkably large, and then I still have to go through page after page to find relevant item. How do I even begin to categorise them? Is there a known process for designers to do this, let alone retro fit?

posted by b33j to Technology (12 answers total)

Best answer: I can think of a couple of options.

1. Set up your own damn server at home and connect to that over the Internet when elsewhere. This is one such product, but there are scads more. "personal cloud" is a good phrase to search on. You may have a wifi router at home already that lets you plug in a hard drive and do this.
2. A portable storage device with a wireless interface can act as a local server: something like this. You'd carry it with you everywhere. Again, there are other options.

For whatever its worth, each of these options has pros and cons relative to cloud storage in terms of security. E-mailing files to and from yourself may be less secure than using the cloud, depending on which cloud service you're comparing it to.
posted by adamrice at 7:14 PM on February 28, 2017

Response by poster: One of the reasons for not storing in the cloud is that sometimes I don't have internet access (when working on a boat, for example) so I need the files to be physically with me.
posted by b33j at 7:24 PM on February 28, 2017

Best answer: lugging my 5 kilogram laptop with me

Forgive the obviousness, but: The easiest/simplest solution to this, by far, is to have a single machine that you use, especially when you can't use cloud storage. And you can absolutely get a machine powerful enough for Illustrator and PowerPoint for a quarter of that weight, or a third of the weight if you want to easily/speedily have several files open at once. Then dock it to a monitor and keyboard when you want a more desktop-like setup. If you get something fast enough, docked to peripherals, do you need to use more than one computer at all?

Barring that, you should be able to set things up so that certain machines can't overwrite existing files on your portable hard drive; they'd only be able to read them or create new files. I'd know how to do this (i.e. that it's definitely possible) in a Unix-like system, but not Windows off the top of my head.
posted by supercres at 8:01 PM on February 28, 2017 [1 favorite]

Best answer: Oh, and the software that creates the categorized, tagged catalog you're talking about in (2) is called digital asset management. There's one built into Adobe CS that would probably integrate nicely with Illustrator. Otherwise, you'd export all the bits as individual files and manage them in a file system.
posted by supercres at 8:07 PM on February 28, 2017

Best answer: ...sometimes I don't have internet access (when working on a boat, for example) so I need the files to be physically with me.

Then you need some kind of storage, and USB hard drives are honestly the least troublesome.

I understand that

I don't want to connect my hard drive to my work PC ever again.

because

1000s of random files on my portable backup drive (which has >65k files) had access permissions changed which made copying the files elsewhere impossible without going into security settings. (I'm now going through and fixing this shit and it is fucking annoying and has cost me about a thousand dollars in lost time and expert help).

I'm betting that what's happened is not so much that the permissions got changed on your hard drive, but that the machine account on your work PC got changed during the OS upgrade (which I'm betting was actually a from-scratch reinstallation).

Assuming that the hard drive was originally set up the way Windows usually does that by default, you would then find that any file created on it via your old Windows installation would not end up with proper permissions for the new one.

Your IT people should have known better than to let that situation arise, but many don't.

Windows can cause this kind of annoyance with almost any kind of locally attached storage. Simply avoiding USB hard disks in particular is no protection against it. Awareness of it, and getting the permissions set up appropriately for a drive that will move from PC to PC, is far more effective.

Get a hard drive (or preferably a SSD, for robustness) that you will use specifically for your portable data. You should definitely not be lugging a backup drive around with you all day, especially not on a boat; you should back up your portable data onto your backup drive, at home, along with everything else you back up, and leave the backup drive locked safely in a drawer.

Format your portable-data drive with NTFS. Before copying anything onto it, right-click the drive itself under This PC in Windows Explorer, expose the Security tab, and set permissions on the whole thing to Allow Full Control for Everyone for all files and folders.

This essentially turns off all NTFS's security on that drive. Afterwards, regardless of which PC you use to add files to it, those files will end up fully accessible to any other PC.
posted by flabdablet at 9:17 PM on February 28, 2017 [3 favorites]

Best answer: I want to create an index for myself so I don't have to constantly reinvent the wheel but how?

I do this kind of thing by creating folders full of shortcuts. Any time you've made something that might be useful elsewhere, right-click on it and choose Copy; then visit all the applicable category folders, right-click and choose Paste Shortcut.

Copying a shortcut into a category folder is pretty much equivalent to adding that category to the file as a searchable tag.
posted by flabdablet at 9:20 PM on February 28, 2017

Response by poster: Oh wow. Great answers. Very helpful. I'll give this sone thought. (I didn't want to spend any more money but I guess I really have to, to get right type of laptop, that will work with 2 monitors etc.)
posted by b33j at 10:32 PM on February 28, 2017

Do you have a plan for what happens if you lose your hard drive, or someone steals it? I understand your need for offline access, but personally I'm terrified of losing my laptop to the point that I use cloud backup services (Dropbox, but I've used and liked Backblaze in the past) to make absolutely sure that I don't lose any data should my laptop get lost or stolen.
posted by simonw at 5:25 AM on March 1, 2017

1000s of random files on my portable backup drive (which has >65k files) had access permissions changed which made copying the files elsewhere impossible without going into security settings. (I'm now going through and fixing this shit and it is fucking annoying and has cost me about a thousand dollars in lost time and expert help).

Unless you had a complex permissions scheme on those files, this should be relatively quick and easy to fix. You set permissions on the top level and push those down by checking the box that says "Replace all child permission entries with inheritable permission entries from this object." If you can't do that on your university computer, disconnect the drive and plug it into a computer you own.
posted by cnc at 9:05 AM on March 1, 2017

For the Part 2 index, consider making yourself a simple, local web page that has local, relative links to everything you need. Whether that exists on a USB drive, laptop, etc. is up to you. It's going to be some work to maintain, but a co-worker does this and it works well for him. If you go this route, you might consider Tiddlywiki .
posted by cnc at 9:11 AM on March 1, 2017

Response by poster: Unless you had a complex permissions scheme on those files, this should be relatively quick and easy to fix. You set permissions on the top level and push those down by checking the box that says "Replace all child permission entries with inheritable permission entries from this object."

You'd think so, wouldn't you? I thought so. The expert I employed thought so, and even went so far as to show me how - and I felt small and like a typical end user, until he realised it didn't work.


Do you have a plan for what happens if you lose your hard drive, or someone steals it?It is now back up on the SSD of my PC.
posted by b33j at 2:19 PM on March 1, 2017

I thought so. The expert I employed thought so, and even went so far as to show me how - and I felt small and like a typical end user, until he realised it didn't work.

To make that (almost) guaranteed to work, you need (a) to be logged in as Administrator while running Windows in Safe Mode (b) to do a recursive Take Ownership of all the files concerned and only then (c) set the security permissions the way you want them. Deviation from this plan does indeed give Windows many unexpected opportunities to foil it.

It really is always possible to wrestle Windows to the ground over this and win, but it's far, far easier to start with a blank drive formatted and set up with inheritable Full Control for Everyone permissions than to do that.

Another option is to format the portable data drive using ExFAT (or even FAT32, if you know none of your files will ever be bigger than 4GiB) instead of NTFS. The FAT filesystems don't support security restrictions in any form; closest they get is the Read Only attribute. The tradeoff is that the FAT filesystems are rather less robust in the face of unexpected power loss or unplugging than NTFS, so you'd want to be even more rigorous with backups.
posted by flabdablet at 8:25 PM on March 2, 2017