Sorting Image Thumbs from a pile
December 1, 2011 7:54 PM Subscribe
Lazy man's image sort: What is the easiest way to grab this batch of image thumbs from my mysql web database to use on my Windows machine?
I have a bunch of images in a database, and I want to grab the already extant thumbnails of certain groups of them. They are currently stored in a YYYYMM directory tree--the file names are completely random, with no associating metadata. My database contains pointers to these files along with a bunch of other information about them, such as category and author.
For another project, I need all of the thumbnails of a certain category and author in one place, on my local workstation. The query is easy to write, obviously, but I'm kind of stymied as to how best grab the result. I do not have a copy of mysql running locally.
My first thought was to simply place them all on a single web page on the server and save it from Firefox; this would seemingly save all of the images into one place on my local box (I don't care about the file names at that point). But: One of the queries returns 27,000 results. Even at an average image size of only 1.5K that's a big web page, probably too big to not crash and burn, eh? I liked this idea because it's about as lazy a way to do this as I can imagine, but it seems destined to blow something up. Or will it work?
Another thought was to simply copy the files returned by the query from their native directory to a "results" directory and grab that batch via FTP. Again, I think 27,000 file copies will probably take too long for PHP to process on my shared host, and a directory with 27K items is a bit unwieldy to download, isn't it?
I have the YYYYMM directories and images backed up locally, so another option, if I knew how, would be to save the query results as a text file and somehow, with an unknown program or script, use that to copy the relevant local files to a new local "results" directory (I run Windows 7). I could easily format the output text file to say things like "cp z:\\local\file\tree\201101\aBdDgl.jpg z:\\local\file\tree\results\cat1author1\", with each file on a new line...but I lack the knowledge of how to take that text document and do anything useful with it. Is this the best way, and if so, what does the text document contain, and how do I run it?
I should note the files actually live on my Linux server, which I have a friend admin for me, since my expertise is not systems and I don't want it to be. But running UNIX commands or feeding that text file through some sort of program on that side is possible, though I'd be even more in the dark there.
Any other ideas welcome!
I have a bunch of images in a database, and I want to grab the already extant thumbnails of certain groups of them. They are currently stored in a YYYYMM directory tree--the file names are completely random, with no associating metadata. My database contains pointers to these files along with a bunch of other information about them, such as category and author.
For another project, I need all of the thumbnails of a certain category and author in one place, on my local workstation. The query is easy to write, obviously, but I'm kind of stymied as to how best grab the result. I do not have a copy of mysql running locally.
My first thought was to simply place them all on a single web page on the server and save it from Firefox; this would seemingly save all of the images into one place on my local box (I don't care about the file names at that point). But: One of the queries returns 27,000 results. Even at an average image size of only 1.5K that's a big web page, probably too big to not crash and burn, eh? I liked this idea because it's about as lazy a way to do this as I can imagine, but it seems destined to blow something up. Or will it work?
Another thought was to simply copy the files returned by the query from their native directory to a "results" directory and grab that batch via FTP. Again, I think 27,000 file copies will probably take too long for PHP to process on my shared host, and a directory with 27K items is a bit unwieldy to download, isn't it?
I have the YYYYMM directories and images backed up locally, so another option, if I knew how, would be to save the query results as a text file and somehow, with an unknown program or script, use that to copy the relevant local files to a new local "results" directory (I run Windows 7). I could easily format the output text file to say things like "cp z:\\local\file\tree\201101\aBdDgl.jpg z:\\local\file\tree\results\cat1author1\", with each file on a new line...but I lack the knowledge of how to take that text document and do anything useful with it. Is this the best way, and if so, what does the text document contain, and how do I run it?
I should note the files actually live on my Linux server, which I have a friend admin for me, since my expertise is not systems and I don't want it to be. But running UNIX commands or feeding that text file through some sort of program on that side is possible, though I'd be even more in the dark there.
Any other ideas welcome!
Response by poster: Yes, that will work, I think. File names aren't important, I just need a single directory for each query I need to run. The query itself isn't a problem, I understand that part, or well enough to get by.
I gather there isn't any problem (other than the time it takes to execute them) that stacking up 27,000 copy statements in a .bat file will bring?
posted by maxwelton at 8:52 PM on December 1, 2011
I gather there isn't any problem (other than the time it takes to execute them) that stacking up 27,000 copy statements in a .bat file will bring?
posted by maxwelton at 8:52 PM on December 1, 2011
« Older Lock an application open, focused, for a period of... | Looking for a short PCI video card Newer »
This thread is closed to new comments.
z:\\local\file\tree\201101\aBdDgl.jpgand
z:\\local\file\tree\results\cat1author1\As your desired directory structure. So you want to keep the random filenames or do you want to rename them as well?
And does this indicate that in the database what you have is a column named, say, "image_path" and it has "/201101/aBdDgl.jpg" in it, is that right?
Basically what I would be trying to do is use the local mirror of the files, and creating a
.batfile that has a long sequence ofcopystatements in it.It's really your call as to how you want these files organized and create a target path structure that you want, whether that's with new names, with sensible english names, maybe names based on database id's or dates or whatever. If you provide more info about what's in your database I think folks can help you craft a query that can do this.
posted by artlung at 8:40 PM on December 1, 2011