Comments on: How to go from "this_" to "this" ?

Question: How to go from "this_" to "this" ?

Robot Johnny — Thu, 23 Mar 2006 08:59:45 -0800

Batch unzipping and file renaming on UNIX or OS X? All_ my_ files_ look_ like_ this!_.gz

I recently lost all files on one of my domains, but was able to recover everything from my server's backup. However, all the files now have an underscore appended to the end of the filename (index.html is now index.html_, for example). All the files and all the folders are like this.

And some of them are gzipped, so some files are now filename.cgi_.gz.

What's the quickest, easiest way to unzip everything and remove the _ from the end of the files, including those within nested folders? Is there a shell command I can use? (or a way to do this using Automator on OS X if I copy everything from my server to my computer?)

By: Malor

Malor — Thu, 23 Mar 2006 09:01:38 -0800

In Windows, that _ extension can mean the file is compressed. Are they the undamaged files, you just need to rename them?

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 09:04:10 -0800

Yeah, they just need renaming... they are otherwise exact copies.

By: zsazsa

zsazsa — Thu, 23 Mar 2006 09:09:46 -0800

"gunzip *gz" will ungzip everything.

"rename 's/_$//' *" will remove the _ from all of the files. rename may not be on OSX, it's a fairly standard part of Linux at least.

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 09:13:38 -0800

Thanks zsazsa, but will that remove only the _ on the end of the filename? Or every occurance of _? (Some of my files have a _ elsewhere in the filename that I obviously want to keep)

Also, will these commands apply to files within nested folders? Or will I have to go into each folder and repeat the command?

By: leftoverboy

leftoverboy — Thu, 23 Mar 2006 09:16:03 -0800

If you have menu scripts turned on there is a script
Finder Scripts > Replace Text in Item Names

The script opens a little dialog box that allows you to replace '_' with ' '.

If you don't have menu scripts on you should, there are lots of useful actions available. Spotlight 'Applescript Utility' Open it and click the check boxes 'Show Script Menu in menu bar' and 'Show Library scripts'

Like your sites, glad to pay you back for the entertainment.

By: leftoverboy

leftoverboy — Thu, 23 Mar 2006 09:17:40 -0800

Doh! Just say preview. To only grab the end under score find and replace '_." with '.' and you will get only the ones at the end,

By: zsazsa

zsazsa — Thu, 23 Mar 2006 09:18:18 -0800

Oops. I didn't see the recursive argument.

"gunzip -r ." for the ungzipping...

"find -exec rename 's/_$//' {} \;" should do it.

Yes, that will only remove a _ on the end of a filename. A '$' in a regular expression means "end of a line/string", so it will only match underscores on the ends of filenames.

By: zsazsa

zsazsa — Thu, 23 Mar 2006 09:19:33 -0800

(By recursive, I mean it'll work in nested folders.)

By: zsazsa

zsazsa — Thu, 23 Mar 2006 09:31:11 -0800

(If rename doesn't come with macos, you can put this script in the dir you want and do:

"find . -exec perl rename.perl 's/_$//' {} \;"

Or if you'd rather not hassle with this Unix arcanery, leftoverboy's way looks fine.

(I added a '.' to the find command because some finds are different and need it, and I don't have access to a Mac right now to see if that's the case. It doesn't hurt to use it.)

By: Malor

Malor — Thu, 23 Mar 2006 09:35:10 -0800

If all else fails, here is a short batch script that should work. MAKE SURE you run this ONLY on a backup of the files. I have only tested this with Linux. It will recurse directories and seems fine, but I have not tested this extensively.



!/bin/sh

# first, set the word break to be newline only.

# This is easier for dealing with filenames.

IFS=$'\n'



# Do a search for all _ files, and then pipe that result

# through sed...stripping away the _



for i in `find . -name "*_"  | sed -e s/\_$//`; do

  mv $i\_ $i

done

By: nicwolff

nicwolff — Thu, 23 Mar 2006 09:36:05 -0800

There's no "rename" in OS X. Try

find / -name '*_' -print0 | perl -n0e '$old = $_; chop; chop; rename $old, $_;'

By: nicwolff

nicwolff — Thu, 23 Mar 2006 09:37:33 -0800

Definitely use -print0 by the way, file names on the Mac often contain spaces.

By: Malor

Malor — Thu, 23 Mar 2006 09:40:05 -0800

A bit more documentation... that recurses through your directories and generates a list of files ending with _. It strips the _ by piping the result through sed.

Then, for each line, it does:

mv filename_ filename

The perl renamer may be fine too... but I don't know if it will handle spaces properly. My shell script will... that's what the IFS command does.

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 10:01:54 -0800

Thanks everyone.

zsazsa, the unzipping worked, but like others stated, "rename" doesn't work

nicwolff, I tried using your method, but it started to search my entire computer so I had to terminate it... how do I limit it to my current directory (and its recursive directories?)

Malor, if I use your method, how do I run a batch script? (I'm new to fiddling around in OS X's terminal)

Also, I'm not limited to OS X's Terminal... the files also exist on my Dreamhost server, so any commands that work on their Unix servers are welcome, too.

By: nicwolff

nicwolff — Thu, 23 Mar 2006 10:06:46 -0800

Oh, sorry, you said "everything" so I figured you wanted to do the whole filesystem. Just change the "/" to a "." to do only the current directory and its subdirectories:

find . -name '*_' -print0 | perl -n0e '$old = $_; chop; chop; rename $old, $_;'

By: nicwolff

nicwolff — Thu, 23 Mar 2006 10:07:20 -0800

And that'll work fine on any UNIX that has Perl installed.

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 10:13:00 -0800

Thanks nicwolff!
Sorry to keep being a nitpicker, but that totally worked for the main directory, but did nothing to the files in its subdirectories.

By: Malor

Malor — Thu, 23 Mar 2006 10:18:28 -0800

Robot johnny: use Text Editor to create a new file, fixfiles.sh, in your home directory. (that's the easy place to find it.) Copy that script in, verbatim. Save it.

Then open a terminal. cd to the directory you stored your backups in. (make sure it's not the only copy you have!). Run the script with ~/fixfiles.sh. It searches from the current directory down for _ files.

If for some weird reason you plan to run it a lot, let me know and I'll tell you how to install it semi-permanently.

By: Malor

Malor — Thu, 23 Mar 2006 10:19:27 -0800

Crap! Run the script with:

sh ~/fixfiles.sh

By: nicwolff

nicwolff — Thu, 23 Mar 2006 10:26:26 -0800

Something else is wrong then; that should work. Does

find . -name '*_' -print

give you a list of the files in the subdirectories?

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 10:31:53 -0800

Thanks, Malor! Worked like a charm.

A slight problem though: as the script renamed subdirectories ending in _, it spat out error messages that files within those directories couldn't be found (I assume because the name had been changed in mid-script).

So I just repeated the script until there were no more error messages.

Everyone here helped in some way, so time to hand out the "best answers". Thanks so much!

On preview: Yes, nicwolff, just checked on a backup copy of the offending files and that command does list all the files. Don't know why the renaming didn't work. Thanks for your help!

By: Robot Johnny

Robot Johnny — Thu, 23 Mar 2006 10:34:26 -0800

Oh, and thanks leftoverboy, too, but your method wouldn't work because the _ came after the extension (ie hello.jpg_)

By: Malor

Malor — Thu, 23 Mar 2006 10:48:51 -0800

Robot, I ALMOST posted back with that explicit instruction... I realized that might happen. But I didn't want to make it sound overcomplex, and I was having trouble explaining it simply. I'm glad you figured it out. :)

By: aberrant

aberrant — Thu, 23 Mar 2006 11:56:47 -0800

Um, you might want to consider quoting the filename variables in those examples - if the files have spaces in them, then you're going to get into IFS problems. (I learned this the hard way). For example, the

mv $i\_ $i

in Malor's example should be

mv "$i\_" "$i"

By: Malor

Malor — Thu, 23 Mar 2006 13:34:44 -0800

aberrant, it worked perfectly in my testing with filenames with spaces.

By setting IFS to newline only, and having newlines embedded in the filename, it seems to work without needing the quotes.