BASH: Handling odd characters in a variable?

November 18, 2008 1:58 PM

BASH Scripting. I'm trying to do a simple "mv" operation with a not-so-simple filename that contains special characters (as in brackets and hyphens). The filename itself is stored within a variable. The operation fails, and I can understand why, but I don't know how to get around it.

A sample filename would be something like:

[2008] Region {XX}.pdf

...and is stored in $sFilename. Not all of the filenames are the same format although all of them start with the year in square brackets.

I've got the rest of the script set up to parse for the year in the initial set of [ ] brackets and create a folder (./2008) with that name. The second part of the script is supposed to move the aforementioned file into the created folder. I attempt to do this with the command:

mv "$sFilename" ./$sPrefix

...where $sPrefix is the parsed item from the brackets. It errs out with the message:

mv: cannot stat `[2008]': No such file or directory.

It seems to me the shell is choking on the filename. What's the best way to work around this WITHOUT renaming the files (since the program that imports them is Windows-based and requires they be intact)?

Thanks

posted by Ziggy Zaga to computers & internet (11 answers total)

are you sure that the variable is getting the whole file name and not just that first [2008]?
you can echo it and its fine, but then if you do the mv it doesn't work?
posted by alkupe at 2:05 PM on November 18, 2008

mv "${sFilename}" ./$sPrefix
worked for me using your example filename.
posted by majick at 2:07 PM on November 18, 2008

alkupe has it. Your variable doesn't contain what you think it contains.
posted by sbutler at 2:11 PM on November 18, 2008


~/scratch/ziggy $ ls
[2008] Region {XX}.pdf
~/scratch/ziggy $ ls -1 | grep Region | while read name ; do echo $name ; prefix=`echo $name | sed -e 's/.*\[\(.*\)].*/\1/g'` ; mkdir -p $prefix ; mv "$name" $prefix ; done
[2008] Region {XX}.pdf
~/scratch/ziggy $ ls
2008
~/scratch/ziggy $ ls 2008/
[2008] Region {XX}.pdf
~/scratch/ziggy $

posted by doteatop at 2:11 PM on November 18, 2008

I'm tentatively marking your answers as best, alkupe and doteatop. I put an echo command before the move command to show me the filename before it moves it, and the value stored in $sFilename is indeed not correct to begin with.

I'll use your solution if I can't figure it out on my own, doteatop. Thanks for giving me a reference though.
posted by Ziggy Zaga at 2:25 PM on November 18, 2008

I'm going to mention mmv, just because it's one of my favorite command line utilities. It may not be specifically useful for the situation, but I think you'll like it.

mmv "\[*\]*" "#1/\[#1\]#2"

Just to warn you, it won't create directories for you. It's really smart about not overwriting/deleting files inadvertently, and figures out all the rename cases up front to prevent disaster. I find myself using it a lot, when I would have used a short shell or perl script in the past, and find it much safer and less error prone.

As you can see, you use # in the target name string to refer back to matched components in the source name.
posted by darkshade at 3:15 PM on November 18, 2008 [4 favorites]

$ ls
[2008] Region {XX}.pdf
$ for f in *Region*; do mkdir -p ${f:1:4}; mv "$f" $y; done
$ ls -R
.:
2008

./2008:
[2008] Region {XX}.pdf


The '${f:1:4}' extracts 4 characters starting after the first which would be the date. The 'mkdir -p' makes the directory and doesn't complain if it already exists. You need the quotes around the $f when moving the file because it has special characters in it.

$ for f in *Region*; do y=${f#[}; y=${y%%]*}; mkdir -p $y; mv "$f" $y; done

This does the same, but first removes the shortest match of '[' from the beginning of the string and then removes the longest match of ']*' from the end of the string. Take a read through the Advanced Bash-Scripting Guide, it's worth it just to have an idea of what you can do.
posted by zengargoyle at 3:54 PM on November 18, 2008 [1 favorite]

It's worth putting double-quotes around any parameter expansion where the parameter can hold a filename or parts of one, just on general principles. I'd personally use

for f in *Region*; do y="${f#[}"; y="${y%%]*}"; echo mkdir -p "$y"; echo mv "$f" "$y"; done

at first, and remove the echo commands only when I was convinced that the mkdir and mv commands were being generated as I expected.
posted by flabdablet at 4:13 PM on November 18, 2008

Slightly unrelated: If you still can't figure it out, I'd recommend switching from Bash to a one-off {Perl, Python, Ruby, your favorite language for fast programming} script, even if it's just for this mini-operation. You'd move away from quote interpolation to something more structured (library calls, variables) plus a stronger debugger than echo.
posted by shadytrees at 6:23 PM on November 18, 2008

darkshade: that's a really nice command. Thanks.
posted by odinsdream at 6:56 PM on November 18, 2008

I've got it now; thanks. The initial problem was because of my poorly-constructed For loop which did not grab the entire file name, only the first word. I've switched it to a While loop (per doteatop's solution) and it worked.
posted by Ziggy Zaga at 8:53 AM on November 19, 2008