Does anyone know how to deal with frozen barber-pole Spotlight indexing in Mac OS X Leopard?

October 31, 2007 11:57 AM

Will the Spotlight indexing on my Leopard update never end? It's been on blue barber pole for 12 hours now and it's dragging everything with it. What can I do?

Thank you for taking an interest! Last night, I slid a brand new shop-bought Mac OS X Leopard into my old-model 24-inch iMac, updating from Tiger. Everything went well but after 3 or 4 hours Spotlight was still indexing away, about a quarter of the way down the blue pole, indicating the usual estimate of 543000 hours to go. Today it's been going for another 9 hours and worse: there is no longer any progress line, just a stripey barber pole thingy.

Possibly useful information for any kind heart: the mdworker on the Activity Monitor swings between 30 and 70; my Active RAM is around 1.16 GB; I only have about 300 GB on my 500 GB hard drive; I've disconnected my external hard drives (didn't make any difference) and tried rebooting and disconnecting for a few minutes. Otherwise, everything seems to be working - but very, very slowly.

Any ideas? I did get through to Apple support and was sent Capture Data software (which has been dragging for 2 hours now), which I'm to mail back to them. The guy I spoke too was very nice but I was his first Leopard complaint so he hadn't heard of indexing problems. Needless to say, any help will be much appreciated!

posted by MiguelCardoso to computers & internet (22 comments total)

I would first reboot your machine from your leopard install dvd, open up disk utility (it is under the utilities list at the top of your screen) and select your boot drive and choose "repair permissions" from the right side of the screen.

I would also verify the disk, to ensure there isn't any data structure or drive issues.

If everything passes /cleans up ok. I would reboot back to your leopard install, and open up terminal (in Applications/Utilities)

and run

sudo mdutil -E /

which will force spotlight to start the index over again.

I would think that your system has some bad permissions or corrupt files on it that are causing spotlight to hang. Or it's own cache got corrupted, so it is having trouble starting over / updating its index.
posted by mrzarquon at 12:34 PM on October 31, 2007

Did you check your Console app to see if any errors are showing up there?
posted by mkultra at 12:59 PM on October 31, 2007

I thought I was having amazing performance problems during the first couple hours of having Leopard until I noticed the Spotlight indexing going on. To be honest, since I have only a small laptop hard drive, I set the computer aside and let it index by itself for a couple hours. I'm assuming all other applications are closed, and that you're not using the computer during the indexing? That is what sped mine up.
posted by mikeh at 1:19 PM on October 31, 2007

Thank you for your suggestions.

mrzarquon: I repaired and verified: everything was OK. In the terminal, however, after "sudo mdutil -E /", I got prompted for a password but couldn't type it in.

mkultra: I went and checked the Console and it was indeed full of errors.

mikeh: I know, it's quite frightening. I got it going from Jaguar to Tiger. But this time I was forewarned and just went on to bed. It's definitely not your usual indexing - no it feels like a hang.

Meanwhile, I've been inspecting the mdworker thing on the AM and it seems to be getting through the files - just very, very slowly.

I do have tens of thousands of pdfs, djvus, chms and other book and article files - but they all indexed fine on Tiger.

I'm ready to trash anything just to get the Spotlight working and the bloody computer running at an acceptable speed.

I'm sorry it's such a boring problem!
posted by MiguelCardoso at 2:00 PM on October 31, 2007

When you run top do you see mdimport running and using up lots of CPU time or memory? It's strange. I had no real issue with spotlight upgrading from Tiger. What sort of errors are being spit out in the console?
posted by chunking express at 2:03 PM on October 31, 2007

I got prompted for a password but couldn't type it in.

That statement confused me a bit... do you just not know it? Or was it refusing your regular password? 'sudo' should just take your login password.
posted by Malor at 2:26 PM on October 31, 2007

When you run top do you see mdimport running and using up lots of CPU time or memory? It's strange.

Yup.

What sort of errors are being spit out in the console?

Honestly, chunking express? It all seems pretty dire. I hope the quoted bits are relevant. They're certainly snippy:

GetZoneDataQuestionCallback - recursed to root label of 31/10/2007 20:41:49 mDNSResponder[23] ERROR: GetZoneData_QuestionCallback - recursed to root label of _wkgrpsvr._tcp.bigmec.members.mac.com. without finding SOA 31/10/2007 20:41:49 mDNSResponder[23] ERROR: startLLQHandshakeCallback _wkgrpsvr._tcp.bigmec.members.mac.com. (PTR) invoked with error code -65538
31/10/2007 14:16:52 mdworker[665] ATSFontFindFromContainer failed: count = 2. (lots of times)31/10/2007 14:29:58 Front Row[2307] _loadTexture failed at CGBitmapContextCreate image size: nan nan 31/10/2007 15:09:29 FrontRow[2307] Alias unresolvable at
31/10/2007 15:27:20 kernel GFX0: family specific matching fails 31/10/2007 15:36:04 mds[22] (/Volumes/Filmes, 31/10/2007 15:36:04 kernel disk1s3: media is not present. MuÃÅsica & Livros/.Spotlight-V100/Store-V1/Stores/F62AD8B6-57B9-4232-86A8-E42D31640832)(Error) IndexCI in store_stream_flush:store_stream_flush write err:6
31/10/2007 15:41:04 FrontRow[247] Image loader has failed to load 10 images and is unable to enable the queue, abandoning
31/10/2007 17:42:49 /System/Library/PrivateFrameworks/Apple80211.framework/Resources/airport[642] Error: Scan failed (82)

31/10/2007 17:52:47 Software Update[713] Package

Authoring Error: installation-check results requires a message

31/10/2007 17:52:47 Software Update[713] Package Authoring Error: installation-check results requires a message
31/10/2007 19:41:27 com.apple.RemoteUI[824] No accelerated IMDCT transform found
31/10/2007 20:27:27 mDNSResponder[23] ERROR: GetZoneData_QuestionCallback - recursed to root label of _wkgrpsvr._tcp.bigmec.members.mac.com. without finding SOA
31/10/2007 20:27:27 mDNSResponder[23] ERROR: GetZoneData_QuestionCallback - recursed to root label of _wkgrpsvr._tcp.bigmec.members.mac.com. without finding SOA
31/10/2007 20:28:25 mdworker[81] (Error) SyncInfo: Boot-cache avoidance timed out!

Once again, I'm sorry about all this!
posted by MiguelCardoso at 2:35 PM on October 31, 2007

"I got prompted for a password but couldn't type it in."

That statement confused me a bit... do you just not know it? Or was it refusing your regular password? 'sudo' should just take your login password.

Hi, Maior! No, I do know my login password and I've used it on Leopard with no problem. What I mean is that Terminal, after a warning about using sudo, wouldn't let me type anything after its password prompt.

I just tried again and it looks better:

ast login: Wed Oct 31 20:47:27 on ttys000
Macintosh:~ miguelcardoso$ sudo mdutil -E/

WARNING: Improper use of the sudo command could lead to data loss
or the deletion of important system files. Please double-check your
typing when using sudo. Type "man sudo" for more information.

To proceed, enter your password, or type Ctrl-C to abort.

Password:
mdutil: invalid option -- /
Usage: mdutil -pEsa -i (on|off) volume ...
Utility to manage Spotlight indexes.
-p Publish metadata.
-i (on|off) Turn indexing on or off.
-E Erase and rebuild index.
-s Print indexing status.
-a Apply command to all volumes.
-v Display verbose information.
Macintosh:~ miguelcardoso$ -E
-bash: -E: command not found
Macintosh:~ miguelcardoso$ -i
-bash: -i: command not found

Am I any closer? *crosses fingers*
posted by MiguelCardoso at 2:42 PM on October 31, 2007

Wait, it's got better, I think:

Macintosh:~ miguelcardoso$ sudo mdutil -E /
/:
Indexing enabled.
Macintosh:~ miguelcardoso$

mdworker reacts bitterly:
31/10/2007 21:44:30 com.apple.launchd[1] (0x10b120.mdworker[597]) Exited: Killed

But mdworker is finally quiet, at zero. Spotlight is still stripey.

Now how do I restart indexing so it will actually work?
posted by MiguelCardoso at 2:49 PM on October 31, 2007

When sudo asks for your password, it won't display any stars or dots, so you're typing blind. It looks like it took the password just fine.
posted by zsazsa at 2:50 PM on October 31, 2007

looks like the index app (mdworker) finally died, but it status hasn't updated the spotlight menu.

I would restart your machine and see if the menu status has updated.

If not, you can manually delete the spotlight index, using the following command in terminal:

sudo rm -r "/.Spotlight-V100"

(you should be able to copy / paste that, and be very careful to make sure you typed it right).

Then do the
sudo mdutil -E /

again, it will not prompt you for a password this time (sudo gives you a 5 minute window from the first time you type your password).

It should then say:
/:
Volume index removed. Index will be rebuilt automatically.

and the whole process will start over.
posted by mrzarquon at 3:13 PM on October 31, 2007

Wow, mrzarquon! I do think you've cracked it!

I did as you said (thanks for your considerate instructions and spacing, btw) and got:

Macintosh:~ miguelcardoso$ sudo rm -r "/.Spotlight-V100"
Password:
Macintosh:~ miguelcardoso$ sudo mdutil -E /
/:
Indexing enabled.

And indeed normal indexing has begun and is proceeding swiftly, with an encouraging indication of "2 hours remaining"!

To be frank, I'd never asked a technical question online before, because I never thought anyone would bother and, if someone would, they'd need someone with a minimum ability in order to explain the problem.

Well, you and the all the other helpful fellow users (thanks for that essential password info, zsa zsa!) have proven me wonderfully wrong. Cheers!

This only applies, of course, if it doesn't hang again. ,)
posted by MiguelCardoso at 3:54 PM on October 31, 2007

Nice. I wonder what caused the mess up in the first place.
posted by chunking express at 4:04 PM on October 31, 2007

Damn - it hung again but I've repeated the prescription and it's back on track. I'll leave it running on its own for a while and report back tomorrow! Thanks again, you guys!
posted by MiguelCardoso at 4:17 PM on October 31, 2007

Miguel- Thanks, I try my hardest to not be the IT nerd everyone is afraid to ask questions of.

It sounds like it might be an issue with file names or structure on your disk that have invalid paths etc. which can cause all sorts of weirdness when spotlight is trying to figure out their content.

My next step would be to turn off spotlight on the drive, so it stops indexing (Open spotlight in system preferences, and click on the privacy tab, then drag "Macintosh HD" into it). It will ignore and not try to spotlight your hard drive. Spotlight wont work for searching file content, but you should still be able to find files by their name. Now you atleast has a system that isn't continually trying to index itself. (but no indexing).

I would then run DiskWarrior (http://www.alsoft.com/DiskWarrior/support.html) which will scan and verify that your directory information is intact, and replace it (not deleting files, just making sure the catalogue is redone properly) with a proper version. It is not a magic bullet, but anyone who has a mac should have it, as it is a great troubleshooting utility for these situations. (you can boot off the CD they send with it to run the utility, as the software you install the computer isn't updated for Leopard, but it can fix the hard drive fine if there are problems).

You can also try apple's online discussion forum, and see if there is something I missed (i haven't seen this problem before). http://discussions.apple.com/index.jspa
posted by mrzarquon at 6:38 PM on October 31, 2007

Now you at least has a system that isn't continually trying to index itself

Looks like a desirable feature from here, mrzarquin! Many thanks for the nifty Plan B. I've just come across EasyFind which does a great little job (in case anyone's in a similar situation) and will tide me over until Spotlight is able to function properly.

I've been putting off buying DiskWarrior but the time has clearly come, what the hell.

I did look through the apple forum and there´s a guy there, name of TC!, about 3/4 down the page, who has the very same problem but not as clean and sure as yours.

I hope some day I may be a fraction of the help you were to me! All the best and a glass of port raised to your health, mrzarquon, indexed and registered to perfection!
posted by MiguelCardoso at 8:16 PM on October 31, 2007

there´s a guy there (...) who has the very same problem but not as clean and sure as yours.

Hee hee - I meant his solution isn't as clear and sure as yours, not that you both have problems but his are messier and more dubious!
posted by MiguelCardoso at 8:20 PM on October 31, 2007

Miguel-

From the linked forum, this step appears to help locate the bad file the index is getting stuck on, if you feel like giving it another shot:

Go to activity monitor, look for mdimport, get info, and then go to the "files in use" tab. At the bottom, after all the frameworks, you'll see the file spotlight is currently indexing. Trash it or move it to a drive not on your mac, or rename it to a type that isn't indexed by spotlight, then kill mdimport.

(in leopard it is mdworker instead)

If it is getting stuck on multiple files, in random locations, it may speak of a larger issue, possible hard drive failure or directory problems (diskwarrior will fix the latter).
posted by mrzarquon at 8:25 PM on October 31, 2007

Hi mrzarquon! Good morning! Just thought I'd dash off a brief note to thank you for the excellent tip: gazing at the file-munching in mdworker has raised this nightmare to the level of an average soap opera. It's slow but things do end up by happening, if you're patient and undemanding enough.

Meanwhile, *cue theme from "Gone With The Wind"* I left Spotlight running for another 12 hours and it seemed to be slowly making its way through. But today it's back to barber-pole - the little convoluted Sevillian bastard - but, after reading your kind answer, it seems not to be completely hung, as it is indeed indexing stuff, in its own fashion.

Here are my classic ignoramus's thoughts, strictly for laughs: I wonder whether Yep is making things stickier. It's a wonderful tagging app which I recently bought (i.e. can easily reinstall and relicense) to deal with all my pdf files and with which I have been delighted. I understand it draws on Spotlight to do its work and perhaps Leopard, with its Quicklook and better indexing, is having a jealous tiff with it?

Anyway, to show how the slow-moving boa constrictor is winding its way up the diskus tree swallowing one fat squirrel at a time, here are two fascinating snapshots, still warm from the mdworker :

At 13h56m Lisbon time (I'm cutting length and line-breaking to avoid breaking the page)

PDF.mdimporter/Contents/MacOS/PDF
/private/var/folders/zz/zzzivhr
/System/Library/ColorSync/Profiles
/sRGB Profile.icc
/Audio.mdimporter/Contents/MacOS/Audio
Application.mdimporter/Contents
/MacOS/Application
/Caches/com.apple.IntlDataCache.le.tecx
/Archives.mdimporter/Contents/MacOS/Archives
/System/Library/Spotlight
/Mail.mdimporter/Contents/MacOS/Mail
/System/Library/Spotlight/MacOS/Automator
/System/Library/Frameworks/
/Library/Fonts/Arial/..namedfork/rsrc
/System/Library/Spotlight/
Image.mdimporter/Contents/MacOS/Image

/System/Library/Fonts/Symbol.dfont
/Library/Fonts/Arial Black/..namedfork/rsrc
//RichText.mdimporter/Contents
/MacOS/RichText
/.mdimporter/Contents/MacOS/Bookmarks
/Library/Fonts/Georgia/..namedfork/rsrc
/Library/Fonts/Impact.ttf
/Library/Fonts/Times New Roman
/..namedfork/rsrc
/Library/Fonts/Verdana/..namedfork/rsrc
/System/Library/Frameworks
/ApplicationServices
/System/Library/Frameworks/
LatentSemanticMapping.
/Library/Fonts/Arial Narrow
/..namedfork/rsrc
/usr/share/icu/icudt36l.dat
/Library/Fonts/Optima.dfont
/Library/Fonts/Arial.ttf
/Library/Spotlight/Microsoft Office
Frameworks/Message.framework
/Versions/B/Message
/Library/Fonts/Palatino/..namedfork/rsrc
/System/Library/Fonts/ZapfDingbats.dfont
/RawCamera.bundle/Contents/
MacOS/RawCamera
/Library/Fonts/Andale Mono.ttf
/Library/Fonts/Courier New
/..namedfork/rsrc
/System/Library/Fonts/Times.dfont
/Library/Fonts/Tahoma Bold.ttf
/System/Library/Fonts/Courier.dfont
/Library/QuickTime
/DivX Decoder.component/
/Library/Fonts/Tahoma.ttf
/Library/Fonts/Trebuchet
MS/..namedfork/rsrc
/System/Library/Fonts/Helvetica.dfont
/Library/Fonts/GillSans.dfont
/Library/Caches/com.apple
.LaunchServices-023501.csstore
//Library/Fonts/Comic Sans MS/
..namedfork/rsrc
/System/Library/Fonts/AppleGothic.ttf
/Users/miguelcardoso/Livros/A Ler
/ABC of Palliative Care.pdf
/dev/null
/dev/null
/dev/null
count=0, state=0x2
/Library/Spotlight
/System/Library/Spotlight
/Users/miguelcardoso/Library/Preferences
/.DS_Store
count=0, state=0x2
/ABC of Palliative Care.pdf
/Users/miguelcardoso/Library/
Application Support/AddressBoo
k/AddressBook-v22.abcddb
/A/Frameworks/HIToolbox.framework
/Versions/A/Resources/English
.lproj/Localized.rsrc
/System/Library/Frameworks
/ApplicationServices.framework



At 14h14m, this much (at least) had been added:



/System/Library/Frameworks/CoreServices.
/System/Library/Spotlight/PDF
.mdimporter/Contents/MacOS/PDF
/private/var/folders/zz/zzzivhrRnAmv
iuee++++ME+++42/-Caches
/CoreGraphics.framework/Versions/
A/Resources/libFontStreams.A.dylib
/System/Library/ColorSync/Profiles
/sRGB Profile.icc
/System/Library/Spotlight
/Audio.mdimporter/Contents/MacOS/Audio
/System/Library/Spotlight//MacOS/Application
/System/Library/Caches/com
.apple.IntlDataCache.le.tecx
/System/Library/Spotlights/MacOS/Archives
/System/Library/Spotlight/
Mail.mdimporter/Contents/MacOS/Mail
/System/Library/Spotlight//Automator
/System/Library/Frs/A/ExceptionHandling
/Library/Fonts/Arial/..namedfork/rsrc
/System/Library/Spotlightr
/Contents/MacOS/Image

/Library/Fonts/Palatino/..namedfork/rsrc
/System/Library/Fonts/ZapfDingbats.dfont
/System/Library/CoreServices/RawCamera
.bundle/Contents/MacOS/RawCamera
/Library/Fonts/Andale Mono.ttf
/Library/Fonts/Courier New/
..namedfork/rsrc
/System/Library/Frameworks
/Carbon.framework
/Versions/A/Frameworks/HIToolbo
x.framework
/Versions/A/Resources/English
.lproj/Localized.rsrc
/System/Library/Fonts/Times.dfont
/Library/Fonts/Tahoma Bold.ttf
/System/Library/Fonts/Courier.dfont
/Library/QuickTime/DivX Decode
r.component/Contents
/MacOS/DivX Decoder
/Library/Fonts/Tahoma.ttf
/Library/Fonts/Trebuchet
MS/namedfork/rsrc
/Library/Fonts/Arial Bold.ttf
/System/Library/Fonts/Helvetica.dfont
/Library/Fonts/Arial Italic.ttf
/Library/Fonts/GillSans.dfont
/Library/Fonts/Times New Roman.ttf
/Library/Caches/com.apple.
LaunchServices-023501.csstore
//Users/miguelcardoso/Livros
/Anthropology of Naming.pdf
/Library/Fonts/Comic Sans M
S/..namedfork/rsrc
/System/Library/Fonts/AppleGothic.ttf
/dev/null
/dev/null
/dev/null
count=0, state=0x2
/Library/Spotlight
/System/Library/Spotlight
/Users/miguelcardoso/Livros/Livros/.DS_Store
count=0, state=0x2
/Users/miguelcardoso/L/Livros/Anthropology of Naming.pdf


Good stuff, no? It looks like it's indexing, bringing in new stuff, though with a lot of repetition (e.g. Palliative Care pdf and Anthropology of Naming pdf) even though no longer giving me the consolatory courtesy of the progress bar.

I wonder whether I should just wait another day or two before buying DiskWarrior(which I can't really afford right now).

But again. Thank you so much, mrzarquon! You're a gentleman and a scholar, Sir! All the best to you!
posted by MiguelCardoso at 7:49 AM on November 1, 2007

Yep indexes it's own files, and doesn't appear to modify the original PDF, so I don't imagine a possible conflict there.

Unfortunately, it doesn't appear to be a "here is this one file that is causing problems", so I would begin to suspect the hard drive (but disk util hasn't reported anything) or directory structure.

You say you have 200gb free, so running low on disk space shouldn't be a problem.

I would disable spotlight, and use the other search tool you found, until you can pickup a copy of diskwarrior to run on it. To be on the safe side, I would also make a copy / burn a dvd of all your important files, incase this is symptomatic of a slow hard drive failure.
posted by mrzarquon at 9:05 AM on November 1, 2007

Unfortunately, it doesn't appear to be a "here is this one file that is causing problems", so I would begin to suspect the hard drive (but disk util hasn't reported anything) or directory structure.

*South Carolina accent*
Well, mrzarquon, wasn't I saying, from the very beginning, it was directory structure all along?

No, you´re right. Meanwhile, I'm really impressed with Euro-Apple: the wise and resourceful guy I'd talked to yesterday phoned me (from Cork in Ireland) and talked me through a series of tests for two hours or so. Amazing.

Summing up: Spotlight is indexing away with mdworker quite frantic but no data/sec is read or written. (When indexing from the terminal, data read & written/sec was normal ). Trashing all 3 relevant caches and restarting Spotlight made no difference. Same with other efforts.

So I'm doing exactly as you say (and Clever Apple Guy agrees - he even noted one of your cunning sudo moves - that deadly Spotlight-V100 one) and have taken Spotlight out of my life and computer. The rest of the system is running swiftly and smoothly and I'll wait calmly for the Apple engineers to analyse the data capture folder I sent them and start saving up for diskwarrior.

Thank you with a big brass band, mrzarquon and everybody else who helped! I'll report back when the problem is solved!

Cheers!
posted by MiguelCardoso at 12:14 PM on November 1, 2007

Thats good to hear. Keep me updated, just shoot me an email when you get a resolution form apple / if diskwarrior did it, my address is in my profile. (I have a suspicion I might have to deal with this again as more users start upgrading to leopard).
posted by mrzarquon at 1:26 PM on November 1, 2007