I feel like I'm in a very boring episode of Doctor Who
September 4, 2013 8:09 AM Subscribe
I'm having a weird problem with Dropbox, and googling is failing me because lots of people can't find their Dropbox files. But I can't find them in a very special and exciting way, involving differences between Finder and the command line. What am I dealing with here?
I use a Dropbox account across a couple of computers. The oldest member of the group is a Macbook Pro running 10.6.7. Lately, it syncs correctly to all appearances, *so long as I only attempt to interact with my files through Finder.* New folders and files sync to the Dropbox account, deleted files delete correctly, all is well.
However, when accessed through a terminal, and I think any program using the unix file interface (Matlab definitely, maybe others), my file directory is stuck in the past/an alternate timestream. Both file directories appear to be fully 'real', but they don't talk to each other. To illustrate:
In Finder, I can:
Make a new folder, drop a new textfile into it. Both sync happily to dropbox, persists system restart, etc. However, in the terminal this folder doesn't appear to ls -a, and it cannot be cd'd into.
In the terminal:
mkdir fakefile
cd fakefile
echo "sometext" >> myfile.txt
creates a folder and text file which can be printed to the terminal, persists past system restart, etc. etc. But it is not visible in Finder, and it doesn't sync to the Dropbox account. The result of this is that my terminal directory is largely from the past, modulo a variety of test updates and files created by other users. I'm inclined to think that this is the more broken of the directory versions, because of a secondhand report/experience of dataloss in that folder (someone was writing files to a folder which exists only in Terminal land, and those datafiles are not present anymore. Possibly this is unrelated user error though.)
All of this behavior is 1) unique to the Dropbox folder, but 2) persists when the computer is not connected to the internet and/or syncing is paused. Folders and files created e.g. in the Desktop folder appear normally in both Finder & terminal, delete appropriately, etc. I can create a file in either the terminal timestream or Finder timestream of the Dropbox folder, and if I then copy that file to the Desktop, both versions of the directory know about it.
Sadly, I don't know exactly when this happened or what might have started the problem- this computer is used mostly not by me, but by a small group of people who navigate the filesystem using Finder, and (should be) entering only a very small set of shell and Matlab commands.
What the hell is this, and more importantly, how do I fix it? I've not yet taken the nuclear option of removing Dropbox entirely from this computer and attempting a reinstall, both because it will derail several people's work until I can get it back online, and on the off chance of breaking something further before I know what's going on...
I use a Dropbox account across a couple of computers. The oldest member of the group is a Macbook Pro running 10.6.7. Lately, it syncs correctly to all appearances, *so long as I only attempt to interact with my files through Finder.* New folders and files sync to the Dropbox account, deleted files delete correctly, all is well.
However, when accessed through a terminal, and I think any program using the unix file interface (Matlab definitely, maybe others), my file directory is stuck in the past/an alternate timestream. Both file directories appear to be fully 'real', but they don't talk to each other. To illustrate:
In Finder, I can:
Make a new folder, drop a new textfile into it. Both sync happily to dropbox, persists system restart, etc. However, in the terminal this folder doesn't appear to ls -a, and it cannot be cd'd into.
In the terminal:
mkdir fakefile
cd fakefile
echo "sometext" >> myfile.txt
creates a folder and text file which can be printed to the terminal, persists past system restart, etc. etc. But it is not visible in Finder, and it doesn't sync to the Dropbox account. The result of this is that my terminal directory is largely from the past, modulo a variety of test updates and files created by other users. I'm inclined to think that this is the more broken of the directory versions, because of a secondhand report/experience of dataloss in that folder (someone was writing files to a folder which exists only in Terminal land, and those datafiles are not present anymore. Possibly this is unrelated user error though.)
All of this behavior is 1) unique to the Dropbox folder, but 2) persists when the computer is not connected to the internet and/or syncing is paused. Folders and files created e.g. in the Desktop folder appear normally in both Finder & terminal, delete appropriately, etc. I can create a file in either the terminal timestream or Finder timestream of the Dropbox folder, and if I then copy that file to the Desktop, both versions of the directory know about it.
Sadly, I don't know exactly when this happened or what might have started the problem- this computer is used mostly not by me, but by a small group of people who navigate the filesystem using Finder, and (should be) entering only a very small set of shell and Matlab commands.
What the hell is this, and more importantly, how do I fix it? I've not yet taken the nuclear option of removing Dropbox entirely from this computer and attempting a reinstall, both because it will derail several people's work until I can get it back online, and on the off chance of breaking something further before I know what's going on...
Response by poster: Thanks for the questions!
The web interface (and all other computers on the dropbox account) agree with Finder
sudo ls -a also shows the terminal version/'wrong' directory
I found and checked both ~/.dropbox and ~/Dropbox/.dropbox.cache, that's not where I am - this is definitely the main Dropbox folder.
posted by heyforfour at 8:33 AM on September 4, 2013
The web interface (and all other computers on the dropbox account) agree with Finder
sudo ls -a also shows the terminal version/'wrong' directory
I found and checked both ~/.dropbox and ~/Dropbox/.dropbox.cache, that's not where I am - this is definitely the main Dropbox folder.
posted by heyforfour at 8:33 AM on September 4, 2013
Response by poster: Whoops, missed your last question - apologies if I'm missing some terminology here, but no signing out or user switching is involved, this is all occurring as a single user.
posted by heyforfour at 8:37 AM on September 4, 2013
posted by heyforfour at 8:37 AM on September 4, 2013
Is this problem unique to a single computer? Try pointing Dropbox at a different directory, let everything sync, and try again from there.
posted by mkultra at 8:48 AM on September 4, 2013
posted by mkultra at 8:48 AM on September 4, 2013
What happens when you delete or modify files from the shell?
Can you log into the Dropbox web interface and see what file changes it has logged in the 'Events' portion of the interface?
posted by suedehead at 9:07 AM on September 4, 2013
Can you log into the Dropbox web interface and see what file changes it has logged in the 'Events' portion of the interface?
posted by suedehead at 9:07 AM on September 4, 2013
Does opening on of these files with a GUI text editor force a sync to happen?
posted by ridogi at 9:15 AM on September 4, 2013
posted by ridogi at 9:15 AM on September 4, 2013
Response by poster: (An update while I try the longer syncing solution mkultra suggests...)
- Unique to a single computer, yes!
- On the dropbox web interface, it logs only events that were conducted in Finder
- When I delete or modify files from the shell, they disappear from the shell, but persist in Finder. They are gone as far as ls -a is concerned, but still appear in the Finder and online in Dropbox, and no Dropbox event is logged.
And on preview - assuming from my brief google that symlinks refers to the everyday kind of shortcut/alias that I am used to, no, not involved.
posted by heyforfour at 9:16 AM on September 4, 2013
- Unique to a single computer, yes!
- On the dropbox web interface, it logs only events that were conducted in Finder
- When I delete or modify files from the shell, they disappear from the shell, but persist in Finder. They are gone as far as ls -a is concerned, but still appear in the Finder and online in Dropbox, and no Dropbox event is logged.
And on preview - assuming from my brief google that symlinks refers to the everyday kind of shortcut/alias that I am used to, no, not involved.
posted by heyforfour at 9:16 AM on September 4, 2013
One more thought, do you have selective sync turned on?
Symlinks are similar to aliases, but function differently and are created with ln.
posted by ridogi at 9:22 AM on September 4, 2013
Symlinks are similar to aliases, but function differently and are created with ln.
posted by ridogi at 9:22 AM on September 4, 2013
Mac OS existed before OS X came along and removed the original wheels and bolted on a solid UNIX undercarriage. I think you're poking at one of the seams between them.
Specifically, the UNIX filesystem does not use things like resource forks, so they were implemented ad hoc: file.txt gets its metadata in ._.file.txt when you copy things on a USB stick, for example.
The Dropbox service is monitoring events in the Finder database - I don't have the exact terminology in front of me - and an event in that database triggers a sync event. Direct UNIX filesystem access (cat text >> file.txt) does not trigger an update to that db, I think - it gets picked up in one of the periodic sweeps. In the meantime, the Dropbox monitoring service is getting confused.
You wanted fixes.
* Why are you on 10.6.7 instead of the one true Snow Leopard release (10.6.8)? And you could update to Mountain Lion too - it isn't bad. (I skipped Lion, like I'm guessing you skipped Leopard.)
* We collaborate with Dropbox folders, and the way we like to do it is: no direct read/write/edit actions in the folder. Do everything in a copy, and periodically rsync the copy folder into your Dropbox subfolder. Then Dropbox will reliably replicate your version to others. Added bonus: you don't litter the folder with temporary files. Downside: it doesn't magically just work any more. Oh well.
posted by RedOrGreen at 9:30 AM on September 4, 2013
Specifically, the UNIX filesystem does not use things like resource forks, so they were implemented ad hoc: file.txt gets its metadata in ._.file.txt when you copy things on a USB stick, for example.
The Dropbox service is monitoring events in the Finder database - I don't have the exact terminology in front of me - and an event in that database triggers a sync event. Direct UNIX filesystem access (cat text >> file.txt) does not trigger an update to that db, I think - it gets picked up in one of the periodic sweeps. In the meantime, the Dropbox monitoring service is getting confused.
You wanted fixes.
* Why are you on 10.6.7 instead of the one true Snow Leopard release (10.6.8)? And you could update to Mountain Lion too - it isn't bad. (I skipped Lion, like I'm guessing you skipped Leopard.)
* We collaborate with Dropbox folders, and the way we like to do it is: no direct read/write/edit actions in the folder. Do everything in a copy, and periodically rsync the copy folder into your Dropbox subfolder. Then Dropbox will reliably replicate your version to others. Added bonus: you don't litter the folder with temporary files. Downside: it doesn't magically just work any more. Oh well.
posted by RedOrGreen at 9:30 AM on September 4, 2013
Best answer: Also, it really sounds like your direct access folder has become separate from your Finder folder. Are you sure there isn't a user error here, where you're actually working on two separate copies of the directory that have slowly diverged over time?
Very sure?
Very very sure?
posted by RedOrGreen at 9:33 AM on September 4, 2013
Very sure?
Very very sure?
posted by RedOrGreen at 9:33 AM on September 4, 2013
When I delete or modify files from the shell, they disappear from the shell, but persist in Finder. They are gone as far as ls -a is concerned, but still appear in the Finder and online in Dropbox, and no Dropbox event is logged.
Wait - so if you create a blank text file in Finder called blankfile.txt, it appears in the terminal with "ls -a", yes? And if you do "touch blankfile.txt", and then "ls -a", then all of a sudden blankfile.txt disappears?
Can you try "ls -atl", or "find ."?
posted by suedehead at 10:05 AM on September 4, 2013
Wait - so if you create a blank text file in Finder called blankfile.txt, it appears in the terminal with "ls -a", yes? And if you do "touch blankfile.txt", and then "ls -a", then all of a sudden blankfile.txt disappears?
Can you try "ls -atl", or "find ."?
posted by suedehead at 10:05 AM on September 4, 2013
Response by poster: Does opening on of these files with a GUI text editor force a sync to happen?
Nope. open -a TextEdit fileCreatedInTerminal.txt opens it in Textedit, but does not force a sync. Interestingly, when I open a 'terminal only' file in the gui editor and then attempt to save it; I see the terminal version of the directory structure in the save menu. I can click around and save it to a shared/nondropbox place like the Desktop, but whenever I navigate back into the dropbox folder in this context, I'm in terminal-world.
posted by heyforfour at 10:28 AM on September 4, 2013
Nope. open -a TextEdit fileCreatedInTerminal.txt opens it in Textedit, but does not force a sync. Interestingly, when I open a 'terminal only' file in the gui editor and then attempt to save it; I see the terminal version of the directory structure in the save menu. I can click around and save it to a shared/nondropbox place like the Desktop, but whenever I navigate back into the dropbox folder in this context, I'm in terminal-world.
posted by heyforfour at 10:28 AM on September 4, 2013
Best answer: I can't help but wonder if these really are the same directory. Would you mind humouring me?
posted by Zozo at 10:38 AM on September 4, 2013
- In the Terminal, while you're in your Dropbox folder, what is the output of pwd?
- In Terminal, type cd and hit the spacebar. Then drag your Dropbox folder from the Finder onto the Terminal window (the path to the folder you just dropped should appear) and hit Enter. Now what's the output of pwd?
posted by Zozo at 10:38 AM on September 4, 2013
Response by poster: Oh frabjous day! Zozo, my terminal (or my clumsy hands) didn't support dragging files into the terminal, but you and RedOrGreen put me on the right track - at some point the dropbox sync location was changed (as was the pinned folder on the Dock), from /Users/myname/Dropbox to /Users/myname/Documents/Dropbox - I'm guessing by either a Dropbox upgrade, another user (or myself in a moment of amnesia!), meaning that all my terminal links were accessing the old copy.
I'm terribly pleased to have this be user error rather than some undiscovered rift in the unix/mac continuum, and thanks to all for helping me sort this out!
posted by heyforfour at 10:49 AM on September 4, 2013 [1 favorite]
I'm terribly pleased to have this be user error rather than some undiscovered rift in the unix/mac continuum, and thanks to all for helping me sort this out!
posted by heyforfour at 10:49 AM on September 4, 2013 [1 favorite]
This thread is closed to new comments.
Are you definitely in the Dropbox folder and not the invisible ~/.dropbox or ~/Dropbox/.dropbox.cache directories?
What does the web interface show?
Is the terminal user the same as the GUI user?
posted by ridogi at 8:29 AM on September 4, 2013