volume sizes

Doug Lassiter · 03-20-2013, 11:37 AM

OK, I've seen this a few times, and it's kind of aggravating. When I do "Get Info" on a volume, it can give me a "Used" size that is much larger than what Finder lists as actually being in that volume, when I total up the files and folders that are visible. Sometimes by factors of two or more. I presume this extra space is taken up in "hidden" files and folders somewhere. I have to presume that a lot of that stuff is just detritus that doesn't need to be backed up. How do I go about cleaning that up?

DeltaMac · 03-20-2013, 12:33 PM

Yep - your Finder knows how much space is really there. It just doesn't tell you that a lot of items are in hidden folders
Those various hidden folders are mostly part of your system. They are normally hidden, as there's little in those folders that most users ever need to be concerned about.
If you REALLY want to check out what other folders you have, use a utility (such as TinkerTool, or a few others) that allows you to "Show hidden files". Someone else may describe how to use a terminal command to do that, and that's fine, too.
My hidden folders hold about 7GB of files. Most of that is a single 4GB sleepimage file. The size of THAT file will depend on how much RAM you have installed. If you have 16GB of RAM (not uncommon on today's Macs), then that one sleepimage file will be more than 16GB!
I think intelligent backup software may ignore a few of those (essentially temporary) files (which rebuild automatically, if absent)
Viewing hidden folders will also show you the various folders that a standard or admin user doesn't have permission to modify or even view.

In my opinion, there's seldom anything that is worth "cleaning up" in those hidden folders (and a lot that will leave you with a 'bricked' system if you decide to randomly delete some files in those folders

Doug Lassiter · 03-20-2013, 01:13 PM

Well, I can tolerate some you-don't-need-too-know-about-this files. But I have one machine for which "Get Info" is looking at 85GB, and yet the "visible" files only total up to 22GB. That's a lot more than what I don't need to know about. I've heard about even worse cases.

For example, when I'm doing a backup into a 50MB partition, it's a little silly for my system to insist that I don't have enough room to do it!

So I'm not wanting to get rid of hidden files in general, but just what has to be a lot of unused stuff. At least some explanation about why my disk is dominated by stuff I don't know about.

DeltaMac · 03-20-2013, 01:46 PM

I agree, that appears to be way too much.
My expectation is that the data is NOT unused, but more likely OVERused (or improperly stored or created, due to some odd glitch in your system). For example, it could be temp files used for burning DVDs, that is not being automatically removed after the DVD is burned. As you can imagine, keeping 4 to 8 GB files unnecessarily could eat up space in large amounts.

As one method to find that out, you can use the free app, OmniDiskSweeper, which will scan your whole drive, and show you exactly where that space is being used, even in hidden files/folders.
I would suggest (because the files might not be easy to identify) that you come back with folder/file names, and someone here may give you a good guess (hopefully an "educated" guess) about those files, and if they can be safely deleted.
In my experience, when WAY too much space is used, you may have a runaway log file, or some other process that has simply created an enormously inflated file somewhere. That utility OmniDiskSweeper, should help you determine that. http://www.omnigroup.com/products/omnidisksweeper/

hayne · 03-20-2013, 06:38 PM

Sometimes the issue is that the program that is doing the reporting of disk space used (here Finder) does not have permission to find out how much space is being used by a file.
E.g. if a log file belongs to the 'root' user and has permissions set to prevent other users from reading it, or is in a folder with such permissions, then 'root' privileges are needed to determine how much space is used by that file.
I don't know whether Finder makes use of 'root' privileges to find out how much disk space is used, but if it doesn't, then that's an easy way for the discrepancy to arise.

And utility programs such as OmniDiskSweeper may also have this problem - unless they have taken special steps to run with 'root' privileges, they will not be able to report on (let alone clean up) any disk space taken by files that are not readable by your user account.

Hence the most reliable way of finding out what is taking up space is as follows:
----
Open a Terminal window (Terminal is under /Applications/Utilities) and enter the following command:
sudo du -x -h -d 1 /
(It is best to copy & paste the command to be sure to get it right.)
Look to see which sub-folder is taking up the most space and repeat with that folder. If you have trouble interpreting the results, just copy & paste them back here and we can help.

Doug Lassiter · 03-20-2013, 08:57 PM

Hmm. Thanks. In fact, when I run OmniDiskSweeper, the volume totals 25.9GB (and yet, when I launch it, and select the volume, it sees 88.2GB on that volume -- presumably just read from the Finder). So OmniDiskSweeper isn't seeing what's in it.

I did the root command, and .... ha! -- it sees 44GB in /Users! OmniDiskSweeper only saw 0.6GB there. In that /Users directory, I see much of that 44GB is lumped in /User/joe, who is a user (my elderly, and fairly operating system-illiterate dad, actually). A lot of admitted detritus, Music, Pictures, and even a load of unemptied trash. OK, so that's it. I have no big problem with any of that. BUT, the Finder doesn't see it! That's where my problem is. When I do a Finder list, Joe's account shows that it has "59.1MB" in it (which was suspiciously small).

OK, I guess Joe and I have to dive into Joe's account and find out what permissions 99% of these files have that an Administrator doesn't get to see. What else should I be looking for? I think I set this all up for him long ago, and I suspect I made a big error somewhere along the line in doing that.

DeltaMac · 03-20-2013, 09:39 PM

An admin account still won't see files that belong to another account.
Even though you can "administer" Joe's account, you don't own Joe's files, and therefore don't get access to the files (and the finder doesn't let you see Joe's files, even as an admin)
You CAN log in to the root user, where you will be able to weed out large files (and empty Joe's trash folder, too) regardless of who "owns" those files. The root is the next level above an admin account, and won't ask for authentication to do ANYTHING (good or evil) to the system. And, OmniDiskSweeper will also show it all to you while you are logged in to root.

Hayne's tip about using the terminal will also do the same thing for you, as you are then using a sudo to elevate your admin user temporarily to a superuser (I think sudo/superuser is the same as the root user), And, there's the additional advantage that sudo times out after a few minutes, so won't accidentally leave your Mac in the vulnerable condition of an enabled root user.

hayne · 03-20-2013, 09:54 PM

Be alert to the possibility that there is malware on the system (e.g. in Joe's account) and that perhaps the disk space is due to your computer being used as a repository for "wares" etc. Or maybe it's just a huge amount of detritus. But I'd try to understand where the files that are taking up space came from before cleaning up.

Doug Lassiter · 03-21-2013, 09:42 AM

Quote:

Originally Posted by DeltaMac


	And, OmniDiskSweeper will also show it all to you while you are logged in to root.

How do I use OmniDiskSweeper while logged in as root? That could be handy. I thought root could only be accessed through the terminal. That might ensure against doing bad stuff while in root.

DeltaMac · 03-21-2013, 11:22 AM

For the root user, you would need to enable the root account, then log out, and log back in to your root user. It is not available until the root account is enabled.
The method depends on which version of OS X you are using, but you can search for "enable root user" in your Mac help menu, which will show you the steps to get there.

Using software when logged in to the root user is exactly the same as when using software when logged in to your normal account.
However, you won't need permission to do anything, which is why it is hazardous to leave your computer in that condition.

Doug Lassiter · 03-21-2013, 02:31 PM

OK, well, I logged in as root, and went to Joe's account. Did "ls -l" on the various directories and files within that were not evidently getting summed by the Finder, in assessing file size. They are all owned by Joe -- rw permissions at least. Group is "staff", which at least has rx access.

For example, I can go into his large iTunes directory, and bore all the way down to the mp3s. Ownership on those looks fine.

I think what's going on is simply that when I use Get Info, or a Finder list view, or an OmniDiskSweeper catalog on /User/Joe, it gives me the total of what I'm allowed to see, and doesn't include in that total what I'm not allowed to see. BUT, when I use Disk Utility, it tells me the total of everything there, whether I'm allowed to see any of it or not. I guess that's what was confusing me.

So the issue isn't "hidden files" (which I know how to reveal), but files that I just don't have permission to see.

The bottom line then is that nothing is wrong, but that Joe's account was just dominating the volume. It just didn't occur to me that the Finder would not only not show me Joe's files, but wouldn't even tell me how much Joe had in his directory. I guess that's fair, but it does make non-root system management a bit hard. You'd like to believe an Administrator could look and see how big any user directories really are.

saint.duo · 03-22-2013, 02:17 PM

If you are comfortable with the terminal, I regularly use the following command to get the size of every user account's folder without enabling root or delving into the folders.

Code:

sudo du -sh /Users/*

This adds up the contents of each user folder and lists the summary of sizes, in easily readable sizes.

ganbustein · 03-23-2013, 03:43 PM

Are you on Lion or later? Is this a portable computer? You really should supply this information without making us ask.

The reason I ask is that, with Lion, Time Machine has a new feature called "local snapshots". This feature is on by default on portables, and off by default on desktops.

What it does is, when your computer is not attached to its Time Machine backup disk, it backs up to a hidden filesystem on your computer instead. Later, when you do connect to your TM backup disk, this local backup gets copied to the normal backup, and purged. The end result is that, if you take your laptop on the road for a week, during which you're working furiously on your new novel, you'll still see hourly snapshots of your masterpiece all week long. By the time you get back from your trip, most of the hourly backups will have expired, of course, but the "real" backup disk still gets the ones that haven't and all the daily snapshots for the whole week. In other words, your backup disk ends up with exactly the same information it would have had if it had been connected to the portable the whole time. Without the "local snapshots" feature, Time Machine would see only the version that existed at the end of the week.

These local snapshots are in a hidden filesystem, but they do take up disk space just the same. Finder will report the space as unused, because Time Machine will automatically give it back if anyone needs it, but Disk Utility and Get Info will show it as in use, because it is.

To see the current size of your local snapshots, go to About This Mac..., click on More Info..., and select the Storage tab.

acme.mail.order · 03-23-2013, 08:42 PM

Quote:

Originally Posted by Doug Lassiter


	I think what's going on is simply that when I use Get Info, or a Finder list view, or an OmniDiskSweeper catalog on /User/Joe, it gives me the total of what I'm allowed to see, and doesn't include in that total what I'm not allowed to see. BUT, when I use Disk Utility, it tells me the total of everything there, whether I'm allowed to see any of it or not. I guess that's what was confusing me.

That is correct. Info on a folder requires listing each file and folder, and that requires suitable permissions.

Info on a disk (either in the finder or Disk Utility) simply looks at the disk directory which tells you how many blocks the disk has, and how many are in use. No file-level read required. The command-line version is `df` (Display Free space), most commonly with the -h option to do the math for you.

Another very handy utility for finding large files is GrandPerspective, also permissions sensitive.

Doug Lassiter · 03-24-2013, 11:30 AM

Thanks to all for the advice. I guess this was just a matter of me not being sensitive to what permissions enabled, and what they didn't enable.

The idea of using "sudo du -sh /Users/*" is a good one, though one would have thought that an Administrator could have this functionality as part of the OS without having to use root. Seems that functionality is one that should be easily available to a system administrator. I think that's what had me baffled. The privileges of admin seem to be a bit weak.

I'm working 10.6.8 on the machine in question, though I have Lion on others. That being the case, it's useful to be aware of local snapshots.

03-20-2013, 11:37 AM	#1
Doug Lassiter Major Leaguer Join Date: Apr 2004 Location: Burnett, Texas Posts: 286	volume sizes OK, I've seen this a few times, and it's kind of aggravating. When I do "Get Info" on a volume, it can give me a "Used" size that is much larger than what Finder lists as actually being in that volume, when I total up the files and folders that are visible. Sometimes by factors of two or more. I presume this extra space is taken up in "hidden" files and folders somewhere. I have to presume that a lot of that stuff is just detritus that doesn't need to be backed up. How do I go about cleaning that up?

03-20-2013, 06:38 PM	#5
hayne Site Admin Join Date: Jan 2002 Location: Montreal Posts: 31,938	Sometimes the issue is that the program that is doing the reporting of disk space used (here Finder) does not have permission to find out how much space is being used by a file. E.g. if a log file belongs to the 'root' user and has permissions set to prevent other users from reading it, or is in a folder with such permissions, then 'root' privileges are needed to determine how much space is used by that file. I don't know whether Finder makes use of 'root' privileges to find out how much disk space is used, but if it doesn't, then that's an easy way for the discrepancy to arise. And utility programs such as OmniDiskSweeper may also have this problem - unless they have taken special steps to run with 'root' privileges, they will not be able to report on (let alone clean up) any disk space taken by files that are not readable by your user account. Hence the most reliable way of finding out what is taking up space is as follows: ---- Open a Terminal window (Terminal is under /Applications/Utilities) and enter the following command: sudo du -x -h -d 1 / (It is best to copy & paste the command to be sure to get it right.) Look to see which sub-folder is taking up the most space and repeat with that folder. If you have trouble interpreting the results, just copy & paste them back here and we can help. __________________ hayne.net/macosx.html

03-20-2013, 09:54 PM	#8
hayne Site Admin Join Date: Jan 2002 Location: Montreal Posts: 31,938	Be alert to the possibility that there is malware on the system (e.g. in Joe's account) and that perhaps the disk space is due to your computer being used as a repository for "wares" etc. Or maybe it's just a huge amount of detritus. But I'd try to understand where the files that are taking up space came from before cleaning up. __________________ hayne.net/macosx.html

03-22-2013, 02:17 PM	#12
saint.duo MVP Join Date: Jan 2002 Location: Seattle Posts: 1,077	If you are comfortable with the terminal, I regularly use the following command to get the size of every user account's folder without enabling root or delving into the folders. Code: sudo du -sh /Users/* This adds up the contents of each user folder and lists the summary of sizes, in easily readable sizes.

Thread Tools
Show Printable Version Email this Page
Display Modes	Rate This Thread
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode	Rate This Thread:

03-20-2013, 12:33 PM	#2
DeltaMac League Commissioner Join Date: Jan 2002 Posts: 7,941	Yep - your Finder knows how much space is really there. It just doesn't tell you that a lot of items are in hidden folders Those various hidden folders are mostly part of your system. They are normally hidden, as there's little in those folders that most users ever need to be concerned about. If you REALLY want to check out what other folders you have, use a utility (such as TinkerTool, or a few others) that allows you to "Show hidden files". Someone else may describe how to use a terminal command to do that, and that's fine, too. My hidden folders hold about 7GB of files. Most of that is a single 4GB sleepimage file. The size of THAT file will depend on how much RAM you have installed. If you have 16GB of RAM (not uncommon on today's Macs), then that one sleepimage file will be more than 16GB! I think intelligent backup software may ignore a few of those (essentially temporary) files (which rebuild automatically, if absent) Viewing hidden folders will also show you the various folders that a standard or admin user doesn't have permission to modify or even view. In my opinion, there's seldom anything that is worth "cleaning up" in those hidden folders (and a lot that will leave you with a 'bricked' system if you decide to randomly delete some files in those folders

03-20-2013, 01:13 PM	#3
Doug Lassiter Major Leaguer Join Date: Apr 2004 Location: Burnett, Texas Posts: 286	Well, I can tolerate some you-don't-need-too-know-about-this files. But I have one machine for which "Get Info" is looking at 85GB, and yet the "visible" files only total up to 22GB. That's a lot more than what I don't need to know about. I've heard about even worse cases. For example, when I'm doing a backup into a 50MB partition, it's a little silly for my system to insist that I don't have enough room to do it! So I'm not wanting to get rid of hidden files in general, but just what has to be a lot of unused stuff. At least some explanation about why my disk is dominated by stuff I don't know about.

03-20-2013, 01:46 PM	#4
DeltaMac League Commissioner Join Date: Jan 2002 Posts: 7,941	I agree, that appears to be way too much. My expectation is that the data is NOT unused, but more likely OVERused (or improperly stored or created, due to some odd glitch in your system). For example, it could be temp files used for burning DVDs, that is not being automatically removed after the DVD is burned. As you can imagine, keeping 4 to 8 GB files unnecessarily could eat up space in large amounts. As one method to find that out, you can use the free app, OmniDiskSweeper, which will scan your whole drive, and show you exactly where that space is being used, even in hidden files/folders. I would suggest (because the files might not be easy to identify) that you come back with folder/file names, and someone here may give you a good guess (hopefully an "educated" guess) about those files, and if they can be safely deleted. In my experience, when WAY too much space is used, you may have a runaway log file, or some other process that has simply created an enormously inflated file somewhere. That utility OmniDiskSweeper, should help you determine that. http://www.omnigroup.com/products/omnidisksweeper/

03-20-2013, 08:57 PM	#6
Doug Lassiter Major Leaguer Join Date: Apr 2004 Location: Burnett, Texas Posts: 286	Hmm. Thanks. In fact, when I run OmniDiskSweeper, the volume totals 25.9GB (and yet, when I launch it, and select the volume, it sees 88.2GB on that volume -- presumably just read from the Finder). So OmniDiskSweeper isn't seeing what's in it. I did the root command, and .... ha! -- it sees 44GB in /Users! OmniDiskSweeper only saw 0.6GB there. In that /Users directory, I see much of that 44GB is lumped in /User/joe, who is a user (my elderly, and fairly operating system-illiterate dad, actually). A lot of admitted detritus, Music, Pictures, and even a load of unemptied trash. OK, so that's it. I have no big problem with any of that. BUT, the Finder doesn't see it! That's where my problem is. When I do a Finder list, Joe's account shows that it has "59.1MB" in it (which was suspiciously small). OK, I guess Joe and I have to dive into Joe's account and find out what permissions 99% of these files have that an Administrator doesn't get to see. What else should I be looking for? I think I set this all up for him long ago, and I suspect I made a big error somewhere along the line in doing that.

03-20-2013, 09:39 PM	#7
DeltaMac League Commissioner Join Date: Jan 2002 Posts: 7,941	An admin account still won't see files that belong to another account. Even though you can "administer" Joe's account, you don't own Joe's files, and therefore don't get access to the files (and the finder doesn't let you see Joe's files, even as an admin) You CAN log in to the root user, where you will be able to weed out large files (and empty Joe's trash folder, too) regardless of who "owns" those files. The root is the next level above an admin account, and won't ask for authentication to do ANYTHING (good or evil) to the system. And, OmniDiskSweeper will also show it all to you while you are logged in to root. Hayne's tip about using the terminal will also do the same thing for you, as you are then using a sudo to elevate your admin user temporarily to a superuser (I think sudo/superuser is the same as the root user), And, there's the additional advantage that sudo times out after a few minutes, so won't accidentally leave your Mac in the vulnerable condition of an enabled root user.

03-21-2013, 11:22 AM	#10
DeltaMac League Commissioner Join Date: Jan 2002 Posts: 7,941	For the root user, you would need to enable the root account, then log out, and log back in to your root user. It is not available until the root account is enabled. The method depends on which version of OS X you are using, but you can search for "enable root user" in your Mac help menu, which will show you the steps to get there. Using software when logged in to the root user is exactly the same as when using software when logged in to your normal account. However, you won't need permission to do anything, which is why it is hazardous to leave your computer in that condition.

03-21-2013, 02:31 PM	#11
Doug Lassiter Major Leaguer Join Date: Apr 2004 Location: Burnett, Texas Posts: 286	OK, well, I logged in as root, and went to Joe's account. Did "ls -l" on the various directories and files within that were not evidently getting summed by the Finder, in assessing file size. They are all owned by Joe -- rw permissions at least. Group is "staff", which at least has rx access. For example, I can go into his large iTunes directory, and bore all the way down to the mp3s. Ownership on those looks fine. I think what's going on is simply that when I use Get Info, or a Finder list view, or an OmniDiskSweeper catalog on /User/Joe, it gives me the total of what I'm allowed to see, and doesn't include in that total what I'm not allowed to see. BUT, when I use Disk Utility, it tells me the total of everything there, whether I'm allowed to see any of it or not. I guess that's what was confusing me. So the issue isn't "hidden files" (which I know how to reveal), but files that I just don't have permission to see. The bottom line then is that nothing is wrong, but that Joe's account was just dominating the volume. It just didn't occur to me that the Finder would not only not show me Joe's files, but wouldn't even tell me how much Joe had in his directory. I guess that's fair, but it does make non-root system management a bit hard. You'd like to believe an Administrator could look and see how big any user directories really are.

03-23-2013, 03:43 PM	#13
ganbustein MVP Join Date: Apr 2008 Location: Berkeley CA USA Posts: 1,008	Are you on Lion or later? Is this a portable computer? You really should supply this information without making us ask. The reason I ask is that, with Lion, Time Machine has a new feature called "local snapshots". This feature is on by default on portables, and off by default on desktops. What it does is, when your computer is not attached to its Time Machine backup disk, it backs up to a hidden filesystem on your computer instead. Later, when you do connect to your TM backup disk, this local backup gets copied to the normal backup, and purged. The end result is that, if you take your laptop on the road for a week, during which you're working furiously on your new novel, you'll still see hourly snapshots of your masterpiece all week long. By the time you get back from your trip, most of the hourly backups will have expired, of course, but the "real" backup disk still gets the ones that haven't and all the daily snapshots for the whole week. In other words, your backup disk ends up with exactly the same information it would have had if it had been connected to the portable the whole time. Without the "local snapshots" feature, Time Machine would see only the version that existed at the end of the week. These local snapshots are in a hidden filesystem, but they do take up disk space just the same. Finder will report the space as unused, because Time Machine will automatically give it back if anyone needs it, but Disk Utility and Get Info will show it as in use, because it is. To see the current size of your local snapshots, go to About This Mac..., click on More Info..., and select the Storage tab.

03-24-2013, 11:30 AM	#15
Doug Lassiter Major Leaguer Join Date: Apr 2004 Location: Burnett, Texas Posts: 286	Thanks to all for the advice. I guess this was just a matter of me not being sensitive to what permissions enabled, and what they didn't enable. The idea of using "sudo du -sh /Users/*" is a good one, though one would have thought that an Administrator could have this functionality as part of the OS without having to use root. Seems that functionality is one that should be easily available to a system administrator. I think that's what had me baffled. The privileges of admin seem to be a bit weak. I'm working 10.6.8 on the machine in question, though I have Lion on others. That being the case, it's useful to be aware of local snapshots.