Time Machine Failure (a different one)

[sliberty]

I started getting the dreaded "Not Enough Space" msgs the other day. So I paid close attention to what TM was doing when it ran. Here is what I saw.

At the start of the backup, TM said that it needed to backup up 158MB. This seemed low to me, but maybe I didn't change much. But since I had 15GB free on my TM drive (dedicated), nothing needed to be deleted.

However, about half way through the backup, TM started to "update" the estimate. By the end of the backup, it was saying that it needed to backup 30GB, which there wasn't enough space for, so it failed.

I called Tech Support, and they had no clue. They escalated it to engineering, but have not gotten back to me.

After such failures, I noticed that a file was sitting on the TM drive that should have been cleaned up. It had InProgress ads it's extension. I deleted this file after checking with Tech Support to see if that was safe to do.

In the mean time, I excluded any virtual machine images that I have on my drive as they are big, volatile, and non-critical from a data perspective. Since then, my backups have been succeeding, but I also now have 76GB free (something must have been deleted.

It seems as though the system was taking too long to determine the amount of data it needed to back up, and as a result, was already past the delete step before a proper estimate was reached. Bad design for sure if that is what was going on.

Anyone see this before?

[anthlover]

More information please..
What version of OSX?
Are you backing up to a Time Capsule or HD attached to an Extreme?
Are you backing up wirelessly, ethernet or USB?

Also you might be aware from other threads. There is not a lot of love here for TM, particularly wirelessly which employs sparse bundles that can get corrupt too easily. Everyone will try to help as much as possible with more information. Many of us of just too many incidents and now many of us employ other tools.

Some of the others may know more of the ins and outs of the TM issues you ran into.

Always recommend a combination of multiple separate back up drives (using CCC or Superduper [chiefly for bootable clones of your system which serve as a back up]) and Crash plan is great for offsite unlimited backups (file level) though it can also be used a a local tool too).

[sliberty]

Sorry, I should know better...

- Late 2012 Mac Mini (bought about a month ago - my first Mac). 500GB internal drive with 16GB RAM.
- Mountain Lion 10.8.2
- 1TB WD My Passport connected to Mac Mini via USB 3.0 port used for Time Machine backups
- 500GB WD drive, USB 3.0 attached for a Carbon Copy Cloner image.

The problem actually seems to be more related to my internal drive than my backup drive, in that TM is not calculating the amount of space it will need on my backup drive properly. First it thinks it needs very little space, and then later it decides it needs a lot more space, but by then it has passed the "delete stuff from backup drive" phases, and it is too late. I am not blaming the drive of course - just pointing out that the problem is not really related to where I back up to, but rather it seems related to where I am backing up from.

I do want to continue to have at least a two pronged approach to backups (not using a cloud based service as the 3rd step yet). Should I give up on TM altogether and do something else as my "other" backup? CCC seems OK for what it offers.

Thanks for any thoughts you might have on this.

[anthlover]

Welcome to Mac

Have you added anything non standard to your system? System mod, maintenance, tweaks, etc. sort of stuff, not apps? Make sure you run all the Mac OS updates that have come out. I would also look into the energy saver settings, in particular spinning HD down when possible. They do not always wake up well. In fact major brand Lacie, WD, Seagate, etc all also can have their own sleep settings that can wreak havoc with backups. I usually shut these features off if I can. Alternatively buy drives/cases from places that do not have those features. Macsales. Or 3rd party cases and buy your own drives.

Did you format the drives for the Mac when you got them their likely to be formatted fat32 out of the box. HFS Journaled would be preferred native format.

While we wait for some of our colleagues to offer more insight and perhaps ask your for TM logs.... Here my other thoughts so far:

TM local as you are doing should be a reasonable way to go. CCC can be scheduled to daily or more frequent back ups, it also offers an option to not retire any data. I personally shy away from that feature but I have not used that part of it for a long time.

* It is possible that your two pronged approach is part of the problem? Perhaps the two programs are conflicting. I always pause crashplan when I do back up with CCC. I believe you could do the same with TM by turning it off from control panels. I would test that for a bit and see if that changes the behavior your seeing.

** Also does your data change very frequently? ** ML already is trying to help you with versioning of it own. I still would periodically make multiple (copy revisions) of anything you put a lot of time into.

I would also tell you to strongly consider something like crash plan. It is one of the best. About $60 a year for unlimited storage (helps to have a fast internet connection). It is also able to back up to other computers or HDs.

Also I neglected to mention at the beginning awesome that your backing up

[sliberty]

Thanks for the various tips. I seem to be OK with most of them. No mods to the system. External disks all formatted for Mac (OS X Extended Journaled). I only run CCC when I feel like it so it wasn't running during the problems I mentioned. The big thing that I think might have been going on was that there was a Virtual Machine running at the time. I have since excluded the VMs from my TM backups.

As for it being awesome that I am backing up - I am an IT PM, so I know how important it is. ALways ran various types of backups when I ran Windows (including Ghost, which CCC replaces for me). As a PM, I am always concerned about what might happen

Thanks again for your thoughts. I look forward to hearing from others as well.

[anthlover]

TM scans almost all the time unless you turn it off so it is possible that it was running when you the hiccups were engendered, but you might not have been aware that it was running.

Were you able to suppress your USB externals sleep settings?

I did a quick search earlier and VMs sometimes throw TM off but you excluded them.

Also you did not mention how often your data is changing. If it is infrequent TM process is really not that necessary. You should know too that TM is a rolling back up. It eventually overwrites itself as space is consumed. Of course so does CCC and Superduper but then it is expected.

For a true archive with constant versioning of even with deleted items crash plan is hard to beat. They do offer multi level encryption. Application level and then blowfish level encryption for the data itself with passwords they are not able to give out as they do not have them. They also offer key generation but thats overkill for me.

Good Luck

[sliberty]

Regarding how frequently my data changes, not much changes in real terms. However, before I had excluded the VMs, I could imagine that they might have been "changing" all of the time while running. And with a virtual disk size of 10-30GB, that is a lot of variation from TMs point of view. It doesn't know that only 1 bit in 1 file changed. it would see the entire virtual disk as having changed. That is why I decided to exclude. But I had not excluded them when I was having the trouble. So perhaps the VMs were the entire cause of my trouble????

Crash Plan looks interesting. I have to read more about it as I am a bit confused at the moment. Can I use them for unlimited online backups? or must I have additional disks that they will write to? Also, can it back up my NAS devices too, or just my Mac's internal HDs?

[mclbruce]

Quote: Originally Posted by sliberty ... before I had excluded the VMs, I could imagine that they might have been "changing" all of the time while running. ... it would see the entire virtual disk as having changed. ... But I had not excluded them when I was having the trouble. So perhaps the VMs were the entire cause of my trouble????

I think that is exactly right.

Quote: Originally Posted by sliberty Crash Plan looks interesting. I have to read more about it as I am a bit confused at the moment. Can I use them for unlimited online backups? or must I have additional disks that they will write to? Also, can it back up my NAS devices too, or just my Mac's internal HDs?

For free you cannot back up to their servers. You need space of your own to back up to. I don't think Crashplan will run on a NAS, but if a NAS drive was always mounted on your Mac (or a Windows or Linux machine), I think you could back up the NAS using Crashplan on the Mac (or other appropriate machine).

Also, the free version is not continuous backup like Time Machine. If you want continuous backup, you have to pay, and you get a certain amount of space on their servers for whatever price you pay. I wish they offered continuous backup without the server space for a lower price.

[anthlover]

1) Crash plan paid offers the following online/offsite unlimited backups
The paid version of accounts give unlimited online storage for one computer and as many drives as you have mounted on your computer. Your in complete control of what drives or data gets backed up. You can even back up drives of other computers, if their mapped to your computer.

2) You can also back up TO Other drives or Other computers (these functions are free)

Basically they offer 2 non business plans for people with a lot of data: 1 computer with all of the above for around $50 a year or a family plan for $119 a year which covers up to 10 computers.

I have used their service for a few of years. It works great. I have restored data from it. The program even alerted me when a few of my my old photos was bad (crash had tried to re-back up the files). I was then able to recover the photo from their versioning of my files from their on line back up. If their program had not told me the files were bad I would not have known. I believe my other back up programs missed the problem.

-----------
http://www.crashplan.com/consumer/details.html detailed overview
http://www.crashplan.com/consumer/crashplan-plus.html overview
http://www.crashplan.com/consumer/compare.html product comparison, e..g individual, family, free, etc.

If you are curious I did not find crash plan first. I tested Mozy, Carbonite, Backblaze, Jungledisk, etc. I found other services simply did not work well or throttled my uploading bandwidth. I have FIOS which offers a lot of uplink speed. Only crashplan let me actually upload data at some semblance of the bandwidth I had available. Crashplan has defaults for how much CPU and Bandwidth to take from your computer home network (lan/wan). You can change these and also choose when versus always backups happen.

Naturally like a local back up the biggest hump is the initial full back up of your system, the first online back up could take days. They do offer a seeding option where they send you a drive and you fill it up and then they put its data online. I did not find that necessary.
My dataset is 300GB.

[anthlover]

I almost forgot to mention as an aside. While most feel Apple has gotten whole disk encryption "right" I only use small disk images for data for sensitive information (SS, passwords etc) these are left unmounted and locked until needed and they two are backed up. Long ago I was badly burned by whole disk encryption on client systems. Apple whole disk product is called File Vault.

I am organized enough that these small partitions are fine for my needs.

[sliberty]

Hmmm ... Encryption - something I have not yet addressed at all since making the switch.

Is the Mac encrypting by default? I don't really want encryption right now, so it if it, I need to turn that off. Later, when I am ready (so much to learn...) I can enable it and decide on a plan that suits me.

[anthlover]

File Vault 2 should be off by default. When I was burned it was not an Apple product but a third party one.

Others have more expertise I suppose but my Professional experience and common sense tells me adding complexity where it is not needed is not a good idea. I only apply it where I need it.

*Did the Crash summary help any

Others might disagree.