@CB said:
I think I need a new back-up solution too. Crashplan's client won't stay up for more than a few minutes without crashing for the last few weeks. Anyone else having trouble with it?
I normally recommend CrashPlan to people, but maybe not right now?
Some of my audio project files are rather gigantic, but I would have expected the software would be setup to accommodate this without requiring me to mod ini files by hand. Perhaps the Pro client is.
How did I miss this thread for so long? Sounds like people have covered most of my usual recommendations already, but I'll give my input on them anyway.
SpiderOak is amazing (I'm pretty sure I did a write-up on using SpiderOak a few years ago that was my only published article on this very site). They're a little expensive, but I feel that's worth it for the zero-knowledge encryption guarantee and the reliability of their service. The only time I had problems with them was right after they ran a massive deal where you could get 1TB of storage for a very low yearly price indefinitely and they picked up way more new customers than expected. They had some stability issues due to overloading for a little while after that, but that has long since been resolved. I seriously cannot recommend them enough if you're willing to pay a bit more for a drop dead simple and reliable encrypted backup service.... if you haven't signed up already, do ping me so I can send you a referral link.... every person you refer gets you 1GB of free storage in perpetuity (at least I think they still do that).
OwnCloud - I wouldn't use OwnCloud. It has had issues in the past with losing data, plus the project just got forked and a bunch of the devs left. Otherwise, see NextCloud below.
NextCloud - This is the fork of OwnCloud that a bunch of the devs went to. If I were going to go with one of these two projects, I'd choose this one. I've read that the data loss issues have been resolved, but I've never been particularly fond of the way they do data syncing in the first place. The fact that it's all handled with PHP scripts and plain files on disk is a questionable design choice in my opinion, especially when you start running into PHP script timeout and memory issues. Own/NextCloud are great products when it comes to Calendar sync, Contact sync and things like that, but I wouldn't trust my important files to it when there are better options out there.
Seafile - This is the roll-your-own solution that I use for centralized file storage and sync. It's a bit more complicated to set up than the above two, but it runs as it's own daemon (that you then proxy through a standard webserver) and uses a git-like data storage format which both make it more resilient to data loss in my experience. It also has baked in encryption on a per sync folder level, so you can add a layer of security to your sensitive data, but not bother with that overhead on things that don't need it.
SyncThing - This one isn't so much a backup solution as it is just a file sync solution. I use it for certain backup tasks though. Operates like bittorrent sync, sort of. If you're going to run your own server you can easily use it to automatically back up certain folders any time something changes (and there are various options for keeping historical versions). Probably not what you're looking for, but worth mentioning anyway. I mostly use it to copy backups of certain application's data (like aCar) off of my phone automatically every night.
Duplicity, Deja-Dup, Duplicati - (Note: I haven't used Duplicati before, but have used the other two and it looks quite similar) - These are all great solutions for automating encrypted backups to not inherently secure locations such as Amazon S3. They use standard encryption (GPG) and do incremental backups to save on bandwidth and storage. Combine them with Glacier (if you don't think you'll have to restore files often, since it costs extra to retrieve data from Glacier) and you have a very cost effective encrypted, off-site backup system. In the past, I have done backups to a local NAS device, then had a Linux VM that would use duplicity to encrypt and ship to S3 for "cold storage". Probably the only solution that would meet your cost requirements.
CrashPlan - Used it in the past and do not like it. Their software was always bloated (ugh. Java) and unreliable for me. Plus you're relying on your friends keeping their storage up and running for you to back up to (or paying them for storage, and if you're doing that, IMO, you're better off going with SpiderOak). This thing was removed from my and my wife's computers in less than a few months because it was such a resource hog.
I'm currently running the SpiderOak demo and it looks like it'll do what I need. I'm definitely choked by my network's upstream bandwidth for the initial upload but it seems to be quick enough on down-sync. I'd like a little more shell integration but I've verified that it works the same on both Linux and Windows so that's at least something. I'll probably try a cold restore this weekend.
Yeah, the few qualms I have with SpiderOak are shell integration and the fact that the CLI isn't great (you can run various SpiderOak commands from the CLI but for some reason on Windows the command ends up forking off to the background thus returning before the operation is complete making scripting impossible)
I have zero resource issues with CrashPlan. It works flawlessly and having friends is not required (though it's nice). I pay for CrashPlan pro and have never regretted it.
@primesuspect said:
I have zero resource issues with CrashPlan. It works flawlessly and having friends is not required (though it's nice). I pay for CrashPlan pro and have never regretted it.
This may have to do with how new your computer is. Running it on a decade old MacBook Pro really lets you feel how crap software written in java tends to be when it comes to resources. Same for a 6 year old Windows desktop. Once again, just in my experience.
@AlexDeGruven said:
My CrashPlan Pro app keeps crashing randomly, and I get the emails that my desktop isn't backed up. Can't see any rhyme or reason to it.
That's why I'm switching to my home-rolled solution of Macrium + Amazon. Also $5/mo instead of whatever I'm paying for CrashPlan.
That's the same problem I was having, and increasing the memory allocation fixed it.
I'm having a similar cloud quandary, but not related to backup. I've been using Dropbox for years for my cloud storage and sharing across accounts. For a while I was using SafeMonk to encrypt a subdirectory, then they went out of business and I started using Sookasa's free plan. Now Sookasa is retiring the free plan, and their cheapest paid plan is more expensive than a dedicated encrypted cloud storage solution.
However, I am kind of hooked into the Dropbox ecosystem in the sense that I'd have to convince everyone I share files with to also convert, if I left Dropbox completely. Any suggestions for a similar encryption solution to Sookasa that runs in a subdirectory of Dropbox, but is less expensive than the $100 or so per year Sookasa wants to start charging? I don't really need a lot of space, I'm cloud encrypting under one GB. For a while I tried TrueCrypt but it was too much of a pain to constantly load and unload the container to get it to sync back up to the cloud.
I have been using SpiderOak One for a few months now, and I love it. I Have 250GB of space for around $9/month. For everything that Dropbox has, SpiderOak does it better, with the exception of the mobile app. I guess we'll have to see where the mobile app goes, but I also understand that the mobile app has less features for security reasons.
BTW, here's some advice for anyone thinking about switching to SpiderOak One.
Because we are all advanced users here, it's best to shut off the Hive folder, which you can do in preferences. You'll have to do this for every device that is connected to SpiderOak.
The hive is supposed to give you a replica of Dropbox right out of the box. It's for people who are migrating, but don't understand all of the features yet. But it's pretty useless. You can't even change the folder location.
Start out with selecting the folders on each device that you want to backup. You are given a folder tree, and all you have to do is select the backup folders you want. They will automatically sync to the cloud. You have to complete this step before you can create syncs or shares.
If you want to sync a folder to another device, create an empty folder on the destination device and back it up. Then select the backup folder on the target device and sync it to the empty folder you just created.
Creating a shared public folder is really easy. Again, you have to backup the folder first, then create the share. They will give you a website link for accessing the shared folder publicly, and it includes folder navigation.
Mac & Linux users: Setup a backup/sync folder for your Screenshots to mimic Dropbox functionality.
If you want to sync a folder to another device, create an empty folder on the destination device and back it up. Then select the backup folder on the target device and sync it to the empty folder you just created.
I normally would use "destination" and "target" interchangeably, so I'm a little confused. This probably makes more sense if I was using SpiderOak and had a mental picture of the GUI.
If you want to sync a folder to another device, create an empty folder on the destination device and back it up. Then select the backup folder on the target device and sync it to the empty folder you just created.
I normally would use "destination" and "target" interchangeably, so I'm a little confused. This probably makes more sense if I was using SpiderOak and had a mental picture of the GUI.
Yeah, sorry. I didn't explain that well.
You can create as many syncs as you want, and the sync can be added to as many devices as you want.
You start with a folder that has some files in it. First, you backup that folder to the cloud. Call this Folder A.
Then, on the device you want to sync that folder to, you create an empty folder for it, and backup that folder to the cloud. (Since it's empty it just backs up the metadata for now) Call this Folder B.
Then, when you create the sync, you select Folder A and sync it to Folder B.
After this, both folders A & B will stay in sync to each other. Repeat on a third device, and you have a three-way sync.
Comments
I had to go higher than 3072, but this worked. It's running stable again.
I can once again recommend CrashPlan.
I would expect to find that you have files of X size, such that the noted size has to be X+some extra for crashplan itself
Some of my audio project files are rather gigantic, but I would have expected the software would be setup to accommodate this without requiring me to mod ini files by hand. Perhaps the Pro client is.
It must be a pro thing. I have raw video files and installers that are 4, 5, 6gb single files and I've never messed with anything. Shit just works.
Yep, I imagine it is a "simple" file splitting and reassembly they are doing to live within the memory constraints.
How did I miss this thread for so long? Sounds like people have covered most of my usual recommendations already, but I'll give my input on them anyway.
SpiderOak is amazing (I'm pretty sure I did a write-up on using SpiderOak a few years ago that was my only published article on this very site). They're a little expensive, but I feel that's worth it for the zero-knowledge encryption guarantee and the reliability of their service. The only time I had problems with them was right after they ran a massive deal where you could get 1TB of storage for a very low yearly price indefinitely and they picked up way more new customers than expected. They had some stability issues due to overloading for a little while after that, but that has long since been resolved. I seriously cannot recommend them enough if you're willing to pay a bit more for a drop dead simple and reliable encrypted backup service.... if you haven't signed up already, do ping me so I can send you a referral link.... every person you refer gets you 1GB of free storage in perpetuity (at least I think they still do that).
OwnCloud - I wouldn't use OwnCloud. It has had issues in the past with losing data, plus the project just got forked and a bunch of the devs left. Otherwise, see NextCloud below.
NextCloud - This is the fork of OwnCloud that a bunch of the devs went to. If I were going to go with one of these two projects, I'd choose this one. I've read that the data loss issues have been resolved, but I've never been particularly fond of the way they do data syncing in the first place. The fact that it's all handled with PHP scripts and plain files on disk is a questionable design choice in my opinion, especially when you start running into PHP script timeout and memory issues. Own/NextCloud are great products when it comes to Calendar sync, Contact sync and things like that, but I wouldn't trust my important files to it when there are better options out there.
Seafile - This is the roll-your-own solution that I use for centralized file storage and sync. It's a bit more complicated to set up than the above two, but it runs as it's own daemon (that you then proxy through a standard webserver) and uses a git-like data storage format which both make it more resilient to data loss in my experience. It also has baked in encryption on a per sync folder level, so you can add a layer of security to your sensitive data, but not bother with that overhead on things that don't need it.
SyncThing - This one isn't so much a backup solution as it is just a file sync solution. I use it for certain backup tasks though. Operates like bittorrent sync, sort of. If you're going to run your own server you can easily use it to automatically back up certain folders any time something changes (and there are various options for keeping historical versions). Probably not what you're looking for, but worth mentioning anyway. I mostly use it to copy backups of certain application's data (like aCar) off of my phone automatically every night.
Duplicity, Deja-Dup, Duplicati - (Note: I haven't used Duplicati before, but have used the other two and it looks quite similar) - These are all great solutions for automating encrypted backups to not inherently secure locations such as Amazon S3. They use standard encryption (GPG) and do incremental backups to save on bandwidth and storage. Combine them with Glacier (if you don't think you'll have to restore files often, since it costs extra to retrieve data from Glacier) and you have a very cost effective encrypted, off-site backup system. In the past, I have done backups to a local NAS device, then had a Linux VM that would use duplicity to encrypt and ship to S3 for "cold storage". Probably the only solution that would meet your cost requirements.
CrashPlan - Used it in the past and do not like it. Their software was always bloated (ugh. Java) and unreliable for me. Plus you're relying on your friends keeping their storage up and running for you to back up to (or paying them for storage, and if you're doing that, IMO, you're better off going with SpiderOak). This thing was removed from my and my wife's computers in less than a few months because it was such a resource hog.
I'm currently running the SpiderOak demo and it looks like it'll do what I need. I'm definitely choked by my network's upstream bandwidth for the initial upload but it seems to be quick enough on down-sync. I'd like a little more shell integration but I've verified that it works the same on both Linux and Windows so that's at least something. I'll probably try a cold restore this weekend.
Yeah, the few qualms I have with SpiderOak are shell integration and the fact that the CLI isn't great (you can run various SpiderOak commands from the CLI but for some reason on Windows the command ends up forking off to the background thus returning before the operation is complete making scripting impossible)
I have zero resource issues with CrashPlan. It works flawlessly and having friends is not required (though it's nice). I pay for CrashPlan pro and have never regretted it.
This may have to do with how new your computer is. Running it on a decade old MacBook Pro really lets you feel how crap software written in java tends to be when it comes to resources. Same for a 6 year old Windows desktop. Once again, just in my experience.
My CrashPlan Pro app keeps crashing randomly, and I get the emails that my desktop isn't backed up. Can't see any rhyme or reason to it.
That's why I'm switching to my home-rolled solution of Macrium + Amazon. Also $5/mo instead of whatever I'm paying for CrashPlan.
That's the same problem I was having, and increasing the memory allocation fixed it.
I'm having a similar cloud quandary, but not related to backup. I've been using Dropbox for years for my cloud storage and sharing across accounts. For a while I was using SafeMonk to encrypt a subdirectory, then they went out of business and I started using Sookasa's free plan. Now Sookasa is retiring the free plan, and their cheapest paid plan is more expensive than a dedicated encrypted cloud storage solution.
However, I am kind of hooked into the Dropbox ecosystem in the sense that I'd have to convince everyone I share files with to also convert, if I left Dropbox completely. Any suggestions for a similar encryption solution to Sookasa that runs in a subdirectory of Dropbox, but is less expensive than the $100 or so per year Sookasa wants to start charging? I don't really need a lot of space, I'm cloud encrypting under one GB. For a while I tried TrueCrypt but it was too much of a pain to constantly load and unload the container to get it to sync back up to the cloud.
I have been using SpiderOak One for a few months now, and I love it. I Have 250GB of space for around $9/month. For everything that Dropbox has, SpiderOak does it better, with the exception of the mobile app. I guess we'll have to see where the mobile app goes, but I also understand that the mobile app has less features for security reasons.
BTW, here's some advice for anyone thinking about switching to SpiderOak One.
I normally would use "destination" and "target" interchangeably, so I'm a little confused. This probably makes more sense if I was using SpiderOak and had a mental picture of the GUI.
Yeah, sorry. I didn't explain that well.
You can create as many syncs as you want, and the sync can be added to as many devices as you want.
You start with a folder that has some files in it. First, you backup that folder to the cloud. Call this Folder A.
Then, on the device you want to sync that folder to, you create an empty folder for it, and backup that folder to the cloud. (Since it's empty it just backs up the metadata for now) Call this Folder B.
Then, when you create the sync, you select Folder A and sync it to Folder B.
After this, both folders A & B will stay in sync to each other. Repeat on a third device, and you have a three-way sync.