This topic has been archived, and won't accept reply postings.
My partner needs to back up large quantities of data - up to 10 GB . She backs up to an external drive using Win8 file history - however this is not 100% reliable and appears to randomly not save stuff.
What do people use in terms of a. external drives - is there a more reliable way of doing this other than File history b. cloud storage
Thanks
What do people use in terms of a. external drives - is there a more reliable way of doing this other than File history b. cloud storage
Thanks
In reply to Chris H:
Crashplan is quite good as a cloud backup option and the software that they provide (free) is also quite useful for setting up local backups.
www.code42.com/crashplan
Crashplan is quite good as a cloud backup option and the software that they provide (free) is also quite useful for setting up local backups.
www.code42.com/crashplan
Post edited at 12:36
In reply to Luke90:
I've got a Synology NAS (DS215j) backing up to a second hard drive in the house, and then Crashplan as well in case the house burns down. Backing up about 300GB, so 10GB would be easily doable, works best with a decent internet connection. The NAS also stores all my videos/music for media streaming.
You can also install Crashplan direct to your Windows 8 PC and it will back up continuously whenever your computer is on.
Another option with a second Synology and a helpful friend is installing the second Synology at their house and backing up to that over your internet connection. Haven't done this myself, but it sounds easy to set up.
I've got a Synology NAS (DS215j) backing up to a second hard drive in the house, and then Crashplan as well in case the house burns down. Backing up about 300GB, so 10GB would be easily doable, works best with a decent internet connection. The NAS also stores all my videos/music for media streaming.
You can also install Crashplan direct to your Windows 8 PC and it will back up continuously whenever your computer is on.
Another option with a second Synology and a helpful friend is installing the second Synology at their house and backing up to that over your internet connection. Haven't done this myself, but it sounds easy to set up.
In reply to Chris H:
Fastest and cheapest way would be to just store the stuff on the external drive and mirror that to a usb stick. Cost of the stick would be approx £10. There's no need to store the data on the machine's hdd.
Fastest and cheapest way would be to just store the stuff on the external drive and mirror that to a usb stick. Cost of the stick would be approx £10. There's no need to store the data on the machine's hdd.
1
In reply to Chris H:
10GB is nothing these days; as ex0 says, it will easily fit onto a USB stick. The thing to consider is how often and how much of the data changes; that's more likely to determine the backup strategy you adopt.
For backing up my entire NAS, I use FreeFileSync, which will work in simple mirror (backup) mode, sync mode (keeping two ends up-to-date) and file version management mode. Like most sync programs, it only updates files that have changed. It doesn't do any silly mucking about with strange, compressed backup files; it just makes a copy of your working storage (so the backup copy can be used transparently). It can generate a batch job that you can run to do the sync and then shut down the computer. That could be incorporated into a daily working routine.
Then you'd probably want to hold a device remotely, in case of fire, etc. You could do weekly swap of local and remote sticks. the most you would then lose would be a week's work.
If the data is sensitive, you could use encrypted USB sticks.
Or use a secure cloud storage service.
10GB is nothing these days; as ex0 says, it will easily fit onto a USB stick. The thing to consider is how often and how much of the data changes; that's more likely to determine the backup strategy you adopt.
For backing up my entire NAS, I use FreeFileSync, which will work in simple mirror (backup) mode, sync mode (keeping two ends up-to-date) and file version management mode. Like most sync programs, it only updates files that have changed. It doesn't do any silly mucking about with strange, compressed backup files; it just makes a copy of your working storage (so the backup copy can be used transparently). It can generate a batch job that you can run to do the sync and then shut down the computer. That could be incorporated into a daily working routine.
Then you'd probably want to hold a device remotely, in case of fire, etc. You could do weekly swap of local and remote sticks. the most you would then lose would be a week's work.
If the data is sensitive, you could use encrypted USB sticks.
Or use a secure cloud storage service.
In reply to Chris H:
10Gb is nothing
1) Get a Microsoft Hotmail/outlook account
2) This comes with cloud storage called onedrive, I think you get 10Gb free but increments are pennies anyway
3) Set up a OneDrive folder on your hard disk - this is a folder where the contents will automatically be synchronized with your cloud OneDrive storage
4) Use this is as your data folder(s) and your data will automatically be backed up free of charge
If you want to be more sophisticated, you could copy your current data folder into your onedrive folder using robocopy, another Microsoft freebie that is fiddly to set up but you could arrange things so that if, say, your partner inadvertently deleted stuff on their local drive, the files would still be on OneDrive.
10Gb is nothing
1) Get a Microsoft Hotmail/outlook account
2) This comes with cloud storage called onedrive, I think you get 10Gb free but increments are pennies anyway
3) Set up a OneDrive folder on your hard disk - this is a folder where the contents will automatically be synchronized with your cloud OneDrive storage
4) Use this is as your data folder(s) and your data will automatically be backed up free of charge
If you want to be more sophisticated, you could copy your current data folder into your onedrive folder using robocopy, another Microsoft freebie that is fiddly to set up but you could arrange things so that if, say, your partner inadvertently deleted stuff on their local drive, the files would still be on OneDrive.
Post edited at 21:58
2
In reply to Chris H:
"Proper" backup procedure is to store data in three places; 2x local (i.e. the original files, and then backed up to a second hard-drive, or external hard-drive) and then 1x off-site (in case of fire/theft), commonly cloud-based.
It partly depends on what the data is, and how important it is, as to whether you need to go for off-site storage.
I use CrashPlan to backup over 30gb of photos (and another 30gb of music), on the basis that if we got burgled and the PC and external HDD got nicked, I would be devastated to lose my photos ("proper" photos, not phone shots).
Google Photos now offers unlimited photo storage too, as long as you're happy with them compressing them.
As others have said, 10GB isn't very much, and will fit on a decent memory stick - as long as that isn't left in the PC (kept at work for example), that should be fine
"Proper" backup procedure is to store data in three places; 2x local (i.e. the original files, and then backed up to a second hard-drive, or external hard-drive) and then 1x off-site (in case of fire/theft), commonly cloud-based.
It partly depends on what the data is, and how important it is, as to whether you need to go for off-site storage.
I use CrashPlan to backup over 30gb of photos (and another 30gb of music), on the basis that if we got burgled and the PC and external HDD got nicked, I would be devastated to lose my photos ("proper" photos, not phone shots).
Google Photos now offers unlimited photo storage too, as long as you're happy with them compressing them.
As others have said, 10GB isn't very much, and will fit on a decent memory stick - as long as that isn't left in the PC (kept at work for example), that should be fine
In reply to Durbs:
Superb! I have been umming and ahhring about what to do with my 100GB of photos. I note that Google Music allows you to upload 20,000 songs!
Edit: According to my plan, you only get 15GB for free and that is across Google photos and Drive.
> Google Photos now offers unlimited photo storage too, as long as you're happy with them compressing them.
Superb! I have been umming and ahhring about what to do with my 100GB of photos. I note that Google Music allows you to upload 20,000 songs!
Edit: According to my plan, you only get 15GB for free and that is across Google photos and Drive.
Post edited at 14:37
In reply to ByEek:
Depending on if you use them for other stuff Amazon Prime can be good for photo storage.
> Edit: According to my plan, you only get 15GB for free and that is across Google photos and Drive.
Depending on if you use them for other stuff Amazon Prime can be good for photo storage.
In reply to ByEek:
You get 15gb in Google Drive, but explicitly in Google Photos:
"Per your Google Photos settings, High Quality uploads are stored for free but Original quality uploads will consume your Drive storage."
> Edit: According to my plan, you only get 15GB for free and that is across Google photos and Drive.
You get 15gb in Google Drive, but explicitly in Google Photos:
"Per your Google Photos settings, High Quality uploads are stored for free but Original quality uploads will consume your Drive storage."
In reply to captain paranoia:
As a minor comment: mirror != backup
That is: what happens when your prime copy gets corrupted/messed about with - and then is happily mirrored to the 'backup' copy?
> For backing up my entire NAS, I use FreeFileSync, which will work in simple mirror (backup) mode, sync mode (keeping two ends up-to-date) and file version management mode. Like most sync programs, it only updates files that have changed ...
As a minor comment: mirror != backup
That is: what happens when your prime copy gets corrupted/messed about with - and then is happily mirrored to the 'backup' copy?
2
In reply to Rob Parsons:
Is that not the same as backup though? It is surely possible for corrupted files to be backed up too? The only difference is you hopefully have a number of incremental backups so can back track. But it all depends when you spot the corruption.
> That is: what happens when your prime copy gets corrupted/messed about with - and then is happily mirrored to the 'backup' copy?
Is that not the same as backup though? It is surely possible for corrupted files to be backed up too? The only difference is you hopefully have a number of incremental backups so can back track. But it all depends when you spot the corruption.
In reply to ByEek:
Related, but not the same. In a reasonable backup system, you might be able to retrieve a file in the state it was on any particular day over the last year or so, for example.
More generally, you can come a spectacular cropper on mirroring systems when, for example, you inadvertently delete a whole load of files in the prime copy, and then do the mirroring run - which applies the same mass deletions to your 'back up' copies. Oh dear ...
I'm not trying to complicate matters here; just flagging up the differences, and some potential gotchas. As usual, it all depends on exactly what you're trying to achieve, and how much trouble (and time, and expense) you're prepared to take over it.
> Is that not the same as backup though?
Related, but not the same. In a reasonable backup system, you might be able to retrieve a file in the state it was on any particular day over the last year or so, for example.
More generally, you can come a spectacular cropper on mirroring systems when, for example, you inadvertently delete a whole load of files in the prime copy, and then do the mirroring run - which applies the same mass deletions to your 'back up' copies. Oh dear ...
I'm not trying to complicate matters here; just flagging up the differences, and some potential gotchas. As usual, it all depends on exactly what you're trying to achieve, and how much trouble (and time, and expense) you're prepared to take over it.
Post edited at 16:05
2
In reply to Durbs:
Chuck the 100gb onto me.ga. Just register a couple of accounts with a couple different email addrs. You get 50gb/account free and the data is encrypted client side so you control the encryption keys.
Chuck the 100gb onto me.ga. Just register a couple of accounts with a couple different email addrs. You get 50gb/account free and the data is encrypted client side so you control the encryption keys.
Post edited at 16:46
In reply to Rob Parsons:
It depends on the algorithm used to determine change, and what is corrupted. I choose to use file date as the basis of change. So corruptions to my working copy that didn't change the date wouldn't cause the mirror backup to be modified. If you chose to use file content comparison, or length comparison, then corruption is likely to be propagated. But file content comparison is very expensive in terms of data transfer, and CPU time, so most people are unlikely to select it. Corruption of the body of the file is unlikely to modify the length, so, again, that wouldn't be propagated. Anything that corrupts the date or file length is likely to corrupt the file header in the directory, and thus likely to be detected during a delta scan (header CRC mismatch), and thus be reported.
There are many forms of backup.
The simplest form of backup is just a bulk copy from one disk to another, performed manually. The downside is that it will always copy everything, and will therefore take a long time. It will also not cope with deletions, folder moves, etc. Everything will be there, but it will get out of sync with the working drive. It also suffers from the corruption problem.
The next level up is to use a difference-detecting copier, say xcopy on PC. This will only copy changes, so works a lot quicker once the backup is established. It has the same problem of not deleting old/moved stuff, and the corruption problem.
Mirroring using a difference detection will sort out the problem of removal of the old versions, so that your backup tracks additions and deletions to your working copy. The downside is that if you delete a file from your working copy, or it get corrupted, those changes are copied to the backup. This is especially bad if it operates automatically, especially 'live mirroring', where if you delete a file on your working copy, it is automatically and immediately removed from the mirror. As such, it's probably best used under manual control, when the working copy is known to be 'stable'. I use this with a 'two tier backup', where I have a working copy, a routine backup mirror, and a long-term backup mirror that only gets refreshed infrequently (it's an offsite mirror). I suppose I could come up with a scheme to record changes to my working copy from the date of the last long-term backup, and hope that those don't get corrupted...
Syncing allows two copies to be 'working copies', with some means of arbitrating which of the copies to use if they are different; most commonly, keeping the latest copy (not always the best strategy, depending on the frequency backup is performed).
Reversionary backup, where you can restore files that have been deleted, or corrupted, is the most sophisticated, and requires a versioned backup store, which may be implemented as some version management system, storing file differences, or simply keeping multiple dated versions of each file, or by using multiple backup devices (e.g. backup tapes in old skool). The downside of this system is the increased file storage, or the use of compressed file differences (so it's not easy to use it as a working copy without doing a backup restore), or the need to maintain a strict backup device rotation scheme. It you use a device rotation, you only defer the corruption problem, unless you take measures to actively detect file corruption.
They're all forms of backup. You just have to pick one that is appropriate for your situation.
All of them are better than having no backup at all...
Do you have any suggestions for detecting file corruption on all files on a working copy? Running fsck before you backup, perhaps? I'm always open to suggested improvements in my routine.
> That is: what happens when your prime copy gets corrupted/messed about with - and then is happily mirrored to the 'backup' copy?
It depends on the algorithm used to determine change, and what is corrupted. I choose to use file date as the basis of change. So corruptions to my working copy that didn't change the date wouldn't cause the mirror backup to be modified. If you chose to use file content comparison, or length comparison, then corruption is likely to be propagated. But file content comparison is very expensive in terms of data transfer, and CPU time, so most people are unlikely to select it. Corruption of the body of the file is unlikely to modify the length, so, again, that wouldn't be propagated. Anything that corrupts the date or file length is likely to corrupt the file header in the directory, and thus likely to be detected during a delta scan (header CRC mismatch), and thus be reported.
> As a minor comment: mirror != backup
There are many forms of backup.
The simplest form of backup is just a bulk copy from one disk to another, performed manually. The downside is that it will always copy everything, and will therefore take a long time. It will also not cope with deletions, folder moves, etc. Everything will be there, but it will get out of sync with the working drive. It also suffers from the corruption problem.
The next level up is to use a difference-detecting copier, say xcopy on PC. This will only copy changes, so works a lot quicker once the backup is established. It has the same problem of not deleting old/moved stuff, and the corruption problem.
Mirroring using a difference detection will sort out the problem of removal of the old versions, so that your backup tracks additions and deletions to your working copy. The downside is that if you delete a file from your working copy, or it get corrupted, those changes are copied to the backup. This is especially bad if it operates automatically, especially 'live mirroring', where if you delete a file on your working copy, it is automatically and immediately removed from the mirror. As such, it's probably best used under manual control, when the working copy is known to be 'stable'. I use this with a 'two tier backup', where I have a working copy, a routine backup mirror, and a long-term backup mirror that only gets refreshed infrequently (it's an offsite mirror). I suppose I could come up with a scheme to record changes to my working copy from the date of the last long-term backup, and hope that those don't get corrupted...
Syncing allows two copies to be 'working copies', with some means of arbitrating which of the copies to use if they are different; most commonly, keeping the latest copy (not always the best strategy, depending on the frequency backup is performed).
Reversionary backup, where you can restore files that have been deleted, or corrupted, is the most sophisticated, and requires a versioned backup store, which may be implemented as some version management system, storing file differences, or simply keeping multiple dated versions of each file, or by using multiple backup devices (e.g. backup tapes in old skool). The downside of this system is the increased file storage, or the use of compressed file differences (so it's not easy to use it as a working copy without doing a backup restore), or the need to maintain a strict backup device rotation scheme. It you use a device rotation, you only defer the corruption problem, unless you take measures to actively detect file corruption.
They're all forms of backup. You just have to pick one that is appropriate for your situation.
All of them are better than having no backup at all...
Do you have any suggestions for detecting file corruption on all files on a working copy? Running fsck before you backup, perhaps? I'm always open to suggested improvements in my routine.
In reply to captain paranoia:
Thanks - a thoughtful post.
*Definitely* that! And, in the OP's case, it sounds like multiple mirrored copies will be a good solution: the overall size of the data involved is small.
That's an interesting question which raises more sophisticated issues than the ones I was flagging up; I do not have any glib answers, and the whole thing requires serious thought.
Clearly, if you have a file system which fails fsck, you have major problems which you'd hope would make themselves visible in some obvious way. But fsck checks probably (?) have nothing much to say about lower-level 'bit flip' style problems which might be insidiously spoiling your files. There, one might hope that better filesystems like ZFS might help us out - but I have not studied those enough to know. Until those are in use, then I imagine the only viable method - for static files, anyway - is to periodically take hashes, and compare those with known good values.
Nor have I really pondered (until now, anyway!) how the overall environment in this respect is changing with the move to solid-state drives, flash storage, etc. And yet another element here is the possibility of bad RAM which might cause invisible bit-flip errors in the process of copying files, when performing whatever back-up process you've arranged.
All that sounds alarmist doesn't it? That's not the intention, but do I think that these of issues are real, and not purely theoretical. Certainly worthy of more thought, anyway.
I feel I am derailing this thread, however! So apologies to the OP for that.
Thanks - a thoughtful post.
> All of them are better than having no backup at all ...
*Definitely* that! And, in the OP's case, it sounds like multiple mirrored copies will be a good solution: the overall size of the data involved is small.
> Do you have any suggestions for detecting file corruption on all files on a working copy?
That's an interesting question which raises more sophisticated issues than the ones I was flagging up; I do not have any glib answers, and the whole thing requires serious thought.
Clearly, if you have a file system which fails fsck, you have major problems which you'd hope would make themselves visible in some obvious way. But fsck checks probably (?) have nothing much to say about lower-level 'bit flip' style problems which might be insidiously spoiling your files. There, one might hope that better filesystems like ZFS might help us out - but I have not studied those enough to know. Until those are in use, then I imagine the only viable method - for static files, anyway - is to periodically take hashes, and compare those with known good values.
Nor have I really pondered (until now, anyway!) how the overall environment in this respect is changing with the move to solid-state drives, flash storage, etc. And yet another element here is the possibility of bad RAM which might cause invisible bit-flip errors in the process of copying files, when performing whatever back-up process you've arranged.
All that sounds alarmist doesn't it? That's not the intention, but do I think that these of issues are real, and not purely theoretical. Certainly worthy of more thought, anyway.
I feel I am derailing this thread, however! So apologies to the OP for that.
In reply to Chris H:
Chris surprised no one has asked this so far.
What does she want to protect against and how "automatic" does it need to be? This will help determine an appropriate backup strategy as there is no one size fits all when it comes to "backup" solutions.
What is it to protect against?
Hard disk failure?
Accidental deletions ?
Accidental updates / overwrites?
Etc....
Chris surprised no one has asked this so far.
What does she want to protect against and how "automatic" does it need to be? This will help determine an appropriate backup strategy as there is no one size fits all when it comes to "backup" solutions.
What is it to protect against?
Hard disk failure?
Accidental deletions ?
Accidental updates / overwrites?
Etc....
In reply to Chris H:
Thanks everyone for some really useful ideas -
Orgsm - mainly to protect against total data loss by HD failure etc and would ideally be totally automatic.
Thanks everyone for some really useful ideas -
Orgsm - mainly to protect against total data loss by HD failure etc and would ideally be totally automatic.
In reply to Chris H:
Then the solution is simple, don't store the data on the hdd. There's no reason to do so these days.
Then the solution is simple, don't store the data on the hdd. There's no reason to do so these days.
In reply to Removed User:
Flash drives fail, too. Or are you suggesting a cloud-based storage? Cloud providers disappear without trace, too... Cloud systems have been hacked, so the use of cloud storage depends on what it is that's being stored, and how sensitive it is... I seem to recall that to comply with European personal data protection law, you're not allowed (as a corporation) to store data on US servers...
Flash drives fail, too. Or are you suggesting a cloud-based storage? Cloud providers disappear without trace, too... Cloud systems have been hacked, so the use of cloud storage depends on what it is that's being stored, and how sensitive it is... I seem to recall that to comply with European personal data protection law, you're not allowed (as a corporation) to store data on US servers...
In reply to captain paranoia:
Only an idiot would store data on US servers in this day and age.
Flash drives do indeed fail, so do externals, but they are easy and cheap to replace and don't require any dicking about reinstalling software like you'd have to do if your HDD failed. Store the data on an external of some sort, access it from the computer, back it up to a second external. Any problems with the machine (eg extreme malware/rootkit infections, any sort of encryption based malware like cryptolocker which are becoming more and more popular because they work, corrupted windows files etc) and you can nuke it and reinstall windows and your data is fine because it's not stored on the hdd.
It's safer, quicker and more convenient to store your stuff on externals than it is to store it on your HDD.
Only an idiot would store data on US servers in this day and age.
Flash drives do indeed fail, so do externals, but they are easy and cheap to replace and don't require any dicking about reinstalling software like you'd have to do if your HDD failed. Store the data on an external of some sort, access it from the computer, back it up to a second external. Any problems with the machine (eg extreme malware/rootkit infections, any sort of encryption based malware like cryptolocker which are becoming more and more popular because they work, corrupted windows files etc) and you can nuke it and reinstall windows and your data is fine because it's not stored on the hdd.
It's safer, quicker and more convenient to store your stuff on externals than it is to store it on your HDD.
In reply to captain paranoia: It's improbable that Microsoft (OneDrive) will disappear without trace, and if they do we will all have significantly bigger worries than our stored photos.
They *might* become flaky, but that would be have to be at exactly the same time that the live date became unavailable - after all, backup is exactly that. A point some folk miss.
They *might* become flaky, but that would be have to be at exactly the same time that the live date became unavailable - after all, backup is exactly that. A point some folk miss.
In reply to Rob Exile Ward:
I was addressing ex0's suggestion of cloud storage to replace working and backup copies (or at least the meaning I inferred from his comment "don't store the data on the hdd").
> They *might* become flaky, but that would be have to be at exactly the same time that the live date became unavailable - after all, backup is exactly that.
I was addressing ex0's suggestion of cloud storage to replace working and backup copies (or at least the meaning I inferred from his comment "don't store the data on the hdd").
This topic has been archived, and won't accept reply postings.
Elsewhere on the site
Loading Notifications...