Re: Back-ups on CD/DVD with DAR (was Re: On backing up)

[pshute]

It depends how much data you have. Backing up to more than one disk, let alone dozens, is very tedious.



If you have a lot, it's probably suitable for incremental backups, but finding files on incremental backups is even more tedious, as you have to search them all.



As someone mentioned, 2TB external drives are getting very cheap now. That's the equivalent of hundreds of DVDs.



I use an external disk, keeping multiple backups on it. I ought to get a second one to rotate. I also make prints occasionally, but only of the very best.





--------------------------

Sent using BlackBerry



----- Original Message -----

From: Johnny <[hidden email]>

To: digiKam - Home Manage your photographs as a professional with the power of open source <[hidden email]>

Sent: Thu Feb 24 05:33:24 2011
Subject: [Digikam-users] Back-ups on CD/DVD with DAR (was Re: On backing up)



Another option is to use CD/DVD as a third option instead of foreign

servers. I am strongly considering DAR as a tool for this, which is a

Linux tool to make multiple DVDs, if necessary, and allow for

incremental backups and indexing.



However, I didn't get around to it; any opinions/experiences welcome, or

I'll report back if I implement it.



http://dar.linux.free.fr/

--

Johnny

_______________________________________________

Digikam-users mailing list

[hidden email]

https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Johnny]

Peter Shute <[hidden email]> writes:

> It depends how much data you have. Backing up to more than one disk,
> let alone dozens, is very tedious.
And this is what DAR is addressing.

> If you have a lot, it's probably suitable for incremental backups, but
> finding files on incremental backups is even more tedious, as you have
> to search them all.
And this is what DAR is addressing by providing an index to use.

> As someone mentioned, 2TB external drives are getting very cheap
> now. That's the equivalent of hundreds of DVDs.
Yes, you can have several hard drives, but redundancy and
diversification is different.

--
Johnny
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Am Mittwoch 23 Februar 2011, um 19:58:41 schrieb Peter Shute:

> It depends how much data you have. Backing up to more than one disk, let
> alone dozens, is very tedious.
>
> If you have a lot, it's probably suitable for incremental backups, but
> finding files on incremental backups is even more tedious, as you have to
> search them all.
>
> As someone mentioned, 2TB external drives are getting very cheap now.
> That's the equivalent of hundreds of DVDs.
>
> I use an external disk, keeping multiple backups on it. I ought to get a
> second one to rotate. I also make prints occasionally, but only of the
> very best.
>

I also prefer a fast local backup. I use a separate backup server that offers a large NFS network directory for each client.
The server stores all client data on a (linux-soft)mirror-RAID, so every file exists three times. (One on the client and two on the server.)
I use rsnapshot ( http://rsnapshot.org/ ) for an incremental backup based on hardlinks.
Everyday I make a backup (triggered by anacron) , rsnapshot creates hardlinks for every file that have not changed.  The changed files will be rsynced.
That way I have fast compact differential backup but the complete file structure.
Backups get rotated and I leave
1 of a day
7 of a week
4 of a month
12 of a year

These 24 backups fulfil all my needs.

Every client has a permanent bind mounted read-only access to the backup directory.

So every user can easily restore. Here the original dir:

 ls -l /home/schnebeck/Bilder/

-rw-r--r--  1 schnebeck schnebeck    15177 2009-08-29 02:16 06-05.pto
-rw-r--r--  1 schnebeck schnebeck    24833 2009-08-29 02:16 06-05.pto.mk
drwxr-xr-x  2 schnebeck schnebeck     4096 2009-01-25 18:28 C702
-rw-r--r--  1 schnebeck schnebeck 14310400 2011-02-02 23:41 digikam4.db
drwxr-xr-x 85 schnebeck schnebeck     4096 2011-01-29 15:46 EOS
drwxr-xr-x  3 schnebeck schnebeck     4096 2010-05-27 20:56 Import
drwxr-xr-x 12 schnebeck schnebeck     4096 2006-10-16 03:36 IXUS
drwxr-xr-x  3 schnebeck schnebeck     4096 2010-05-20 19:42 Nokia N900
drwxr-xr-x  2 schnebeck schnebeck     4096 2010-09-25 13:03 S95
-rw-r--r--  1 schnebeck schnebeck 62966784 2011-02-02 23:41 thumbnails-digikam.db

Here the read-only restore dir:

ls /restore/

daily.0  daily.1  daily.2  daily.3  daily.4  daily.5  daily.6  monthly.0  monthly.1  monthly.2  monthly.3  monthly.4  monthly.5  weekly.0  weekly.1  weekly.2  weekly.3

ls -l /restore/monthly.5/home/schnebeck/Bilder/

-rw-r--r-- 17 schnebeck schnebeck    15177 2009-08-29 02:16 06-05.pto
-rw-r--r-- 17 schnebeck schnebeck    24833 2009-08-29 02:16 06-05.pto.mk
drwxr-xr-x  2 schnebeck schnebeck     4096 2009-01-25 18:28 C702
-rw-r--r--  1 schnebeck schnebeck 10167296 2010-07-10 00:21 digikam4.db
drwxr-xr-x 81 schnebeck schnebeck     4096 2010-07-09 23:43 EOS
drwxr-xr-x  3 schnebeck schnebeck     4096 2010-05-27 20:56 Import
drwxr-xr-x 12 schnebeck schnebeck     4096 2006-10-16 03:36 IXUS
drwxr-xr-x  3 schnebeck schnebeck     4096 2010-05-20 19:42 Nokia N900
-rw-r--r--  1 schnebeck schnebeck 23864320 2010-07-10 00:21 thumbnails-digikam.db

(Ha! Meanwhile I bought another cam :-)

You see, its quite the same pathname on the server and easy to find.
It would be nice to write a dolphin plugin that aggregates all possible restore path of a file and shows these as a special restore file property.
GSoC anyone? ;-)

I can only recommend such a separate NAS Backup Server. Oh, I forgot something: for me every harddisk has a best before date. After 3.5 years every disk will be replaced without question. I note this
date on the disk and in my electronic calendar. Of course, the server will sent mails if there are S.M.A.R.T. or RAID errors.

HTH

  Thorsten

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 23/02/2011 22:09, Thorsten Schnebeck a écrit :

> I also prefer a fast local backup. I use a separate backup server
> that offers a large NFS network directory for each client.

I see this as a heavy risk. I already had a backup script that happen
to delete the backup and the original at the same time :-( - never
understood what happen exactly, but I ended with all the files, but
zero length (and no hardware problem, no notification, so I noticed
this at restore time!)

now I have usb hard drives (two, never connected at the same time) and
Blu-Ray disks*
of course nothing is never perfect...

jdd

--
http://www.dodin.net
http://www.dailymotion.com/video/xgxog7_clip-l-ombre-et-la-lumiere-3-bad-pigeons_music
http://www.youtube.com/watch?v=FGgv_ZFtV14
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Am Mittwoch 23 Februar 2011, um 23:16:15 schrieb jdd:

> Le 23/02/2011 22:09, Thorsten Schnebeck a écrit :
> > I also prefer a fast local backup. I use a separate backup server
> > that offers a large NFS network directory for each client.
>
> I see this as a heavy risk. I already had a backup script that happen
> to delete the backup and the original at the same time :-( - never
> understood what happen exactly, but I ended with all the files, but
> zero length (and no hardware problem, no notification, so I noticed
> this at restore time!)
>
> now I have usb hard drives (two, never connected at the same time) and
> Blu-Ray disks*
> of course nothing is never perfect...
>
> jdd

Hmm, never saw such an error. The only problem I have sometimes with NFS is
when using it on an ADSL line via OpenVPN and having an IP change on the
client. Now OpenVPN needs some time to recover the connection and NFS can
"hang". But in an Intranet NFS is very reliable I trust a network-harddisk
combination more than every optical media out there. I have also very positive
experiences using Linux ISCSI storage to backup Windows server in an intranet.

But if such an error you describe happens once, the rsync algorithm should
correct it next time. The only time I delete a backup is during rotation. And
if rsync failed or is unhappy cron sends me an email.
 
And to protect the RAID array I use one of these cheap 5min UPS devices +
burst & surge protection.

But as you said:
"nothing is never perfect..." and may I add "a backup is only valuable when
you have a simply working restore procedure"

Bye

  Thorsten
 
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[david-vj]

CONTENTS DELETED

The author has deleted this message.

[Francesco Scaglioni-2]

A second for rsync. Copy files from card to local folder, leaving them on card (rapid photo downloader) then rsync to 2 portable usb drives inseparate places the delete from card. When away from home copy from card direct to local folder and usb drive at the same time (again rapid photo downloader) keep cards until second rsync to spare drive then, and only then, delete from card.


---
(Apologies for brevity, top posting and poor citation - this email was sent from a mobile device)
---
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Le 24/02/2011 01:04, David Vincent-Jones a écrit :
> Using rsync (in synchronize mode) backups never need to be deleted and
> backups are extremely fast and convenient.

not so easy.

When, like me, you often test beta dvd, you have to sort out what have
to be backed and what don't have

even for photos (the subject, here), new cameras with more than 15
Mpix (and sometime more than 2000 shots a week end) makes it difficult
to make backups. gigabits flow NAS are not cheap. one need several...

no simple solution :-(

I Have a local copy of the original jpeg files (I don't use "raw"
photos, as I have no time to manage them), a copy managed by Digikam
(I don't hesitate there to remove all the bad shots, 75%).

Then I make a specal copy for Piwigo web gallery with reduced "high
definition" shots (only 3Mpix approx) for a reasonable transfer time
with my 1Mb upload bandwith.

I rarely use the full original definition. I make mostly Rock band
shots, in very difficult conditions (low light, fast moving dancers -
often 6500 iso with EOS 50D), thats why there are so many shots with
so bad average result

most of the images are never printed more than A4, so if ever I lose
some original definition ones, it's not very important.

jdd

--
http://www.dodin.net
http://www.dailymotion.com/video/xgxog7_clip-l-ombre-et-la-lumiere-3-bad-pigeons_music
http://www.youtube.com/watch?v=FGgv_ZFtV14
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users