what do you use for backups?

[Luca Ferrari]

Hi all,
I'm keeping all my files on a removable hard drive, backupped on a
couple of other removable hard drives (rsync and similar).
One problem I'm facing is that while I work on the "master" I tend to
move, arrange, rename images. This makes backup more difficult,
because I've the same data in different paths between the master and
the replicas.
I'm investigating some more advanced tool, like bord and rsnapshot.
What do you use for your backups?

Just to mention: I'm using ext4 file systems and my data is around 290+ GB.

Thanks,
Luca
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[canodigi]

I use grsync.  There is an option to remove files that no longer exist
on the source.  This gets around the problem when you rename or move a file.


On 6/06/2016 4:36 PM, Luca Ferrari wrote:

> Hi all,
> I'm keeping all my files on a removable hard drive, backupped on a
> couple of other removable hard drives (rsync and similar).
> One problem I'm facing is that while I work on the "master" I tend to
> move, arrange, rename images. This makes backup more difficult,
> because I've the same data in different paths between the master and
> the replicas.
> I'm investigating some more advanced tool, like bord and rsnapshot.
> What do you use for your backups?
>
> Just to mention: I'm using ext4 file systems and my data is around 290+ GB.
>
> Thanks,
> Luca
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Simon Cropper-4]

Hey Luca,

I haven't looked for a while but checked just before responding -- I have 19668 files occupying 284.8 GB.

I regularly backup without any problems.

First, think of you database as images+metadata (in file or as an associated XMP) and a separate database that consolidates this data into a useful form.

Second, remember the database can be readily recreated from the images+metadata. Throughout the time I have been using digikam I have rebuild my database probably 10 times.

So, assuming all your images+metadata are in one directory you can use any operating-system-based backup solution.

Personally I use Ubuntu 14.04 LTS as my operating system and rsync as my backup solution.

My rsync commands are all stored in a bash script, which also starts/stops/extracts data from other computers that the family use (Windows 7 and various linux flavours).

I mirror all my copies so after I am finished a backup I have 3 independent and physically separate copies.

I hope this feedback helps.

On 06/06/16 16:36, Luca Ferrari wrote:

Hi all,
I'm keeping all my files on a removable hard drive, backupped on a
couple of other removable hard drives (rsync and similar).
One problem I'm facing is that while I work on the "master" I tend to
move, arrange, rename images. This makes backup more difficult,
because I've the same data in different paths between the master and
the replicas.
I'm investigating some more advanced tool, like bord and rsnapshot.
What do you use for your backups?

Just to mention: I'm using ext4 file systems and my data is around 290+ GB.

Thanks,
Luca
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

Grsync too :)

I also use external HD with different sizes for backup. Since I have around 800 Gb of data, I only back up the jpg in the smaller ones (e.g. 500 Gb), while the bigger ones (1Tb) also get the RAW files.

Grsync is very comfortable for this, since you can save as many configurations as you like, so I have one for each external hard drive. And I doubt there is something more efficient than rsync out there anyway.

Regards,

Nicolas

On Mon, Jun 6, 2016 at 8:34 AM, . <[hidden email]> wrote:

I use grsync.  There is an option to remove files that no longer exist on the source.  This gets around the problem when you rename or move a file.

On 6/06/2016 4:36 PM, Luca Ferrari wrote:

Hi all,
I'm keeping all my files on a removable hard drive, backupped on a
couple of other removable hard drives (rsync and similar).
One problem I'm facing is that while I work on the "master" I tend to
move, arrange, rename images. This makes backup more difficult,
because I've the same data in different paths between the master and
the replicas.
I'm investigating some more advanced tool, like bord and rsnapshot.
What do you use for your backups?

Just to mention: I'm using ext4 file systems and my data is around 290+ GB.

Thanks,
Luca
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Daniel Bauer-2]

Am 06.06.2016 um 08:36 schrieb Luca Ferrari:

> Hi all,
> I'm keeping all my files on a removable hard drive, backupped on a
> couple of other removable hard drives (rsync and similar).
> One problem I'm facing is that while I work on the "master" I tend to
> move, arrange, rename images. This makes backup more difficult,
> because I've the same data in different paths between the master and
> the replicas.
> I'm investigating some more advanced tool, like bord and rsnapshot.
> What do you use for your backups?
>
> Just to mention: I'm using ext4 file systems and my data is around 290+ GB.
>
> Thanks,
> Luca

Hi Luca,

I use simple rsync and alternate the external disks, so I have a least a
last and a before-last copy.

I have separate backups (twice, stored in different places) for
raw-files, as they are not changing, only new ones added.

Yes, there's a lot of changes of directories, images moving around, and
yes, this leads to a lot of deleting and writing new of same files on
the backup, just because I move the directory. But on the other hand,
this only affects the touched ones, while the great rest stays in peace.

Whenever I go to Switzerland I prepare another set of back-ups to store
it there.

My external hd's are encrypted, so if somebody steals it he's got
nothing more than the hardware...

Maybe my backup strategy is paranoid but I lived several hard disk
crashes (although long time ago, touch on wood...) and so I prefer a
little more work over painful loss. Anyway, the backing-up time gives me
just another reason to go to the beach meanwhile :-)

Daniel


--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Daniel Bauer-2]

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
 > Dropbox! Dropbox's default behavior is to store a copy of every file on
 > every computer you back it up to. Plus there's the copy stored on their
 > cloud. PLUS, if you pay for the extra, they do unlimited file
 > versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year
and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access
to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts
- it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never
leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a
short time storage of smaller amounts of data. But it is not all all
suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

At home, i use a NAS connected to 100Mb ethernet through power supply. An rsync script synchronize at 4:00AM a copy from host computer to the NAS using a smb mount (300Gb). You just need to take a care about the SMB mount options for file naming (UTF8, long file name, rights. etc...) The NAS can be acceded through WIKI by a MAC and a Windows to show images through FS. The NAS support also UPNP, so you can display image with a tablets or a phone.

Gilles Caulier

2016-06-06 18:39 GMT+02:00 Daniel Bauer <[hidden email]>:

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
> Dropbox! Dropbox's default behavior is to store a copy of every file on
> every computer you back it up to. Plus there's the copy stored on their
> cloud. PLUS, if you pay for the extra, they do unlimited file
> versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts - it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a short time storage of smaller amounts of data. But it is not all all suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Michael Havens]

I rsync /home/Documents to a thumb drive and /home/Pictures to another

On Mon, Jun 6, 2016 at 12:52 PM, Gilles Caulier <[hidden email]> wrote:

At home, i use a NAS connected to 100Mb ethernet through power supply. An rsync script synchronize at 4:00AM a copy from host computer to the NAS using a smb mount (300Gb). You just need to take a care about the SMB mount options for file naming (UTF8, long file name, rights. etc...) The NAS can be acceded through WIKI by a MAC and a Windows to show images through FS. The NAS support also UPNP, so you can display image with a tablets or a phone.

Gilles Caulier

2016-06-06 18:39 GMT+02:00 Daniel Bauer <[hidden email]>:

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
> Dropbox! Dropbox's default behavior is to store a copy of every file on
> every computer you back it up to. Plus there's the copy stored on their
> cloud. PLUS, if you pay for the extra, they do unlimited file
> versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts - it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a short time storage of smaller amounts of data. But it is not all all suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

--

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Wilkins, Vern W]

For my web photogallery, database, documents, etc., I rsync all that to a thumb drive kept at work, and also to an external drive at home.  For my raw photos, I’m only able to rsync to an external drive at home.  I have over 1TB of raw images which creates a real problem for free offsite backup. 

 

Also, for those who rsync, be careful.  A while back one of my drives went and I realized that the rsync backups had actually copied corrupted files to the backup.  I also had a case where files were deleted on backup before they were successfully copied (they had been moved on the source).  I fixed the second problem with a different set of rsync options, but I don’t know how the first problem could be solved without some sort of validation.

 

Vern

From: Digikam-users [mailto:[hidden email]] On Behalf Of Michael
Sent: Monday, June 6, 2016 1:39 PM
To: digiKam - Home Manage your photographs as a professional with the power of open source <[hidden email]>
Subject: Re: [Digikam-users] what do you use for backups?

 

I rsync /home/Documents to a thumb drive and /home/Pictures to another

 

On Mon, Jun 6, 2016 at 12:52 PM, Gilles Caulier <[hidden email]> wrote:

At home, i use a NAS connected to 100Mb ethernet through power supply. An rsync script synchronize at 4:00AM a copy from host computer to the NAS using a smb mount (300Gb). You just need to take a care about the SMB mount options for file naming (UTF8, long file name, rights. etc...) The NAS can be acceded through WIKI by a MAC and a Windows to show images through FS. The NAS support also UPNP, so you can display image with a tablets or a phone.

 

Gilles Caulier

 

2016-06-06 18:39 GMT+02:00 Daniel Bauer <[hidden email]>:

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
> Dropbox! Dropbox's default behavior is to store a copy of every file on
> every computer you back it up to. Plus there's the copy stored on their
> cloud. PLUS, if you pay for the extra, they do unlimited file
> versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts - it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a short time storage of smaller amounts of data. But it is not all all suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

--

:-)~MIKE~(-:

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Kamf]

I transfer photos and videos from cameras to a laptop where I perform an initial sorting which involves rating, tags and occasionally some edit of the photo itself. The collection on the laptop has a manual backup with rsync to my server (which I run whenever I add new pictures or makes changes to the existing).

Once finished with sorting, I move the sorted pictures to my server (so I can access them from several “clients” at home). The server, where the photo collection takes about 1.1 TB, runs a daily backup with rsnapshot at a remote server located at a friend.
 
I consider rsnapshot, which basically is an automated way to use rsync, quite feasible. Once setup, it runs daily without any need for me to get involved.
It also saves weekly and monthly snaspshots, i.e. I can find accidently deleted files (time machine).

There are several advantages with the intermediate storage at the laptop. I can bring it and work on the sorting out of home, e.g. on vacation when the camera is heavily used. But as rsnapshot saves old snapshots, I want to be quite finished with the photos when they reach the server.  Otherwise I might end up with two versions of every picture, one without tags and rating and one with.

BR

Anders

2016-06-06 20:20 GMT+02:00 Wilkins, Vern W <[hidden email]>:

For my web photogallery, database, documents, etc., I rsync all that to a thumb drive kept at work, and also to an external drive at home.  For my raw photos, I’m only able to rsync to an external drive at home.  I have over 1TB of raw images which creates a real problem for free offsite backup. 

 

Also, for those who rsync, be careful.  A while back one of my drives went and I realized that the rsync backups had actually copied corrupted files to the backup.  I also had a case where files were deleted on backup before they were successfully copied (they had been moved on the source).  I fixed the second problem with a different set of rsync options, but I don’t know how the first problem could be solved without some sort of validation.

 

Vern

From: Digikam-users [mailto:[hidden email]] On Behalf Of Michael
Sent: Monday, June 6, 2016 1:39 PM
To: digiKam - Home Manage your photographs as a professional with the power of open source <[hidden email]>
Subject: Re: [Digikam-users] what do you use for backups?

 

I rsync /home/Documents to a thumb drive and /home/Pictures to another

 

On Mon, Jun 6, 2016 at 12:52 PM, Gilles Caulier <[hidden email]> wrote:

At home, i use a NAS connected to 100Mb ethernet through power supply. An rsync script synchronize at 4:00AM a copy from host computer to the NAS using a smb mount (300Gb). You just need to take a care about the SMB mount options for file naming (UTF8, long file name, rights. etc...) The NAS can be acceded through WIKI by a MAC and a Windows to show images through FS. The NAS support also UPNP, so you can display image with a tablets or a phone.

 

Gilles Caulier

 

2016-06-06 18:39 GMT+02:00 Daniel Bauer <[hidden email]>:

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
> Dropbox! Dropbox's default behavior is to store a copy of every file on
> every computer you back it up to. Plus there's the copy stored on their
> cloud. PLUS, if you pay for the extra, they do unlimited file
> versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts - it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a short time storage of smaller amounts of data. But it is not all all suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

--

:-)~MIKE~(-:

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Simon Cropper-4]

> I don’t know how the first problem could be solved without some sort of validation.

I try and handle this problem by creating an archive at set times. A 6 month or 12 month backup if you will. I also capture the output from rsync and dump them into logs. I  then use  grep to extract changes and problems and  send them to me via email at the end of the backup. If for some reason I have changes that I can't account for I can always go back to the archive.

For more recent files maybe not assimilated into the archive, I have a rolling daily backup that I can go to if something go awry.

Lots of versions, I know, but disk space is cheap and now that the process is relatively automated I don't really encounter problems any more.

On 07/06/16 04:20, Wilkins, Vern W wrote:

For my web photogallery, database, documents, etc., I rsync all that to a thumb drive kept at work, and also to an external drive at home.  For my raw photos, I’m only able to rsync to an external drive at home.  I have over 1TB of raw images which creates a real problem for free offsite backup. 

 

Also, for those who rsync, be careful.  A while back one of my drives went and I realized that the rsync backups had actually copied corrupted files to the backup.  I also had a case where files were deleted on backup before they were successfully copied (they had been moved on the source).  I fixed the second problem with a different set of rsync options, but I don’t know how the first problem could be solved without some sort of validation.

 

Vern

From: Digikam-users [[hidden email]] On Behalf Of Michael
Sent: Monday, June 6, 2016 1:39 PM
To: digiKam - Home Manage your photographs as a professional with the power of open source [hidden email]
Subject: Re: [Digikam-users] what do you use for backups?

 

I rsync /home/Documents to a thumb drive and /home/Pictures to another

 

On Mon, Jun 6, 2016 at 12:52 PM, Gilles Caulier <[hidden email]> wrote:

At home, i use a NAS connected to 100Mb ethernet through power supply. An rsync script synchronize at 4:00AM a copy from host computer to the NAS using a smb mount (300Gb). You just need to take a care about the SMB mount options for file naming (UTF8, long file name, rights. etc...) The NAS can be acceded through WIKI by a MAC and a Windows to show images through FS. The NAS support also UPNP, so you can display image with a tablets or a phone.

 

Gilles Caulier

 

2016-06-06 18:39 GMT+02:00 Daniel Bauer <[hidden email]>:

Am 06.06.2016 um 16:37 schrieb Michael Fierro:
> Dropbox! Dropbox's default behavior is to store a copy of every file on
> every computer you back it up to. Plus there's the copy stored on their
> cloud. PLUS, if you pay for the extra, they do unlimited file
> versioning. It works incredibly well.


This is fantastic if you only take a 64 GB of photos every half a year and use the time in between for the upload :-)

It is also fantastic if you don't care, that at least dropbox has access to your files.

Then it's great that your backup lasts exactly as long as dropbox lasts - it wouldn't be the first such service that disappears.

Even if upload times would increase by 100, I would never, never, never leave my files somewhere that is completely out of my control.

Dropbox and the like for sure is cool to share some MB of files or for a short time storage of smaller amounts of data. But it is not all all suitable for a save backup.

Daniel
--
Daniel Bauer photographer Basel Barcelona
http://www.daniel-bauer.com
room in Barcelona: https://www.airbnb.es/rooms/2416137

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

 

--

:-)~MIKE~(-:

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Luca Ferrari]

Thank you all for your replies.
First of all I have to say that cloud is not an option for me because
(i) I don't want anyone else have my files and (ii) I don't have an
home internet access with a decent speed to allow me to
upload/download continuously.
I don't encrypt my photos, mainly because I want my wife to be able to
see them without having to scream each time (I encrypt all the other
important stuff).
I also often do an intermediate stage at the home computer, so I have
another "manual" backup at least of the last period.

The problem I'm facing is to ensure that (i) the backup is ok and (ii)
the backup is in sync.
A few months ago one album on my master disk was corrupted. This did
not affected the backup, beacuse such album was "stable" and no more
backupped, but restoring from the backup revealed some duplicated
files due to a massive renaming of images.
While I can use the delete option of rsync (and similar), how can I
trust that the "to-be-deleted" image is really renamed and readable?

So far I'm also thinking about managing my albums via some hash-based
repository (fossil, git), just to catch up the rename process.

Luca
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[bernhard]

there is quite cool feature in rsync that allows you to sync your backup in a directory on you backup medium. Lets say you want to backup /pictures on your pc in directory /week01 on your usb disk. standard procedure is to do the next rsync against  /week01 too which replace changed files.

another option is to tell rsync to do the backup of /pictures to lets say /week02 but use /week01 as base. in that case rsync compares your files in /pictures against your files in /week01. If a file is not changed it created an hard link in /week02 pointing to the same file in /week01. if a file is deleted it will not show up in /week02 (but still in /week01). if a file has changed it is copied to /week02 while you can access the old version in /week01. So you can have several versions of your /pictures directory but you need only disk size from /pictures on your pc plus the space for files that change on every extra backup you do. for that you usb drive need to have a linux filesystem like ext4 to support hardlinks.

regards, bernhard

 

Am 2016-06-07 8:35, schrieb Luca Ferrari:

Thank you all for your replies.
First of all I have to say that cloud is not an option for me because
(i) I don't want anyone else have my files and (ii) I don't have an
home internet access with a decent speed to allow me to
upload/download continuously.
I don't encrypt my photos, mainly because I want my wife to be able to
see them without having to scream each time (I encrypt all the other
important stuff).
I also often do an intermediate stage at the home computer, so I have
another "manual" backup at least of the last period.

The problem I'm facing is to ensure that (i) the backup is ok and (ii)
the backup is in sync.
A few months ago one album on my master disk was corrupted. This did
not affected the backup, beacuse such album was "stable" and no more
backupped, but restoring from the backup revealed some duplicated
files due to a massive renaming of images.
While I can use the delete option of rsync (and similar), how can I
trust that the "to-be-deleted" image is really renamed and readable?

So far I'm also thinking about managing my albums via some hash-based
repository (fossil, git), just to catch up the rename process.

Luca
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Chris Green]

bernhard <[hidden email]> wrote:

> [-- multipart/alternative, encoding 7bit, 125 lines --]
>
>     [-- text/plain, encoding 7bit, charset: UTF-8, 58 lines --]
>
>  
>
> there is quite cool feature in rsync that allows you to sync your backup
> in a directory on you backup medium. Lets say you want to backup
> /pictures on your pc in directory /week01 on your usb disk. standard
> procedure is to do the next rsync against /week01 too which replace
> changed files.
>
> another option is to tell rsync to do the backup of /pictures to lets
> say /week02 but use /week01 as base. in that case rsync compares your
> files in /pictures against your files in /week01. If a file is not
> changed it created an hard link in /week02 pointing to the same file in
> /week01. if a file is deleted it will not show up in /week02 (but still
> in /week01). if a file has changed it is copied to /week02 while you can
> access the old version in /week01. So you can have several versions of
> your /pictures directory but you need only disk size from /pictures on
> your pc plus the space for files that change on every extra backup you
> do. for that you usb drive need to have a linux filesystem like ext4 to
> support hardlinks.
>

Yes, I use this rsync 'hard link' ability with a self-written Python
script to do incremental backups.  On my main desktop system I do
hourly and seven days of daily backups, these go onto a different
drive so at least will protect me from drive failure.  

The I do daily, weekly, monthly and yearly incremental backups to a
remote (well some hundred metres or so) system as well.

Using the rsync hard-link option means that only changes and new files
occupy space.

--
Chris Green
·

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Anders Kamf]

Hard links is what rsnapshot use as well to do incremental backup.

2016-06-07 12:57 GMT+02:00 Chris Green <[hidden email]>:

bernhard <[hidden email]> wrote:
> [-- multipart/alternative, encoding 7bit, 125 lines --]
>
>     [-- text/plain, encoding 7bit, charset: UTF-8, 58 lines --]
>
>
>
> there is quite cool feature in rsync that allows you to sync your backup
> in a directory on you backup medium. Lets say you want to backup
> /pictures on your pc in directory /week01 on your usb disk. standard
> procedure is to do the next rsync against /week01 too which replace
> changed files.
>
> another option is to tell rsync to do the backup of /pictures to lets
> say /week02 but use /week01 as base. in that case rsync compares your
> files in /pictures against your files in /week01. If a file is not
> changed it created an hard link in /week02 pointing to the same file in
> /week01. if a file is deleted it will not show up in /week02 (but still
> in /week01). if a file has changed it is copied to /week02 while you can
> access the old version in /week01. So you can have several versions of
> your /pictures directory but you need only disk size from /pictures on
> your pc plus the space for files that change on every extra backup you
> do. for that you usb drive need to have a linux filesystem like ext4 to
> support hardlinks.
>
Yes, I use this rsync 'hard link' ability with a self-written Python
script to do incremental backups.  On my main desktop system I do
hourly and seven days of daily backups, these go onto a different
drive so at least will protect me from drive failure.

The I do daily, weekly, monthly and yearly incremental backups to a
remote (well some hundred metres or so) system as well.

Using the rsync hard-link option means that only changes and new files
occupy space.

--
Chris Green
·

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Chris Green]

Anders Kamf <[hidden email]> wrote:

> [-- multipart/alternative, encoding 7bit, 130 lines --]
>
>     [-- text/plain, encoding quoted-printable, charset: UTF-8, 52 lines --]
>
> 2016-06-07 12:57 GMT+02:00 Chris Green <[hidden email]>:
>
> > bernhard <[hidden email]> wrote:
> > > [-- multipart/alternative, encoding 7bit, 125 lines --]
> > >
> > >     [-- text/plain, encoding 7bit, charset: UTF-8, 58 lines --]
> > >
> > >
> > >
> > > there is quite cool feature in rsync that allows you to sync your backup
> > > in a directory on you backup medium. Lets say you want to backup
> > > /pictures on your pc in directory /week01 on your usb disk. standard
> > > procedure is to do the next rsync against /week01 too which replace
> > > changed files.
> > >
> > > another option is to tell rsync to do the backup of /pictures to lets
> > > say /week02 but use /week01 as base. in that case rsync compares your
> > > files in /pictures against your files in /week01. If a file is not
> > > changed it created an hard link in /week02 pointing to the same file in
> > > /week01. if a file is deleted it will not show up in /week02 (but still
> > > in /week01). if a file has changed it is copied to /week02 while you can
> > > access the old version in /week01. So you can have several versions of
> > > your /pictures directory but you need only disk size from /pictures on
> > > your pc plus the space for files that change on every extra backup you
> > > do. for that you usb drive need to have a linux filesystem like ext4 to
> > > support hardlinks.
> > >
> > Yes, I use this rsync 'hard link' ability with a self-written Python
> > script to do incremental backups.  On my main desktop system I do
> > hourly and seven days of daily backups, these go onto a different
> > drive so at least will protect me from drive failure.
> >
> > The I do daily, weekly, monthly and yearly incremental backups to a
> > remote (well some hundred metres or so) system as well.
> >
> > Using the rsync hard-link option means that only changes and new files
> > occupy space.
> >
> Hard links is what rsnapshot use as well to do incremental backup.
>

Yes, I know, I used rsnapshot for a while but decided that writing my
own script (which is much less complex than rsnapshot) would give me
more exactly what I wanted.

My incremental backup script is only a hundred lines or so of Python
code.

--
Chris Green
·

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Erick Moreno]

I'm very happy with the back-in-time for my backups.

I configure as many profiles as I want, I can configure how many versions of my file system it will keep, how much space I want before it erase the old versions. Back-in-time uses rsync (obviously) ans hard links to make incremental backups. To me, the best solution.

http://backintime.le-web.org/

[]`s

On Wed, Jun 8, 2016 at 7:29 AM, Chris Green <[hidden email]> wrote:

Anders Kamf <[hidden email]> wrote:
> [-- multipart/alternative, encoding 7bit, 130 lines --]
>
>     [-- text/plain, encoding quoted-printable, charset: UTF-8, 52 lines --]

>
> 2016-06-07 12:57 GMT+02:00 Chris Green <[hidden email]>:
>
> > bernhard <[hidden email]> wrote:
> > > [-- multipart/alternative, encoding 7bit, 125 lines --]
> > >
> > >     [-- text/plain, encoding 7bit, charset: UTF-8, 58 lines --]
> > >
> > >
> > >
> > > there is quite cool feature in rsync that allows you to sync your backup
> > > in a directory on you backup medium. Lets say you want to backup
> > > /pictures on your pc in directory /week01 on your usb disk. standard
> > > procedure is to do the next rsync against /week01 too which replace
> > > changed files.
> > >
> > > another option is to tell rsync to do the backup of /pictures to lets
> > > say /week02 but use /week01 as base. in that case rsync compares your
> > > files in /pictures against your files in /week01. If a file is not
> > > changed it created an hard link in /week02 pointing to the same file in
> > > /week01. if a file is deleted it will not show up in /week02 (but still
> > > in /week01). if a file has changed it is copied to /week02 while you can
> > > access the old version in /week01. So you can have several versions of
> > > your /pictures directory but you need only disk size from /pictures on
> > > your pc plus the space for files that change on every extra backup you
> > > do. for that you usb drive need to have a linux filesystem like ext4 to
> > > support hardlinks.
> > >
> > Yes, I use this rsync 'hard link' ability with a self-written Python
> > script to do incremental backups.  On my main desktop system I do
> > hourly and seven days of daily backups, these go onto a different
> > drive so at least will protect me from drive failure.
> >
> > The I do daily, weekly, monthly and yearly incremental backups to a
> > remote (well some hundred metres or so) system as well.
> >
> > Using the rsync hard-link option means that only changes and new files
> > occupy space.
> >
> Hard links is what rsnapshot use as well to do incremental backup.
>

Yes, I know, I used rsnapshot for a while but decided that writing my
own script (which is much less complex than rsnapshot) would give me
more exactly what I wanted.

My incremental backup script is only a hundred lines or so of Python
code.

--
Chris Green
·

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

--

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Peter Albrecht]

Hi!

I've been using Unison (https://www.cis.upenn.edu/~bcpierce/unison/) for several
years now and am quite happy with it.

Main differences to rsync:
 - unison keeps checksums of the files to sync, so in the next run unison can
   detect whether files have changed or not
 - after detecting this change-states, the list of changed objects is presented
   to the user. And the user decides, what to do. (In most cases: Copy file to
   backup)

This way I can confirm: Yes, I have changed this file on purpose, copy it to the
backup. This is some work, but for me, it is worth the effort, since it will
protect my files from unintentional changes and encryption trojans, having
changed a huge amount of old files, get revealed before this changes are written
to my backups.

There is also an option to tell unison to always scan the backup files content
and compare those to the checksums from the last run. (Default is to only
compare timestamps on Linux.) This way, unison checks wether my backup HDD is
still completly readable and gives me good files.

Drawback: There is no versioning in my setup. I only have one state in time.

So far my two cents.

Regards,
        Peter Albrecht

On 06.06.2016 08:36, Luca Ferrari wrote:

> Hi all,
> I'm keeping all my files on a removable hard drive, backupped on a
> couple of other removable hard drives (rsync and similar).
> One problem I'm facing is that while I work on the "master" I tend to
> move, arrange, rename images. This makes backup more difficult,
> because I've the same data in different paths between the master and
> the replicas.
> I'm investigating some more advanced tool, like bord and rsnapshot.
> What do you use for your backups?
>
> Just to mention: I'm using ext4 file systems and my data is around 290+ GB.
>
> Thanks,
> Luca
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users