[digiKam-users] --> History of a Migration from Mac Photos to Digikam

[Alex Antão]

Hi,

   Sorry for the huge email, but I think it's important to write my history here, in case I can help somebody on the process I passed.

   I'll try to write as less as I can... It's no easy... It's not trivial...

   So, here it is:

   INTRODUCTION (ignore if you want).

   Since I bought my MacBook, on late 2011, I started using Mac Photos.app (iPhoto in the begining) to store my photos. All photos I already had taken until that day I inserted there. 

   And I kept using it for a long time, but since the time has passed, my library was growing fast and my MacBook started to be outdated and slower each update and each adition to my library.

   Photos.app is a excelent program to use, I like it a lot, despite it lacks some good features, and I never had any kind of problem with it (at least I knew about). The only big problem I had was on the change from iPhoto to Photos.app, where I had some photos missing on my library, but present on disk. This was easly fixed reimporting them to the App.

   Recently, I have a Library with about 800Gb of photos and vídeos, about 110.000 items total. But, since the last MACOS update (High Sierra, the last my MacBook supports), the Photos.app became so slow, that a simple deletion as taking so long that I started thinking about alternatives to use.

   But 2 problems made me finally decide that was the time to move on:

            1) Since Mojave, Photos.app will not be updated anymore on my MacBook, it will stay forever on version 3.0 (it's already in 4.0 on Mojave). 

            2) When I tried to export some photos (about 10) I need to a particular use, I got an error on 4 of them ! I discovered that those photos were missing on disc, but present on database (a bug?) Happly I had those photos stored somewhere else (I have my backups on TimeCapsule and also GooglePhotos) and restored them, but this problem stayed on my head and I decided to take some urgent actions.

   THE MIGRATION

      The, I decided to migrate. Looked and tried a lot of options until I found Digikam, and this was the chosen one.

      Of course I would not just stop using one and star using the other, I had to get all my photos from Photos and send to Digikam, independently. 

      Got a 1TB harddrive and started looking for a way to get ALL structure I had on Photos.app (I had a lot of albums, tags, etc...). Again, I tried a lof of ways to export them, but the best choice I found was these scripts written by Jacob on GitHub: https://github.com/orangeturtle739/photos-export

      With them, I could export ALL photos I had to my harddrive, all TAGS, location and other information was stored inside EXIF on each photo, and the album information was not lost.

Digikam could get all information and the all album was preservated. All process took me 4 days to export my library.

      But I was not satisfied, the scripts had some bugs (which made me loose some fotos, or merge albums with the same name) and some information, like folders, was missing. So, I started learning more python and took those scripts to improve them. To improve, I coded a lot of features, like PRESERVE ALL FOLDER STRUCTURE and a lot of things to make the export exactly like I had in Photos.app.

     I had to export my whole library 4 times before all scripts ran perfectly and do not loose any photo and also have my library almost identical to my Photos.app, considering that it is very dificult to compare, because Photos takes care of all file management for us. My next step now will be some adjustment, since some photos does not have dates in EXIF (I don't know yet how they are in Photos.app), but I need to adjust all. I also made another python script on my github to help organize them. It gets the EXIF CreationDate (or other EXIF date field) and move the photos to a YEAR folder (customisable). I also may want to move inside a MONTH/DAY folder (optional).

     All scripts are already updated in GitHub. 

     For me, these are the most complete scripts if you are trying to leave Photos and go to Digikam.

     BIG PROBLEM WITH MISSING DISK

     But I still had a big problem that was bothering me: If I had more than those 4 photos missing on disk ?

     To answer that, I made a simple python script that gets all media I have on Photos Database and checks for it's corresponding file on disk.

      I almost had a heart attack, my script returned 720 missing files on disk !!! OMG....

     So, to try to solve this problem, I made 3 scripts:

          1) The check script generates a CSV file containing the media UUID found in Photos.app and it's corresponding Path

          2) An Applescript that takes this file and checks witha a selection in Photos.app. It creates a new Album there and adds all missing photos to it. This made things easier to me to keep track on photos I need to recover. I downloaded or recovered the photos in that album and copied them to a external folder (to be processed by the following script). This took me time, but it was necessary necessary.

          3) The third and final script, takes the folder I downloaded all missing photos and copies each one according to it's path registered in the CSV file I generated on first step.

 ENDING THE MIGRATION

     After all that work, I could have all files restored phisically on Photos.app. 

     I bought a 1Tb external HD and exported again ALL Photos.app database, using all steps/scripts above. This HD will be placed on my TimeCapsule and shared on my personal network, where I will have access anywhere I want. I just need to think about how to backup it too.

     All this work took a lot of time and effort, and I hope it can help somebody with the same problem as I had. 

GITHUB:

      All my scripts are avaible to download and test om my GitHub: https://github.com/alexantao/apple-photos-scripts

      The script made by Jacob that I updated to export the Photos database are on GitHub: https://github.com/orangeturtle739/photos-export

Thank you for you attention

  Alex Antão.

      

Le 11/10/2019 à 17:25, Alex Antão a écrit :

> Thank you for you attention

lot of works!!

for me I do so:

* always copy with the OS my photos to a date tree, usually on a day
basis, I call this "originals". I try never to edit anything there. I
keep these folders for around 2-3 years or as long as I have room on my
disks

* copy all this to a "edited" folder where digikam have access. There I
remove bad images, edit the images... then these photos a copied to my
online gallery

* usually do this on my internal ssd then move it to my "archives"
disks. I have 3 of them. One is nearly always near my computer, often
switched off. The second one is also near, but only switched on when
necessary. This last one is exchanged randomly for a third one usually
located in an anti fire box located at the other corner of the house (as
far as I can, when I was working it was at work). These archive disks
are 4 or 5 Tb

Like this I'm almost prepared to the disaster :-)

* I try not to remove photos from sd cards - they are so cheap nowaday I
keep them several years before reusing them.

I really like the fact than digikam uses the very same structure as the
OS, lot of work on the image is done at os level

jdd


--
http://dodin.org