Read/write video files' metadata using QtAV/ffmpeg

[AndriusWild]

Hello,

Looks like as long as libavformat/avformat.h presented in the appimage digiKam can read/write video files' metadata via calling the library directly from c++.

Here is an example: https://stackoverflow.com/questions/45050177/adding-metadata-informations-with-ffmpeg

Also looks like ffmpeg is capable of writing whatever tag you want in the mdta atom:
http://www.ffmpeg-archive.org/Write-QuickTime-tags-td4680508.html

[Gilles Caulier-4]

Hi,

I respond late to this file, but i respond : with this kind of simple C source code :

https://ffmpeg.org/doxygen/0.7/metadata-example_8c-source.html

... i can obtain these info from a MP4 file :

[gilles@localhost GIT]$ gcc ffmetadata.c -lavformat -lavutil
[gilles@localhost GIT]$  
[gilles@localhost GIT]$ ./a.out VID_20161224_201626.mp4  
major_brand=mp42
minor_version=0
compatible_brands=isommp42
creation_time=2016-12-25T04:16:26.000000Z
location=+37.7536-122.4330/
location-eng=+37.7536-122.4330/
com.android.version=7.0
com.android.capture.fps=30.000000

Exiftool report more information of course :

[gilles@localhost GIT]$ exiftool VID_20161224_201626.mp4         
ExifTool Version Number         : 10.45
File Name                       : VID_20161224_201626.mp4
Directory                       : .
File Size                       : 12 MB
File Modification Date/Time     : 2017:01:10 00:05:56+01:00
File Access Date/Time           : 2018:02:26 09:56:28+01:00
File Inode Change Date/Time     : 2018:02:26 09:56:24+01:00
File Permissions                : rw-rw-r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 v2 [ISO 14496-14]
Minor Version                   : 0.0.0
Compatible Brands               : isom, mp42
Movie Header Version            : 0
Create Date                     : 2016:12:25 04:16:26
Modify Date                     : 2016:12:25 04:16:26
Time Scale                      : 1000
Duration                        : 5.80 s
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 3
GPS Coordinates                 : 37 deg 45' 12.96" N, 122 deg 25' 58.80" W
Com Android Version             : 7.0
Com Android Capture Fps         : 30
Track Header Version            : 0
Track Create Date               : 2016:12:25 04:16:26
Track Modify Date               : 2016:12:25 04:16:26
Track ID                        : 1
Track Duration                  : 5.67 s
Track Layer                     : 0
Track Volume                    : 0.00%
Image Width                     : 1920
Image Height                    : 1080
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 1920
Source Image Height             : 1080
X Resolution                    : 72
Y Resolution                    : 72
Bit Depth                       : 24
Pixel Aspect Ratio              : 65536:65536
Color Representation            : nclx 1 1 1
Video Frame Rate                : 29.824
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2016:12:25 04:16:26
Media Modify Date               : 2016:12:25 04:16:26
Media Time Scale                : 48000
Media Duration                  : 5.80 s
Handler Type                    : Audio Track
Handler Description             : SoundHandle
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 1
Audio Bits Per Sample           : 16
Audio Sample Rate               : 48000
Movie Data Size                 : 12008560
Movie Data Offset               : 405243
Avg Bitrate                     : 16.6 Mbps
GPS Latitude                    : 37 deg 45' 12.96" N
GPS Longitude                   : 122 deg 25' 58.80" W
Image Size                      : 1920x1080
Megapixels                      : 2.1
Rotation                        : 180
GPS Position                    : 37 deg 45' 12.96" N, 122 deg 25' 58.80" W

... but at least the minimum are there, as date, GPS info, frame per seconds.

Certainly more details can be captured with more code as libav API is huge.

So my conclusion is : yes, we can use libav (ffmpeg) to handle some important metadata from video files.

I cannot guaranty that all video format are supported weel to extract metadata homogeneous.

I must also check how libav work to extract metadata without any decoding video file (the expected way to process), to prevent huge time latency while scanning.

For this purpose, i propose to introduce this kind of code in 6.0.0 branch, not in 5.x, as next and last 5.9.0 release will be published in few days.

Gilles Caulier

2018-01-11 16:32 GMT+01:00 Andrey Goreev <[hidden email]>:

Hello,

Looks like as long as libavformat/avformat.h presented in the appimage digiKam can read/write video files' metadata via calling the library directly from c++.

Here is an example: https://stackoverflow.com/questions/45050177/adding-metadata-informations-with-ffmpeg

Also looks like ffmpeg is capable of writing whatever tag you want in the mdta atom:
http://www.ffmpeg-archive.org/Write-QuickTime-tags-td4680508.html

[Gilles Caulier-4]

Voilà,

https://www.flickr.com/photos/digikam/39601432775/in/pool-digikam-labs

The basis is here and ffmpeg parser result re-routing need to be improved. Code is located in one CPP file :

https://cgit.kde.org/digikam.git/tree/libs/dmetadata/dmetadata_ffmpeg.cpp?h=development/6.0.0

More xmp.video.* tags need to be created from ffmpeg results. Even if video support is not compiled into Exiv2, all tags are listed in database (in fact only the Exiv2 video parsers are disabled). Internally, digiKam still use Exiv2 container and metadata manager to play with media info.

The list of XMP video tags are listed here :

https://github.com/Exiv2/exiv2/blob/master/src/properties.cpp#L1331

...and certainly more tags can be used in other XMP namespace.

As you can see in my screenshot, GPS info are there and also the date. Other tags are relevant of video properties (codec).

I need to wrap XMP to EXIF. EXIF are really used by database to register items. XMP are mostly string which need to be converted in int or real values (for Geolocation for ex.).

In current dmetadata_ffmpeg.cpp, the multi-stream parser need to be improved as all stream are parsed and metadata overwritten if video has more than one video stream in container.

At least the start of video support with ffmpeg is here. All contribution are welcome.

Best

Gilles Caulier

2018-02-26 10:05 GMT+01:00 Gilles Caulier <[hidden email]>:

Hi,

I respond late to this file, but i respond : with this kind of simple C source code :

https://ffmpeg.org/doxygen/0.7/metadata-example_8c-source.html

... i can obtain these info from a MP4 file :

[gilles@localhost GIT]$ gcc ffmetadata.c -lavformat -lavutil
[gilles@localhost GIT]$  
[gilles@localhost GIT]$ ./a.out VID_20161224_201626.mp4  
major_brand=mp42
minor_version=0
compatible_brands=isommp42
creation_time=2016-12-25T04:16:26.000000Z
location=+37.7536-122.4330/
location-eng=+37.7536-122.4330/
com.android.version=7.0
com.android.capture.fps=30.000000

Exiftool report more information of course :

[gilles@localhost GIT]$ exiftool VID_20161224_201626.mp4         
ExifTool Version Number         : 10.45
File Name                       : VID_20161224_201626.mp4
Directory                       : .
File Size                       : 12 MB
File Modification Date/Time     : 2017:01:10 00:05:56+01:00
File Access Date/Time           : 2018:02:26 09:56:28+01:00
File Inode Change Date/Time     : 2018:02:26 09:56:24+01:00
File Permissions                : rw-rw-r--
File Type                       : MP4
File Type Extension             : mp4
MIME Type                       : video/mp4
Major Brand                     : MP4 v2 [ISO 14496-14]
Minor Version                   : 0.0.0
Compatible Brands               : isom, mp42
Movie Header Version            : 0
Create Date                     : 2016:12:25 04:16:26
Modify Date                     : 2016:12:25 04:16:26
Time Scale                      : 1000
Duration                        : 5.80 s
Preferred Rate                  : 1
Preferred Volume                : 100.00%
Preview Time                    : 0 s
Preview Duration                : 0 s
Poster Time                     : 0 s
Selection Time                  : 0 s
Selection Duration              : 0 s
Current Time                    : 0 s
Next Track ID                   : 3
GPS Coordinates                 : 37 deg 45' 12.96" N, 122 deg 25' 58.80" W
Com Android Version             : 7.0
Com Android Capture Fps         : 30
Track Header Version            : 0
Track Create Date               : 2016:12:25 04:16:26
Track Modify Date               : 2016:12:25 04:16:26
Track ID                        : 1
Track Duration                  : 5.67 s
Track Layer                     : 0
Track Volume                    : 0.00%
Image Width                     : 1920
Image Height                    : 1080
Graphics Mode                   : srcCopy
Op Color                        : 0 0 0
Compressor ID                   : avc1
Source Image Width              : 1920
Source Image Height             : 1080
X Resolution                    : 72
Y Resolution                    : 72
Bit Depth                       : 24
Pixel Aspect Ratio              : 65536:65536
Color Representation            : nclx 1 1 1
Video Frame Rate                : 29.824
Matrix Structure                : 1 0 0 0 1 0 0 0 1
Media Header Version            : 0
Media Create Date               : 2016:12:25 04:16:26
Media Modify Date               : 2016:12:25 04:16:26
Media Time Scale                : 48000
Media Duration                  : 5.80 s
Handler Type                    : Audio Track
Handler Description             : SoundHandle
Balance                         : 0
Audio Format                    : mp4a
Audio Channels                  : 1
Audio Bits Per Sample           : 16
Audio Sample Rate               : 48000
Movie Data Size                 : 12008560
Movie Data Offset               : 405243
Avg Bitrate                     : 16.6 Mbps
GPS Latitude                    : 37 deg 45' 12.96" N
GPS Longitude                   : 122 deg 25' 58.80" W
Image Size                      : 1920x1080
Megapixels                      : 2.1
Rotation                        : 180
GPS Position                    : 37 deg 45' 12.96" N, 122 deg 25' 58.80" W

... but at least the minimum are there, as date, GPS info, frame per seconds.

Certainly more details can be captured with more code as libav API is huge.

So my conclusion is : yes, we can use libav (ffmpeg) to handle some important metadata from video files.

I cannot guaranty that all video format are supported weel to extract metadata homogeneous.

I must also check how libav work to extract metadata without any decoding video file (the expected way to process), to prevent huge time latency while scanning.

For this purpose, i propose to introduce this kind of code in 6.0.0 branch, not in 5.x, as next and last 5.9.0 release will be published in few days.

Gilles Caulier

2018-01-11 16:32 GMT+01:00 Andrey Goreev <[hidden email]>:

Hello,

Looks like as long as libavformat/avformat.h presented in the appimage digiKam can read/write video files' metadata via calling the library directly from c++.

Here is an example: https://stackoverflow.com/questions/45050177/adding-metadata-informations-with-ffmpeg

Also looks like ffmpeg is capable of writing whatever tag you want in the mdta atom:
http://www.ffmpeg-archive.org/Write-QuickTime-tags-td4680508.html