[Feature] Merge Faces from Metadata with detected faces

[hjed]

I have searched the existing feature requests to make sure this is not a duplicate request.

• Yes

The feature

In #4348 immich implemented reading face metadata from files.

These faces show up in the people view, etc but they are often duplicated with immich's own detected faces. When there is a significant overlap with an ML detected face immich should merge the detected faces with the one provided by metadata (and ideally their linked groups too).

My observation is that most detected face bounding boxes detected by digikam very closely match the size and shape of the bounding boxes generated by immich for the same faces so this would be very useful for portability.

Platform

• Server
• Web
• Mobile

[tye-mustafa]

my guess the tags come from xml are bad i tried with Lightroom and exported it to immich ended up same,

[hjed]

The ones from digikam work great except they don't merge with the autodetected ones which is really annoying because it means to use immich's better algorithm I'd have to go through a retag all faces (and having gone through this with several over programs and since immich doesn't support exporting faces its a big time sink that I'll have to do again next time I change programs w/ the associated data loss, etc with that)

[5TDg9]

Is this really needed? This is probably going to get a little bit messy... i would be statisfied, if the face tags generated by immich would be just somehow "exportable" in a format, that other software (Digikam, XnView, etc.) is able to read or extract.
Instead of merging two face tag datas - one from immich and one from another software (e. g. Digikam), they could surely co-exist both in the XML-File with an special description-tag like mentioned here
(how mylio is doing it). This would be lot easier to implememt than the merging.

[hjed]

It means that every face has to be retagged in immich at the moment to use the immich features and is duplicated in the UI. The data should be portable.

[rhatguy]

I think the way forward for something like this is a comparison from a % of size perspective. For example if Immich detects a region that overlaps say 90% (guessing at an appropriate % here) with a region from metadata, it should assume those two faces are the same. Using a % should help with different size faces rather than an absolute like number of overlapping pixels.

It would be super nice if Immich had a function somewhere that could report these overlaps and "correct" them if the user confirmed. For example, metadata said this face was here....Immich detected it here (overlapping by 90%). If user confirms, update the pictures metadata to match Immich's face region.

[dnnr]

For what it's worth, digikam internally appears to do a very similar thing. Sometimes there can be multiple face regions (detected but unrecognized) on the same face, so that they overlap by a lot. If the user assigns a specific name to one of them, the other one disappears. Maybe it's worth checking their implementation for how they decide this?

[Finke3]

Totally agree. If Immich detects an existing face from meta data which overlaps it should assume those two faces are the same!

[Finke3]

I'm facing the same. My photos are mounted via external library and parts already tagged with faces by Lightroom. Now every photo contains two face regocnition boxes and will shown twice by searching via mobile app.

[Banewraith]

Using V2.3.1
I would like this as well. I spent a lot of time getting the names for faces found in old family photos. These assets are spread across several Immich instances. I wrote some api calls to extract the faces regions and names from the original Immich instance and burned them into a copy of the asset using exiftool. I then drop these exported version into the external folder of my Immich V2.3.1 instance.

After time each asset will show two faces. One with the name from metadata and the other recently detected. The bounding boxes look to be identical. When I name the version without a name, it will integrate with the correct person. No duplicated faces or regions after that.

So I can just manually name all these new detected faces since I have the names from the metadata and they shows up in the asset info screen. Thanks for adding the face import option. Not sure when it was added. I've been on v1.135 for a while

I'm working to update my api calls for v2.3 but, it would be great if there was a automagical way of combining faces based on bounding box cords.

In V1.133.x
They were found with an api call to "/api/assets/$AssetUUID"
The resulting json has a people array and that has a faces array that contains the bounding box info.

This snip does not show an example of two faces with the same bounding box but it is a start.

  "people": [
    {
      "id": "abf1205a-16a6-4bdd-b779-d58d40bfec33",
      "name": "xxx, Lyddie",
      "birthDate": null,
      "thumbnailPath": "upload/thumbs/fc8718a2-d975-40a9-8ec9-357641b1f49d/ab/f1/abf1205a-16a6-4bdd-b779-d58d40bfec33.jpeg",
      "isHidden": false,
      "isFavorite": false,
      "updatedAt": "2025-06-13T15:03:49.320112-05:00",
      "faces": [
        {
          "id": "2c89ecdc-548f-4922-86d0-f298ef6e8576",
          "imageHeight": 1440,
          "imageWidth": 2120,
          "boundingBoxX1": 1143,
          "boundingBoxX2": 1434,
          "boundingBoxY1": 326,
          "boundingBoxY2": 588,
          "sourceType": "machine-learning"
        }
      ]
    },
    {
      "id": "4bc1b036-30db-41ef-be85-05e946e3f242",
      "name": "xxx, Sally",
      "birthDate": null,
      "thumbnailPath": "upload/thumbs/fc8718a2-d975-40a9-8ec9-357641b1f49d/4b/c1/4bc1b036-30db-41ef-be85-05e946e3f242.jpeg",
      "isHidden": false,
      "isFavorite": false,
      "updatedAt": "2025-11-06T17:55:05.727535-06:00",
      "faces": [
        {
          "id": "de2fc5bf-a653-4cfc-8649-1ecbef515ddc",
          "imageHeight": 1440,
          "imageWidth": 2120,
          "boundingBoxX1": 271,
          "boundingBoxX2": 585,
          "boundingBoxY1": 70,
          "boundingBoxY2": 492,
          "sourceType": "machine-learning"
        }
      ]
    }
  ],
  "unassignedFaces": [],

[ecollins98]

I went to start a discussion topic on this but noticed this already exists. Id like for the combination of faces to become a thing. If you can do it on google photos and, I believe, apple photos, then why not immich, since those are the apps this one is based on.

[hjed]

Digikam is a good open source example of this too!
This + writing/uplaoding to external libraries are the features that would make me pay for immich

[pauverblom]

I believe the best way to detect this would be to calculate the center of two face regions with their bounding boxes' coordinates. If two bounding boxes have nearly the same center, they in theory correspond to the same person and should be merged.

Now, about merging, I'm not sure what would be the best approach. My idea was to create a new job called "consolidate faces" or something alike, which would do the procedure described above.

I can start to code a proof of concept if you guys think my approach is good.

[bo0tzz]

cc @mertalev

[mertalev]

This already exists in the person repository via IOU. It only handles deduping faces that exist when the face detection job runs, though, i.e. when faces were imported before that job. It doesn't handle the other direction of running metadata extraction after the asset went through face detection.

[Finke3]

Hi @mertalev , @bo0tzz : The most disturbing issue currently is, that persons will be listed twice in the photo details within the mobile App. For example I have a photo which shows two persons and I already added the facial data via Lightroom. Than this photo will be imported to Immich via external Library and Immich also performs facial regocnition for those two persons. In that case the mobile app will show in the photo details 4 persons (the person tagged via lightroom and the same person tagged by Immich).
This also affects the search within mobile App. If you search for one of those persons in the mobile app, the search shows two time the same photo for this person.
This behaviour does not exist within the web ui and only affects the mobile app. Within the web ui of course two facial frames will be shown for the person on the photo, but in the photo details ther person will only show once and also the search result is only one photo.
From my point of view we need to distingluish here two things:

1. Immich is adding facial data also for existing facial data.
2. The current mobile app behaves different from the web ui and for photos with duplicate assigned facial data, the same person will shown twice within photo details on the mobile App .

The second bullet point is from my point of view a bug, and it would make me so much happy if this could be fixed on short notice (see #23090).

[pauverblom]

This already exists in the person repository via IOU. It only handles deduping faces that exist when the face detection job runs, though, i.e. when faces were imported before that job. It doesn't handle the other direction of running metadata extraction after the asset went through face detection.

That's weird. I just ran the face detection job on my whole gallery and the duplicated faces didn't merge. Granted I ran it in "refresh" mode. Should I reset the face data -> import sidecar face data -> run face detection?

[5TDg9]

Unfortunately, I can't contribute any code at the moment... but I can contribute some of my "brainpower" :)

The problem everywhere with face-tagging is: there is NO standard way of doing it!

Merge information? Maybe, but don't reinvent the wheel:

As immich becomes increasingly popular due to its ease of use, perhaps the focus should be more on establishing an open-source quasi-standard instead of developing workarounds that attempt to merge the various solutions of closed-source companies.

Include 3D in depth-information ?!:

This new quasi-standard should perhaps also include the 3D information that already lies dormant in many photos as a hidden treasure and will become increasingly relevant in the future: https://developers.google.com/ar/develop/depth?hl=en

In my opinion, XML-based configurations (sidecar-files) aren't the best approach for daily use with large photo collections, as this would lead to an excessive number of read and write operations. However, sidecar files shouldn't be completely disregarded, as they might be useful for backups, imports and exports in the admin's backend of immich.

Support for videos !?:

I've been successfully using sidecar files for years to store metadata for videos, since embedding it directly into the file itself, as with EXIF, isn't possible. Regardless of the solution reached, I believe facetagging should not be considered in isolation, but rather a broader perspective should be taken.

My Outlook:

There are already many smart people who have given this topic a lot of thought:

It would be fantastic if we could, for example, draw the attention of developers like @boardhead (Phil Harvey) the developer of the legendary ExifTool or @woss /fork-exiftool who was contributing to it in the past days.... or at least someone who has the intellectual and temporal capacity to integrate this tool into immich:
Maybe @TheLastGimbus is our man for this problem here? He has certainly helped us get away from Google Photos before with his tool.

As one would expect from a legendary tool: Exiftool is already used in one or two places in immich.