The Movie Database Support

When searching for a movie via the API it is useful to apply Unicode normalization to the search query.

For example the Movie "Zurück in die Zukunft 1" can have multiple encodings of the character "ü":

  • Use the character "LATIN SMALL LETTER U WITH DIAERESIS"
  • Use a two character encoding with the letters "u" (LATIN SMALL LETTER U) and " ̈" (COMBINING DIAERESIS)
  • Some other possible encodings...

When I use the encoding with two Unicode characters for the "ü" the API returns no result: http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=Zuru%CC%88ck%20in%20die%20Zukunft%201&year=&language=de

When I use the single character "ü" the API behaves as expected and returns one result: http://api.tmdb.org/3/search/movie?api_key=f7f51775877e0bb6703520952b3c7840&query=Zur%C3%BCck%20in%20die%20Zukunft%201&year=&language=de

This is currently a problem on Mac OS X/ macOS in combination with Kodi because they use filenames in the denormalized form for the API lookup. I filed a separate bug report there: http://trac.kodi.tv/ticket/17308 Because the Kodi bug report is only visible to logged in users I duplicated the content here:


All builtin library scanners have problems with filenames containing umlaut characters.

Example:

Name a Movie "Zurück in die Zukunft 1" (Back to the Future).
Add the containing folder to the library
Look in the event log 

The event log has an entry:

Video library scanner Failed to scan movie: Zurück in die Zukunft 1.mkv

The relevant part of the kodi.log is: ...

When I examine the opened URIs the problem becomes clear. There are multiple ways to encode the character "ü" in unicode. The called URI from the scraper uses the string "Zurück" with the u encoded as two characters. See this Unicode inspector for an example: ​http://apps.timwhitlock.info/unicode/inspect?s=Zuru%CC%88ck

The two character encoding is not understood by the tmdb api thus the movie is not found. Maybe one could do some kind of Unicode normalization?

This is a well known problem on Mac OS X/macOS (see: ​http://apple.stackexchange.com/questions/83935/unicode-normalization-for-filenames-and-copied-text-from-pdfs).

QUOTE: HFS+ requires filenames to be in decomposed form (LATIN SMALL LETTER A + COMBINING DIAERESIS) instead of composed form (LATIN SMALL LETTER A WITH DIAERESIS).

2 replies (on page 1 of 1)

Jump to last post

Hi @Pendistic Thanks for the detailed report. Unfortunately, I don't have any time in the near future to look into this. There are multiple layers to this which is why it's complicated. For now all I can say is we only support the single character encoding and it should be the clients job to normalize it to a format we support.

It would be nice to improve one day, no doubt about it but I don't see me having time to look into this in the very near future.

You are right that it's the clients job to normalize. Thank you for the quick reply.

Can't find a movie or TV show? Login to create it.

Global

s focus the search bar
p open profile menu
esc close an open window
? open keyboard shortcut window

On media pages

b go back (or to parent when applicable)
e go to edit page

On TV season pages

(right arrow) go to next season
(left arrow) go to previous season

On TV episode pages

(right arrow) go to next episode
(left arrow) go to previous episode

On all image pages

a open add image window

On all edit pages

t open translation selector
ctrl+ s submit form

On discussion pages

n create new discussion
w toggle watching status
p toggle public/private
c toggle close/open
a open activity
r reply to discussion
l go to last reply
ctrl+ enter submit your message
(right arrow) next page
(left arrow) previous page

Settings

Want to rate or add this item to a list?

hyrje