|
(11) | EP 2 768 168 A1 |
| (12) | EUROPEAN PATENT APPLICATION |
|
|
|
|
||||||||||||||||||||
| (54) | Method to recommend social network threads |
| (57) The invention has for object a method to recommend social network threads (11a, 11b,
11c), with the steps : - a fragment extraction unit (13) extracts a multimedia fragment of the multimedia flow using changes in multimedia flow content properties to delimit the multimedia fragment, - a multimedia analysis unit (15) analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements, - a thread recommendation unit (17) uses the multimedia fragment descriptor to find social network threads (11a, 11b, 11c) related to the multimedia fragment, - the thread recommendation unit (17) recommends the network threads (11a, 11b, 11c) related to the multimedia fragment. |
[0001] The invention relates to a method to create recommendations for social threads in a social network, in particular in relationship with a multimedia stream.
[0002] It has been observed that the activity on social networks, such as facebook™ and twitter™ spikes during major television events such as sport matches and competitions, live music shows and the like.
[0003] For example, the activity of users on twitter™ spiked at record heights around three thousand tweets (messages posted on threads) per seconds when the Japanese team scored against Cameroon on June, 14th 2010 during the FIFA World Cup, at the NBA finals on June, 17th 2010 and at the Japanese victory over Denmark in World Cup, while the absolute record was reached at 7196 tweets per second during the 2011 FIFA Women's World Cup final between Japan and the United States.
[0004] However, to find social network threads related to the currently watched TV or multimedia stream, the user must manually search the social networks using topics and tags as keywords, possibly missing relevant threads.
[0005] It is known to use automated recommendations based on the current activity of the threads, by picking so called "hot topics" with high activity, or to base recommendations on evaluations of social network friends to pick social network threads.
[0006] Such methods mostly rely on historical data, or ratings of other social network users, and do not allow real time following of the social network threads in the social network that are related to a currently attended network stream.
[0007] Therefore, these recommendation methods are not suited, in particular for socially augmented media flows, such as television debates and sports events, where at least one dedicated social thread is followed throughout the show or event, and may be integrated as a feedback or question source in the emission.
[0008] In order to overcome the aforementioned drawbacks, the invention has for object a method to recommend social network threads to a user attending a multimedia flow, comprising the steps :
- a fragment extraction unit extracts a multimedia fragment of the multimedia flow using changes in multimedia flow content properties to delimit the multimedia fragment,
- a multimedia analysis unit analyses content of the multimedia fragment to fill a fragment descriptor containing descriptive elements,
- a thread recommendation unit uses the multimedia fragment descriptor to find social network threads related to the multimedia fragment,
- the thread recommendation unit recommends the network threads related to the multimedia fragment.
[0009] The method according to the invention may also present one or more of the following characteristics, taken separately and/or in combination.
[0010] The multimedia flow comprises one of the following : a video flow, an audio flow, a slide show, a text stream.
[0011] The step in which the multimedia analysis unit analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein an object recognition subunit performs an object recognition on a picture of the multimedia fragment to extract recognized object names as descriptive elements.
[0012] The step in which the multimedia analysis unit analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein a face recognition subunit performs a face recognition on a picture of the multimedia fragment to extract names of recognized people as descriptive elements.
[0013] The step in which the multimedia analysis unit analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein a sound recognition subunits performs a sound recognition on a sound extract of the multimedia fragment to extract recognized sound source names as descriptive elements.
[0014] The step in which the multimedia analysis unit analyses content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein a speech-to-text conversion unit converts speech from a sound extract of the multimedia fragment to extract speech words as descriptive elements.
[0015] The step in which the multimedia analysis unit analyses content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein an optical character recognition unit performs an optical character recognition on a picture of the multimedia fragment to extract character chains as descriptive elements.
[0016] The step in which the multimedia analysis unit analyses content of the multimedia fragment to fill a fragment descriptor containing descriptive elements comprises a step wherein a chromatic analysis subunit performs a chromatic analysis on a picture of the multimedia fragment to extract colour patterns as descriptive elements.
[0017] It further comprises the following steps :
- a social network crawler analyses the social network threads on the social network platform,
- the social network crawler extracts descriptive elements from the analysed social network threads and uses said descriptive elements to fill a social network thread descriptors,
- the filled social network thread descriptors are stored on a database to be compared to multimedia fragment descriptors to find social network threads related to the multimedia fragment.
[0018] Said steps of analysing the social network threads on the social network platform and of extracting descriptive elements from the analysed social network threads may further comprise the steps :
- the social network crawler emerges a conversation tree structure of the social network thread,
- the social network crawler uses structural and topological properties of the conversation tree structure as descriptive elements.
[0019] The steps of analysing the social network threads on the social network platform and of extracting descriptive elements from the analysed social network threads may comprise the steps:
- the social network crawler establishes chronological evolution properties of the social network threads,
- said chronological evolution properties are compared to multimedia flow chronological evolution properties of the multimedia flow using structure similarity metrics.
[0020] The invention also has for object the associated data storage medium storing a machine executable program for performing a method to recommend social threads as previously described and the associated network node configured to perform a method to recommend social network threads to a user as previously described.
[0021] Other characteristics and advantages of the invention will appear at the reading of the description of the following figures, among which:
- figure 1 is a schematic representation of the different apparatus elements involved
in the method,
figure 2 is a schematic flowchart of the main steps of a procedure to create social topic descriptors used in the invention, - figure 3 is a schematic flowchart of the main steps of the method according to the invention.
[0023] On figure 1 is schematically represented a home network 100 with a multimedia receiver 1, comprising in this example a television 3, acting as multimedia display unit, and a base-station 5 connected to the internet and receiving a multimedia stream, here an IPTV (Internet Protocol TeleVision) stream. As an alternative, the multimedia stream may be any other kind of video stream, an audio stream, a slideshow, a text stream such as a news title stream, and the like.
[0024] The home network 100 further comprises a social network browsing node 7, connected via a social network client 9 to a social network database 11, the social network client 9 and database 11 being outside of the home network, for example on the internet. The home network 100 also comprises a fragment extraction unit 13 and a fragment analysis unit 15, here implemented in the base-station 5. It comprises in addition a recommendation unit 17.
[0025] The recommendation unit 17 recommends specific network threads searched using search queries in a social network thread database by proposing said threads to the user for reading, the threads being selected according to specific rules chosen for example as a function of the interests of the user, used in the search queries.
[0026] The social network browsing node 7, for example a personal computer or a smartphone, is connected to the same network as the multimedia receiver 1, for example within the home or local network, which is built around the base station 5 for the user's home and acting as a communication centralizer and as a bridge to the internet.
[0027] The social network browsing node 7 is also connected to a social network via a social network client 9, for example a micro-blogging platform browser, which accesses data stored on remote servers on the internet that form at least part of the content of the social network.
[0028] The multimedia display unit 1 and the social network browser 7 may be fused in a single device, for example a personal computer, a laptop, an electronic pad, a smartphone, a television in particular IP television, or be two distinct devices, for example a television and a smartphone as previously discussed. Connection between the two is assumed, either directly or via a centralizing element such as the base-station 5 or a server.
[0029] The social network client 9 is used to access a social network database 11 comprising multiple social threads 11a to 11c, which are aggregations of essentially user posted messages, possibly in reply to each other, and possibly with enclosed or embedded contents such as pictures, short videos, links, etc. The social network database 11 is in particular located on remote servers forming a social network platform and are accessed via the internet.
[0030] The base station 5 receives IP packets from a multimedia flow provider via the internet, and converts said packets in multimedia content. Said content is then displayed on the television 3.
[0031] The base station 5 comprises in the discussed embodiment a fragment extraction unit 13, and a fragment analysis unit 15.
[0032] The unit 13 is configured to extract parts of the multimedia content and forward them as fragments. In particular, the fragment extraction unit 13 is configured to isolate multimedia fragments, containing frames or pictures, scenes or sound extracts, which form whenever possible a consistent unit.
[0033] The fragment extraction unit 13, along with all units and subunits mentioned afterwards, comprise memory and processing means configured to perform the functions they correspond to, and may be distributed in the devices in the considered network 100 or in internet servers, with possibly more than one unit in a device, while the dedicated memory and processing means may be shared with other, possibly unrelated, functions and units.
[0034] The fragment extraction unit 13 is configured to use statistical and media analysis subunits to extract fragments from the multimedia flow, in particular coherent fragments, corresponding to a relatively complete and independent content. A coherent fragment is for example obtained by grouping in one fragment the multimedia flow content between apparition and disappearance of specific characteristics in the multimedia content (colour, face or object on screen, speaking voice, etc.).
[0035] Such a fragment is for example a single scene, which the unit recognizes as such using fade to black or massive change in the images due to changes in framing as triggers. The fragment extraction unit 13 may also use a face recognition subunit to identify a person on screen, and use the presence of said person as a continuity indicator meaning that the content is still part of the same fragment.
[0036] Considering for example a multimedia stream containing a political debate on television, in which two candidates face each other, a first and most simple way for the fragment extraction unit 13 to isolate fragments is to recognize the changes in the candidate on screen or the speaking voice, so that a fragment corresponds to the single speeches the candidates hold.
[0037] Otherwise or in parallel, the fragment extraction unit 13 may use the statistics of the changes in framing and shots as indicators for continuity or for changing the fragment. In the case of the political debate, a speech of a candidate will often correspond to long single shots either of the candidate himself or of the attending public, while a heated discussion between the candidates will correspond to fast alternating shots of said candidates and other intervening people, with similar alternation patterns in the speaking voices.
[0038] The speech and discussion form each a single and separate fragment and may thus be recognized as such analysing the frame changing frequency.
[0039] This statistical change in framing duration is an indicator for ending the current fragment and grouping the following content in a new one, so as to have one fragment corresponding to the speech, and the next one corresponding to the discussion.
[0040] The fragment extraction unit 13 may be configured to use changes in specific statistics and characteristics of the multimedia content as triggers to set a marker in the multimedia flow time line which delimits one fragment from the next.
[0041] In particular, the fragment extraction unit 13 may be configured to store temporarily on memory, in particular cache memory, the multimedia content or packets. The stored data is then processed, so that fragment delimitation markers may be established. The delimited fragments are then sent to the other units or subunits for further processing.
[0042] For audio or radio flows, the fragment extraction unit 13 may use a change in content nature, for example a transition from speech to music, a recurring jingle, or a change in the speaking voices as fragment delimitation.
[0043] If no marker can be set, a systematic fractioning can be done using predetermined time intervals (e.g. 30 seconds or one minute) to allow real time or almost real time recommendation, and to avoid sending and processing multimedia fragments that are too heavy in data amount. Also, to avoid the fragments to be too numerous while having not enough content for a meaningful search to be conducted, a minimal length (e.g. 3 seconds) may be predetermined for the fragments.
[0044] The fragment extraction unit 13 may also hash the fragments, for example by keeping only every second or third image of a video stream to make the fragments lighter and easier to process. In the case of an audio or radio flow any suitable sound file compression procedure may be used.
[0045] As an example, the fragment extraction unit 13 may comprise an application downloaded and run on the social network browsing node 7, and/or a program stored and run on the base station 5 or on the social network platform.
[0046] The extracted multimedia fragments are sent to a multimedia analysis unit 15, which converts the multimedia fragments in data usable in search queries, by extracting descriptive elements and properties from the fragments. For example, if the fragments comprise a picture of an actor, the analysis unit may use a face recognition subunit to identify said actor, and isolate his name as a relevant descriptive element to use in search queries.
[0047] Other possible data sources include object recognition on the pictures, music and sound recognition on the audio track, optical character recognition (OCR), speech-to-text conversion, chromatic analysis and the like. To perform all this processing, the multimedia analysis unit 15 comprises the associated means to process multimedia data, which may include subunits associated to the aforementioned functions. Said subunits may be shared with the fragment extraction unit 13 for the extracting of the multimedia fragments.
[0048] The processes and techniques used to analyse the multimedia stream are of course varying according to the nature of said stream.
[0049] For example, if the multimedia stream is a radio or more generally a sound stream, the multimedia analysis unit 15 will run voice detection methods to identify the presence of a speech. In case a speech is detected, the analysis unit 15 uses a speech-to-text conversion subunit to transcript the sound in searchable words. If no speech is detected, or in addition or in parallel to the speech-to-text transcription, the analysis unit 15 may use sound or music recognition subunits, for example to identify the title and artist of a played music track, or the name of a sound source (instrument, animal, etc.) and use them as searchable words.
[0050] If the multimedia stream is a video stream or slide show, the multimedia analysis unit 15 will use for example face or object recognition subunits, to identify people or objects on the multimedia fragment. The names obtained are then used as searchable terms.
[0051] The multimedia analysis unit 15 may also use structural or descriptive metadata. For example, on an IPTV program, the title, a short synopsis and/or the cast and authors and producers may be forwarded to the user in a transparent fashion, by multiplexing the data with the multimedia content. Said data may for example serve when the user requests information on the program he is currently watching using an integrated information display function of his television.
[0052] Such metadata about multimedia content is most of the time accessible via application programming interfaces (API), often open and/or public application programming interfaces. Such metadata is added in particular by the multimedia flow provider as an additional service.
[0053] The isolated descriptive elements are used to fill a multimedia fragment descriptor, which contains keywords and statistics extracted from the multimedia fragment, for example a set of weights associated to the keywords and attributed according to the number of occurrences of the considered keyword, or the chronological evolution of specific properties and characteristics of the multimedia fragment.
[0054] The metrics used to search for related threads 11a, 11b, 11c using the descriptor may comprise the known metrics such as objective image quality index (OQI), structural similarity (SSIM) or the Czenakowski distance (CZD). In particular, threads 11a, 11b, 11c with elements within a predetermined distance of the descriptive elements of the fragment descriptor according to the chosen metrics are considered relating to the content of the multimedia fragment.
[0055] The fragment descriptor is then sent to a social thread recommendation unit 17 which is configured to use the fragment descriptor in a search query for social network threads 11a, 11b, 11c which are consequently bound to the content of the currently attended multimedia stream.
[0056] The thread recommendation unit 17 is therefore connected to the social network client 9, configured to give recommended thread identifiers in answer to a query containing recommendation rules.
[0057] The recommendation rules are elaborated using the fragment descriptor and the descriptive elements thereof, possibly taking into account the profile, preferences and personal settings of the user. To do this, the recommendation unit 17 is connected to a profile and settings depository 19 containing the profile, preferences and setting rules of the user.
[0058] Such rules may include user preferences for ordering the results from most to least interesting: nationality, search language, current geographical position and any other elements relevant for the ordering.
[0059] Identifiers of the returned recommended social threads 11a, 11b, 11c are then forwarded to the social browsing node 7. The social network browsing node 7 then only needs to search for the threads 11a, 11b, or 11c associated to the stored identifiers and display at least part of their content for the user to pick the ones he is interested in.
[0060] The recommendation unit 17 is connected to a recommendation database 21, which is configured to store the multimedia fragment descriptors and at least part of the returned results.
[0061] If a search query to the social network client 9 does not yield sufficient result, meaning at least one social thread within a certain distance from the descriptive elements of the multimedia fragment descriptor, the recommendation unit 17 may search within the recommendation database 21 the previously established fragment descriptor closest to the current one yielding no or insufficient results, and use the returned results corresponding to said closest descriptor.
[0062] Since the search in the content of the social threads is time and resource consuming, a preparatory work may be done to elaborate summarized descriptors of the social threads.
[0063] Connected to the social network database 21 is a social network crawler 23, for systematic and thorough browsing of the social thread contents, in particular to forward and update the social network threads that are followed by users. The social network crawler 23 is in particular used to conduct intensive analysis and ordering of the network threads 11a, 11b, 11c to establish thread properties.
[0064] Figure 2 represents in a schematic fashion the main steps of a particular embodiment of a method 200 to analyse the content of social threads 11a, 11b and 11c stored on the social network database 11 using the social network crawler 23 in order to make the search for specific threads less time and resource consuming, and thus reach minimal delay between media fragment display and associated recommendation.
[0065] On figure 2, the first step 201 for a gross filtering of topic candidates is the listing of their titles and topics by the network crawler 23. This implies that the threads 11a, 11b, 11c have previously been given one or more topic descriptors, related to their content and containing for example titles and/or tagged keywords. The social network crawler 23 may in particular be configured to store possible synonym words, general associated themes and lexical fields relating to the titles and/or keywords.
[0066] The following step 203 is the analysis of the content, recurrent words are listed with the number of times they occur in the considered thread, the embedded content is analysed using similar functions and subunits as for extracting significant elements from the multimedia fragment content: face, object and character recognition for embedded images and videos, music and sound recognition for audio tracks.
[0067] The third step 205 is the emerging of the data structures in the selected social threads 11a, 11b, 11c. The emerging of the data structures is the conversion from the initial, chronological linear graph of the messages ("timeline") to a conversation tree structure, where the semantic relationship of the messages is taken in consideration.
[0068] For example, in a conversation tree structure, the messages posted in response to a first one are directly related to the said, without consideration of their posting time and possible unrelated messages posted in between.
[0069] The ramified structure of the obtained graph, with the replies "sprouting" from the initial message, gives it its name of conversation tree. To proceed to the conversion, the social crawler 23 uses a semantic analysis subunit and/or metadata of the messages, which may be associated a tag stating that the considered message is a reply, and identify the message of which it is a reply.
[0070] For example in micro-blogging threads 11a, 11b, 11c it is common to start a reply to a message from a user user1 with the character chain @user1. The social network crawler may be configured to identify such character chains starting with the @ sign, to search for messages from a user whose name is written after the @, and to link the considered message to the identified one as a reply.
[0071] The third step 205, once the data structure is emerged, is the establishing of statistical and topological thread structure properties from the emerged structure.
[0072] The statistical and topological thread structure properties are for example the number of intervening users, as an indicator of popularity, or the average number of posts per user, as an indicator that the thread is followed by its contributors.
[0073] In particular, said thread properties may comprise chronological evolution properties. For example, this implies listing the time frames in which the number of posts is at or below average, and the time frames in which activity spikes. The chronological evolution properties may be used to find similarity patterns with the multimedia fragment contents.
[0074] Examples of properties that vary over time may for example include the number of intervening users, the average length of the discussions (posts in reply to each other) in the conversation tree, average time between a post and the next reply to considered post, average number of replies to each post, and the like.
[0075] Said chronological evolution properties may in particular be compared to multimedia flow chronological evolution properties. Said multimedia flow chronological evolution properties are established by analysing the chronological succession of fragments. The chronological evolution of both multimedia flow and social network thread may then be compared using structure similarity metrics.
[0076] In the case of on demand or more generally not live content, the chronological properties of multimedia flow and network thread are advantageously compared using a relative time line, due to the fact that at the opposite of live content, no explicit synchronization may be assumed,
[0077] Returning to the example of the political debate, monologue speeches of one candidate will probably correspond to low activity periods, while heated debates often correspond to activity spikes. By using pattern similarity detecting subunits such criteria may be used to link fragments and threads.
[0078] All the extracted thread relevant data is put together in a social network thread descriptor in step 207, the thread descriptor is stored on a database (not represented on figure 2) for future use by the social network clients 9 or for internal search engines of the social network platform.
[0079] The search for network threads 11a, 11b, 11c that correspond to a multimedia fragment may accordingly be performed by comparing the social topic descriptors and the multimedia fragment descriptors. Consequently, the thorough and resource consuming search in the social threads 11a, 11b, 11c needs only be done periodically, for example once a predetermined amount of modifications (updates) is detected on the network threads 11a, 11b, 11c or at predetermined time intervals.
[0080] Figure 3 is a schematic flow chart with the main steps of the method 300 to recommend social threads 11a, 11b, 11c represented.
[0081] The first step 301 is the acknowledgement that a multimedia stream is attended. This corresponds to the launching of the application or program corresponding to the method. If no clear and non empty fragment can be identified, an error message may be sent, prompting the user to check for problems in the multimedia flow reception.
[0082] Once a multimedia stream is recognized and content is identified, said content is analysed in step 303 to extract relevant fragments, using the previously described units and subunits.
[0083] The following step 305 is the analysis of the fragment content to identify if a significant fragment has been extracted. This may include an estimation, in particular rash estimation, of the accessible content (keywords, names, metrics, etc.) for the fragment descriptor, and comparing that estimation to a threshold value. If no descriptor or only an insufficient descriptor may be established, the method prescribes a return to previous step 303 to isolate a new multimedia fragment.
[0084] If the fragment is deemed significant (containing at least a predetermined amount of content), the fragment descriptor is established in step 307. Possible embodiments of this step have previously been described.
[0085] In step 309, the fragment descriptor is searched for new content in comparison to the previously established descriptors. If no new content is identified, a new fragment is extracted in a return to step 303.
[0086] If new multimedia fragment content is identified in the descriptor, in step 311 network thread descriptors are collected from a social thread descriptor database, for example using a main keyword or semantic element to collect only the descriptors of a set of potentially related social network threads 11a, 11b, 11c in a previous, rapid sorting.
[0087] In the following step 313 the similarities between the social thread descriptors and the multimedia fragment descriptors are computed using any known method.
[0088] If a social thread is identified as linked to the current multimedia fragment (both descriptors being within a certain distance of each other), said network thread is recommended in step 315. An identifier of the social network thread is forwarded to the social network browsing node 7 to be used to access the content of said thread by the social network browsing node 7 if the user wants to read the identified social network thread.
[0089] If no social network thread descriptor is close enough to the one of the multimedia fragment, the similarity computing element (either the base station 5, the social network browsing node 7 or a server of the social network) may request in step 317 a refreshing of the social network thread descriptors.
[0091] Said refreshing starts with step 319 in which the network crawler 23 checks the social network database DB for new topics in step 319.
[0092] If a thread is newly identified as being related to the multimedia fragment, said thread is analysed in step 321 to establish a complete descriptor of the thread, said descriptor being sent to a social network thread descriptor database DB. Said database DB is then used for subsequent researches so as not to require complete search and analysis of the social network threads 11a, 11b, 11c.
[0093] if no new topic is identified as relating to the multimedia fragment, the network crawler 23 may check already identified threads 11a, 11b, 11c for recent activity (new messages) in a step 323. If a sufficient amount of new content is detected in a previously recommended thread, said social network thread is recommended again (step 321).
[0094] The social network thread descriptors established or updated during the social network crawling are all stored in a database DB following the recommendation, or failure to identify a recommended social network thread.
[0095] The method proposed by the invention allows real time recommendation of social network threads 11a, 11b, 11c based on a multimedia flow attended by a user. This is in particular useful in the case of socially augmented multimedia flows where dedicated social network threads 11a, 11b, 11c are used as feedback or content source.
1. Method to recommend social network threads (11a, 11b, 11c) to a user attending a multimedia
flow, comprising the steps :
- a fragment extraction unit (13) extracts a multimedia fragment of the multimedia flow using changes in multimedia flow content properties to delimit the multimedia fragment,
- a multimedia analysis unit (15) analyses content the multimedia fragment to fill a fragment descriptor containing descriptive elements,
- a thread recommendation unit (17) uses the multimedia fragment descriptor to find social network threads (11a, 11b, 11c) related to the multimedia fragment,
- the thread recommendation unit (17) recommends the social network threads (11a, 11b, 11c) related to the multimedia fragment.
2. Method according to claim 1, wherein the multimedia flow comprises one of the following
: a video flow, an audio flow, a slide show, a text stream.
3. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein an object recognition subunit
performs an object recognition on a picture of the multimedia fragment to extract
recognized object names as descriptive elements.
4. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein a face recognition subunit
performs a face recognition on a picture of the multimedia fragment to extract names
of recognized people as descriptive elements.
5. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein a sound recognition subunits
performs a sound recognition on a sound extract of the multimedia fragment to extract
recognized sound source names as descriptive elements.
6. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein a speech-to-text conversion
unit converts speech from a sound extract of the multimedia fragment to extract speech
words as descriptive elements.
7. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein an optical character recognition
unit performs an optical character recognition on a picture of the multimedia fragment
to extract character chains as descriptive elements.
8. Method according to claim 1, wherein the step in which the multimedia analysis unit
(15) analyses the content of the multimedia fragment to fill a fragment descriptor
containing descriptive elements comprises a step wherein a chromatic analysis subunit
performs a chromatic analysis on a picture of the multimedia fragment to extract colour
patterns as descriptive elements.
9. Method according to any of the precedent claims, wherein it further comprises the
following steps :
- a social network crawler (23) analyses the social network threads (11a, 11b, 11c) on the social network platform,
- the social network crawler (23) extracts descriptive elements from the analysed social network threads (11a, 11b, 11c) and uses said descriptive elements to fill a social network thread descriptors,
- the filled social network thread descriptors are stored on a database (DB) to be compared to multimedia fragment descriptors to find social network threads (11a, 11b, 11c) related to the multimedia fragment.
10. Method according to claim 9, wherein the steps of analysing the social network threads
(11a, 11b, 11c) on the social network platform and of extracting descriptive elements
from the analysed social network threads (11a, 11b, 11c) comprise the steps :
- the social network crawler (23) emerges a conversation tree structure of the social network thread,
- the social network crawler (23) uses structural and topological properties of the conversation tree structure as descriptive elements.
11. Method according to claim 9 or 10, wherein the steps of analysing the social network
threads (11a, 11b, 11c) on the social network platform and of extracting descriptive
elements from the analysed social network threads (11a, 11b, 11c) comprises the steps:
- the social network crawler (23) establishes chronological evolution properties of the social network threads (11a, 11b, 11c),
- said chronological evolution properties are compared to multimedia flow chronological evolution properties of the multimedia flow using structure similarity metrics.
12. Data storage medium storing a machine executable program for performing a method to
recommend social threads (11a, 11b, 11c) comprising the steps :
- a fragment extraction unit (13) extracts a multimedia fragment of the multimedia flow using changes in multimedia flow content properties to delimit the multimedia fragment,
- a multimedia analysis unit (15) analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements,
- a thread recommendation unit (17) uses the multimedia fragment descriptor to find social network threads (11a, 11b, 11c) related to the multimedia fragment,
- the thread recommendation unit (17) recommends the network threads (11a, 11b, 11c) related to the multimedia fragment.
13. Network node configured to perform a method to recommend social network threads (11a,
11b, 11c) to a user, with the steps :
- a fragment extraction unit (13) extracts a multimedia fragment of the multimedia flow using changes in multimedia flow content properties to delimit the multimedia fragment,
- a multimedia analysis unit (15) analyses the content of the multimedia fragment to fill a fragment descriptor containing descriptive elements,
- a thread recommendation unit (17) uses the multimedia fragment descriptor to find social network threads (11a, 11b, 11c) related to the multimedia fragment,
- the thread recommendation unit (17) recommends the network threads (11a, 11b, 11c) related to the multimedia fragment.