(19)
(11)EP 3 579 048 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
30.03.2022 Bulletin 2022/13

(21)Application number: 19178154.1

(22)Date of filing:  04.06.2019
(51)International Patent Classification (IPC): 
G03B 17/54(2021.01)
G03B 21/10(2006.01)
G03B 29/00(2021.01)
G06F 1/16(2006.01)
(52)Cooperative Patent Classification (CPC):
G03B 17/54; G03B 29/00; G03B 21/10; G06F 1/1637; G06F 1/1639; G06F 1/1686; H04M 1/0264; H04M 1/0266

(54)

STORAGE MEDIUM, ELECTRONIC DEVICE AND IMAGE PROCESSING METHOD

SPEICHERMEDIUM, ELEKTRONISCHE VORRICHTUNG UND BILDVERARBEITUNGSVERFAHREN

SUPPORT D'INFORMATIONS, DISPOSITIF ÉLECTRONIQUE ET PROCÉDÉ DE TRAITEMENT D'IMAGES


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 04.06.2018 CN 201810564121
04.06.2018 CN 201820859848 U

(43)Date of publication of application:
11.12.2019 Bulletin 2019/50

(73)Proprietor: GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP., LTD.
Wusha, Chang'an Dongguan, Guangdong 523860 (CN)

(72)Inventors:
  • ZHANG, Lei
    Dongguan, Guangdong 523860 (CN)
  • SUN, Jingyang
    Dongguan, Guangdong 523860 (CN)
  • YANG, Le
    Dongguan, Guangdong 523860 (CN)

(74)Representative: Manitz Finsterwald Patent- und Rechtsanwaltspartnerschaft mbB 
Martin-Greif-Strasse 1
80336 München
80336 München (DE)


(56)References cited: : 
GB-A- 2 552 090
US-A1- 2012 287 327
US-A1- 2009 278 913
  
      
    Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


    Description

    CROSS-REFERENCE TO RELATED APPLICATION



    [0001] This application claims priority to a Chinese application No. 201810564121.7 filed on June 4, 2018, titled "ELECTRONIC DEVICE, IMAGE PROCESSING METHOD AND DEVICE" and a Chinese application No. 201820859848.3 filed on June 4, 2018, titled "ELECTRONIC DEVICE".

    TECHNICAL FIELD



    [0002] The present disclosure relates to mobile terminals, and particularly to a storage medium, an electronic device, and an image processing method.

    BACKGROUND ART



    [0003] With the development of mobile terminal, a front camera is disposed in a mobile terminal, and generally disposed under the display screen of the mobile terminal. Because demand for the screen size is increasing, screen-to-body ratio of the mobile terminals is higher in the same size, and the screen display of the mobile terminal is gradually become full display screen.

    [0004] US 2009/0278913 A1 provides a gaze accurate video conferencing system includes a screen that alternates between a light-scattering state and a substantially transparent state. A camera is positioned behind the screen and is configured to capture images of a user positioned in front of the screen when the screen is in its substantially transparent state. When the screen is in its substantially light-scattering state, a projector projects a display image on the screen.

    [0005] In the related art, for a mobile terminal with full screen, a portion of the full screen is hollowed out or transparent and the front camera is disposed under the portion to realize the shooting function of the front camera. But display an image with full screen and image capture with the front camera can't happen simultaneously.

    DISCLOSURE OF THE INVENTION



    [0006] The present disclosure aims to solve at least one of the technical problems in the related art to some extent. The invention is defined in the independent claims.

    [0007] Therefore, the present disclosure provides an electronic device according to the appended claims 1-7 including a processing unit, a camera and a display screen covering the camera. The camera comprises a plurality of arrayed camera units and includes a projection assembly and an imaging assembly, so that the camera can project a second image under the control of the processing unit in a display state, and can capture third images in an imaging state. The coexistence of the full screen display function and the image acquisition function is realized, and the screen-to-body ratio in the electronic device is truly maximized.

    [0008] The present disclosure provides an image processing method according the appended claims 8-14.

    [0009] The present disclosure provides a computer readable storage medium according to appended claim 15.

    BRIEF DESCRIPTION OF THE DRAWINGS



    [0010] The above described and/or additional aspects and advantages of the present disclosure will become apparent and readily understood, wherein:

    FIG. 1 is a schematic structural view of an electronic device according to an embodiment of the present disclosure;

    FIG. 2 is a schematic structural view of another electronic device according to an embodiment of the present disclosure;

    FIG. 3 is a schematic view showing data flow in an image buffer according to an embodiment of the present disclosure;

    FIG. 4 is a first schematic view of a division of sub-images according to an embodiment of the present disclosure;

    FIG. 5 is a second schematic view of a division of sub-images according to an embodiment of the present disclosure;

    FIG. 6 is a schematic structural view of still another electronic device according to an embodiment of the present disclosure;

    FIG. 7 is an enlarged schematic view of the camera in the electronic device according to an embodiment of the present disclosure;

    FIG. 8 is a schematic structural view of still another electronic device according to an embodiment of the present disclosure;

    FIG. 9 is a schematic flowchart view of the image processing method according to an embodiment of the present disclosure;

    FIG. 10 is a schematic flowchart view of another image processing method according to an embodiment of the present disclosure; and

    FIG. 11 is a schematic structural view of an image processing device according to an embodiment of the present disclosure.


    DETAILED DESCRIPTION OF EMBODIMENTS



    [0011] The embodiments of the present disclosure are described in detail below, and the examples of the embodiments are illustrated in the drawings, wherein the same or similar reference numerals are used to refer to the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are intended to be illustrative, and are not to be construed as limitations to the disclosure.

    [0012] Technologies of full screen are used in electronic devices more and more. Currently, for electronic devices equipped with front cameras, solutions of full screen are as follows.

    [0013] In a first solution, a transparent area corresponding to a camera is set on a full screen. The solution causes the area corresponding to the camera on the full screen cannot display an image.

    [0014] In a second solution, the full screen is made into a special-shaped screen to leave a space corresponding to the camera. In this solution, the full screen is poor in appearance, the screen utilization around the camera is low, the screen size is wasted, and the high ratio of the full screen is not realized.

    [0015] In order to solve the above problems, the present disclosure provides an electronic device, through disposing a projection assembly and an imaging assembly in the electronic device to realize a full screen solution in the electronic device. A display function and an image capturing function can be coexisted in the area corresponding to the camera, the electronic device has nice appearance, and realizes a maximization full screen ratio in the electronic device.

    [0016] The electronic device, and an image processing method and device provided by embodiments of the present disclosure are described below with reference to the accompanying drawing.

    [0017] FIG. 1 is a schematic structural view of an electronic device according an embodiment of the present disclosure. The electronic device may include a mobile terminal or a desktop terminal, such as a mobile phone, an iPad, a palmtop computer, a desktop computer, and the like.

    [0018] As illustrated in FIG. 1, the electronic device may include a processing unit 10, a camera 20 and a display screen 30 covering the camera 20.

    [0019] The display screen 30 is electrically connected to the processing unit 10. The display screen 30 may include a light transmission area 301 and a non-light transmission area 302, and the display screen 30 may be configured to display a first image on the non-light transmission area 302 under the control of the processing unit 10. Wherein, the light transmission area 301 may be rectangular, circular, or other shapes, and is not limited in this embodiment. In addition, for the position of the light transmission area 301, the person having ordinary skill in the art can freely dispose the light transmission area 301 in different position of the display screen 30 according to the requirements of the product, it is not limited in this embodiment.

    [0020] The camera 20 is correspondingly disposed under the light transmission area 301, and includes a projection assembly 202 electrically connected to the processing unit 10 and an imaging assembly 201 electrically connected to the processing unit 10. Under the control of processing unit 10, the projection assembly 202 is configured to project a second image on the display screen 30 thereby to be displayed on the light transmission area 301 of the display screen 30. The imaging assembly 201is also configured to capture a third image through the light transmission area 301 of the display screen 30 under the control of the processing unit 10.

    [0021] The processing unit 10 is configured to control the display screen 30 to display the first image, and control the camera 20 to project the second image in a display state and to capture the third image in an imaging state.

    [0022] In the electronic device according to the embodiment of the present disclosure, the display screen is electrically connected to the processing unit, and the display screen includes the light transmission area and the non-light transmission area. The camera is disposed under the light transmission area, comprises a plurality of arrayed camera units and each camera unit includes a projection assembly electrically connected to the processing unit and an imaging assembly electrically connected to the processing unit, and is configured to project the second image to the display screen by the projection assembly under the control of processing unit, so as to be displayed on the light transmission area of the display screen, and also is configured to capture the third image through the light transmission area of the display screen by the imaging assembly under the control of the processing unit. The processing unit is configured to control the display screen to display the first image, and control the camera to project the second image in the display state and to capture the third image in the imaging state. By setting the projection assembly and the imaging assembly in the camera, in the display state, the projection assembly can project the second image on the display screen under the control of the processing unit, so that the light transmission area of the display screen can also display the first image, thereby full screen display can be realized; in the imaging state, the imaging assembly can capture the third image through the control of the processor, so that the image capturing function can be realized, the full screen display and the image capturing function of the electronic device can coexist, and the ratio of the full screen in the electronic device is maximized.

    [0023] Based on the foregoing embodiment, the embodiment of the present disclosure provides another possible implementation of another electronic device, and FIG. 2 is a schematic structural view of the another electronic device according an embodiment of the present disclosure.

    [0024] As shown in FIG. 2, the processing unit 10 may include an image buffer 101, a processor 102, a first communication interface 103 electrically connected to the camera 20, and a second communication interface 104 electrically connected to the display screen 30.

    [0025] The image buffer 101 is configured to separately buffer a display sub-image and a projection sub-image divided from a fourth image. The fourth image is configured to be displayed on the display screen, wherein the display sub-image is configured to displayed on the non-light transmission area and the projection sub-image is configured to be projected on the light transmission area by the camera to be displayed on the light transmission area.

    [0026] The processor 102, is electrically connected to the image buffer 101, and is configured to read the image buffer 101 to obtain the display sub-image and the projection sub-image, and to transmit the projection sub-image to the first communication interface 103 and to transmit the display sub-image to the second communication interface 104.

    [0027] The first communication interface 103, is electrically connected to the processor 102, and is configured to transmit the projection sub-image as the second image to the camera 20. That is, the projection sub-image is projected as the second image on the light transmission area, or the projection sub-image is the second image. The second communication interface 104, is electrically connected to the processor 102, and is configured to transmit the display sub-image as the first image to the display screen 30. That is, the display sub-image is displayed as the first image on the non-light transmission area, or the display sub-image is the first image. As a possible embodiment, the first communication interface 103 and the second communication interface 104 are display serial interfaces (DSIs) for synchronous transmission. In the display state, the projection sub-image transmitted to the camera 20 by the first communication interface 103 and the display sub-image transmitted to the display screen 30 by the second communication interface 104 can be transmitted synchronously, through the DSIs for synchronous transmission, thereby the projection sub-image and the display sub-image are displayed in the display screen 30synchronously.

    [0028] As a possible embodiment, the image buffer 101 may include a first buffer 1011 and a second buffer 1012, wherein the first buffer 1011 is configured to store the projection sub-image, and the second buffer 1012 is configured to store the display sub-image. FIG. 3 is a schematic view showing data flow in the image buffer according to an embodiment of the present disclosure. As shown in FIG. 3, by dividing different buffer areas, image information corresponding to image pixels in the non-light transmission area of the display screen, that is, the display sub-image, is stored in the first buffer 1011; image information corresponding to image pixels in the light transmission area of the display screen, that is, the projection sub-image, is stored in the second buffer 1012. As such, a corresponding relationship between the storage location and the fourth image in the display screen display area is established. So, when the camera is in the display state, the processor 10 can directly read the projection sub-image from the first buffer 1011 of the image buffer 101, transmit the projection sub-image through the first communication interface 103 to the camera 20 and further to be projected, and read the display sub-image from the second buffer 1012 and transmit the display sub-image through the second communication interface 104 to the display screen 30 to be displayed, thereby improving the efficiency of full screen display.

    [0029] The projection sub-image and the display sub-image stored in the image buffer 101 are obtained by dividing the fourth image, and the fourth image may be divided according to an image shape corresponding to the image pixels corresponding to the non-light transmission area 302 of the display screen 30 and an image shape corresponding to the image pixels corresponding to the light transmission area 301 of the display screen 30. As a possible embodiment of division, FIG. 4 is a schematic view of a division of sub-images according to an embodiment of the present disclosure. As illustrated in FIG. 4, the image corresponding to the image pixels of the light transmission area 301 of the display screen 30 is square, so that the fourth image can be divided according to the shape of the images corresponding to the image pixels corresponding to the light transmission 301 and the non-light transmission area 302 of the display screen 30, thereby to obtain the projection sub-image and the display sub-image as illustrated on right side of FIG. 4. As another possible embodiment of division, FIG. 5 is another schematic view of a division of sub-images according to an embodiment of the present disclosure. As illustrated in FIG. 5, the shape of the images corresponding to the image pixels of the light transmission area 301 of the display screen 30 is circular, so that the fourth image can be divided according to the shape of the images corresponding to the image pixels corresponding to the light transmission 301 and the non-light transmission area 302 of the display screen 30, thereby to obtain the projection sub-image and the display sub-image as illustrated on right side of FIG. 5. It should be noted that, FIG. 4 and FIG. 5 just show two kinds of division of the projection sub-image and the display sub-image according to the images corresponding to the image pixels of the light transmission area 301 and the non-light transmission area 302 of the display screen 30. In actual applications, a person skilled in the art may divide the fourth image according to the image shapes corresponding to the image pixels of the light transmission area 301 and the non-light transmission area 302 of the display screen 30, with a similar division principle, and more examples are not enumerated in this embodiment.

    [0030] In the electronic device of the embodiment of the present disclosure, by setting the projection assembly and the imaging assembly in the camera, in the display state, the projection assembly can project the projection sub-image to the display screen under the control of the processing unit, so that the light transmission area of the display screen can also display the display sub-image, thereby full screen display can be realized; in the imaging state, the imaging assembly can capture the third image under the control of the processor, so that the image capturing function can be realized, the full screen display and the image capturing function of the electronic device can coexist, and the ratio of the full screen in the electronic device is truly maximized and realized. In addition, in the display state, the projection sub-image transmitted to the camera by the first communication interface and the display sub-image transmitted to the display screen by the second communication interface can be transmitted synchronously, through the DSIs for synchronous transmission, thereby the projection sub-image and the display sub-image can be displayed in the display screen synchronously.

    [0031] Based on the above embodiments, an additional electronic device is provided in an embodiment of the present disclosure. FIG. 6 is a schematic structural view of the additional electronic device according to an embodiment of the present disclosure. As shown in FIG. 6, based on the previous embodiment, a projection assembly 202 includes a color film layer 2021, a metal wiring layer 2022, a light emitting element 2023, and a driving element 2024.

    [0032] The driving element 2024 is disposed on a surface of a substrate 2025, and is configured to drive the light emitting element 2023.

    [0033] The metal wiring layer 2022 is disposed on a surface of the driving element 2024, and is electrically connected to the driving element 2024 and the light emitting element 2023.

    [0034] The light emitting element 2023 is disposed on a surface of the metal wiring layer 2022, and is configured to emit light under the driving of the driving element 2024.

    [0035] The color film layer 2021 is disposed on a surface of the light emitting element 2023, and is configured to filter lights.

    [0036] As a possible embodiment, the light emitting element 2023 may further include a metal anode 20231, an organic polymer light emitting layer 20232, and a transparent cathode 20233.

    [0037] The metal anode 20231 is disposed on the surface of the metal wiring layer 2022.

    [0038] The organic polymer light emitting layer 20232 is disposed on a surface of the metal anode 20231.

    [0039] The transparent cathode 20233 is disposed on a surface of the organic polymer light emitting layer 20232.

    [0040] As a possible embodiment, an imaging assembly 201 and the projection assembly 202 share the same substrate 2025. The imaging assembly 201 includes a photosensitive element 2011, the metal wiring layer 2022 shared with the projection assembly 202, and the color film layer 2021 shared with the projection assembly 202.

    [0041] The photosensitive element 2011 is disposed on the surface of the substrate 2025, is configured to detect the light filtered by the color film layer 2021, and is electrically connected to the metal wiring layer 2022.

    [0042] As shown in FIG. 7, the camera 20 includes a plurality of arrayed camera units 210. Each camera unit 210 includes at least one projection assembly 202 and at least one imaging assembly 201, the projection assembly 202 and the imaging assembly 201 are adjacent. Correspondingly, the plurality of camera units 210 include a plurality of projection assemblies 202 and a plurality of imaging assemblies 201. As shown in FIG. 8, the camera 30 is disposed under the light transmission area 301, each projection assembly 202 is configured to project one pixel of the second image on the light transmission area 301, so that the plurality of projection assemblies 202 can project the second image on the light transmission area 301. Each imaging assembly 201 is configured to capture one pixel of the third image, so that the plurality of imaging assemblies 201 can capture the third image through the light transmission area 301.

    [0043] In a possible scene, when the electronic device is in the display state, the projection assembly 202 is activated to project the second image. There may be a plurality of projection assemblies 202, each projection assembly 202 corresponds to one pixel of the second image, and the pixel of the second image is determined according to the second image. For convenience of description, one projection assembly is taken as an example for description in the embodiment. Specifically, as shown in FIG. 6, in the projection assembly 202, a voltage is applied to the light emitting element 2023 under the driving signal of the driving element 2024. For example, the light emitting element 2023 may be an organic light-emitting diode (OLED), and the current flows from the transparent cathode 20233 to the metal anode 20231 and passes through the organic polymer light emitting layer 20232. That is, the transparent cathode 20233 outputs electrons to the organic polymer light emitting layer 20232, the metal anode 20231 outputs holes to the organic polymer light emitting layer 20232, the electrons will combine with the holes in the organic polymer light emitting layer 20232 when the electrons reach the organic polymer light emitting layer 20232. The electron is negatively charged and the hole is positively charged, and they attract each other to excite organic materials of the organic polymer light emitting layer 20232 to emit, thereby realizing the light emission of the light emitting element 2023. Further, the light can be filtered when passes through the color film layer 2021 to generate three primary colors of red, green, and blue, and different filtering requirements can be realized according to the color setting of the filter of the color film layer 2021, thereby the display of the second image can be realized.

    [0044] In another possible scene, when the electronic device is in the imaging state, the imaging assembly 201 can be activated to capture the third image. There may be a plurality of imaging assemblies 201, each imaging assembly 202 corresponds to one pixel of the third image, and the pixel of the third image is configured to generate the third image. For convenience of description, one imaging assembly is taken as an example for description in the embodiment. Specifically, as shown in FIG. 6, light can be filtered when passes through the color film layer 2021, the photosensitive element 2011 is configured to detect the light filtered by the color film layer 2021, so that the photosensitive element 2011 to transfer the detected light from optical information to digital signal, thereby the R, G, and B value of the pixel of the third image corresponding to the imaging assembly 201 can be determined, that is, the third image can be determined, and the third image can be generated according to the pixels corresponding to a plurality of imaging assemblies 201.

    [0045] In the electronic device of the embodiment of the present disclosure, the camera includes a plurality of camera units, and each camera unit includes at least one projection assembly and at least one imaging assembly. In the display state, the projection assembly can project to the display screen under the control of the processing unit, so that the light transmission area of the display screen can also display the first image, thereby full screen display can be realized; in the imaging state, the imaging assembly can capture the third image in the control of the processor, so that the image capture function can be realized, the full screen display and the image capture of the electronic device can coexist. And, in the display state, the projection sub-image transmitted to the camera by the first communication interface and the display sub-image transmitted to the display screen by the second communication interface can be transmitted synchronously, through the DSIs for synchronous transmission, thereby the projection sub-image and the display sub-image displayed in the display screen can be displayed synchronous. In addition, by setting the projection assembly and the imaging assembly in the camera and sharing components, the size of the camera is prevented from being excessively large, thereby the overall size of the electronic device can be reduced, and the cost can be reduced.

    [0046] Based on the above embodiments, the embodiment of the present disclosure further provides an image processing method. FIG. 9 is a schematic flowchart of the image processing method according to an embodiment of the present disclosure, and the method can be specifically performed by the processor of the electronic device described above. As shown in FIG. 9, the method includes the following blocks.

    [0047] In block 701, the camera is controlled to be in the display state or in the imaging state.

    [0048] Specifically, the processing unit controls the camera to be in the display state when the display screen needs to be displayed, and the processing unit controls the camera to be in the imaging state when the camera needs to capture an image.

    [0049] In block 702, the camera is controlled to project the second image when the camera is in the display state.

    [0050] Specifically, when the camera is in the display state, the processing unit obtains the fourth image, divides the fourth image into the display sub-image and the projection sub-image, and transmits the display sub-image to the display screen, transmits the projection sub-image to the camera.

    [0051] In block 703, the camera is controlled to capture the third image when the camera is in the imaging state.

    [0052] In the image processing method of the embodiment provided by the present disclosure, the camera can be controlled to be in the imaging state or the display state. The camera can be controlled to project the second image when the camera is in the projection state, and the camera can be controlled to capture the third image when the camera is in the imaging state, thereby the coexistence of the full screen display and the image capture in the electronic device is realized, and the screen-to-body ratio in the electronic device is truly maximized.

    [0053] Based on the above embodiments, the embodiment of the present disclosure further provides another image processing method. FIG. 10 is another schematic flowchart of another image processing method according to an embodiment of the present disclosure. As shown in FIG. 10, based on the above embodiments, the block 702 may further include the following blocks.

    [0054] In block 7021, the fourth image is obtained.

    [0055] Specifically, the processor reads the fourth image from the image buffer.

    [0056] In block 7022, the fourth image is divided into the display sub-image and the projection sub-image.

    [0057] Specifically, the image buffer is divided into the first buffer and the second buffer, the first buffer is configured to store the projection sub-image, and the second buffer is configured to store the display sub-image. When buffering the fourth image in the image buffer, the fourth image is divided into the display sub-image and the projection sub-image, and the display sub-image is buffered in the second buffer and the projection sub-image buffered in the first buffer. When the camera is in the display state, the processor determines the display sub-image from the fourth image according to the image pixels corresponding to the non-light transmission area of the display screen, and the processor determines the projection sub-image from the fourth image according to the image pixels corresponding to the light transmission area of the display screen.

    [0058] In block 7023, the display sub-image is transmitted to the display screen and the projection sub-image is transmitted to the camera.

    [0059] Specifically, the display sub-image is carried by a first transmission signal, and the projection sub-image is carried by a second transmission signal. The processor synchronizes the first transmission signal of the display screen and the second transmission signal of the camera, and after the synchronization, transmits the synchronized first transmission signal to the display screen and transmits the synchronized second transmission signal to the camera. As a possible embodiment, the first transmission signal and the second transmission signal can be synchronized by the DSI for synchronous transmission, and after the synchronization, the synchronized first transmission signal can be transmitted to the display screen and the synchronized second transmission signal can be transmitted to the camera after synchronization.

    [0060] In the image processing method provided by the embodiments of the present disclosure, the camera is controlled to project the second image when the camera is in the display state, the camera is controlled to capture the third image when the camera is in the imaging state, so that the coexistence of the full screen display and the image capture function of the electronic device can be realized. Simultaneously, the image buffer is divided into storage units corresponding to the display sub-image and the projection sub-image, and when the camera is in the display state, the first transmission signal carrying the display sub-image and the second transmission signal carrying the projection sub-image are synchronized, and transmitted to the corresponding display screen and camera. The image displayed on the light transmission area of the display screen corresponding to the camera and the image displayed on the non-light transmission corresponding the display area of the display screen are completely synchronized, thereby the poor user experience caused by the image display being out of sync is avoided.

    [0061] In order to implement the above embodiments, the present disclosure also provides an image processing device.

    [0062] FIG. 11 is a schematic structural view of the image processing device according to an embodiment of the present disclosure.

    [0063] As shown in FIG. 11, the device includes a state control module 91, a projection control module 92 and an imaging control module 93.

    [0064] The state control module 91 is configured to control the camera in the display state or in the imaging state.

    [0065] The projection control module 92 is configured to control the camera to project the second image when the camera is in the display state.

    [0066] The imaging control module 93 is configured to control the camera to capture the third image when the camera is in the imaging state.

    [0067] Further, in a possible embodiment of the present disclosure, the projection control module 92 may further include an obtaining unit, a dividing unit and a transmitting unit.

    [0068] The obtaining unit is configured to obtain the fourth image.

    [0069] The dividing unit is configured to divide the fourth image into the display sub-image and the projection sub-image.

    [0070] The transmitting unit is configured to transmit the display sub-image to the display screen and transmit the projection sub-image to the camera.

    [0071] In a possible embodiment, the dividing unit is specifically configured to:

    determine the display sub-image from the fourth image according to the image pixels corresponding to the non-light transmission area of the display screen;

    and determine the projection sub-image from the fourth image according to the image pixels corresponding to the light transmission area projected by the camera.



    [0072] In a possible embodiment, the transmitting unit is specifically configured to:

    synchronize the first transmission signal of the display screen and the second transmission signal of the camera, wherein the first transmission signal carries the display sub-image, and the second transmission signal carries the projection sub-image,

    transmit the synchronized first transmission signal to the display screen, and transmit the synchronized second transmission signal to the camera.



    [0073] It should be noted that the foregoing description of the method embodiments is also applicable to the device of the embodiments, and the implementation principles are similar, and details are not described herein again.

    [0074] In the image processing device provided by the embodiments of the present disclosure, the camera is controlled to project the second image when the camera is in the display state, the camera is controlled to capture the third image when the camera is in the imaging state, so that the coexistence of the full screen display and the image capture function of the electronic device can be realized. Simultaneously, the image buffer is divided into storage units corresponding to the display sub-image and the projection sub-image, and when the camera is in the display state, the first transmission signal carrying the display sub-image and the second transmission signal carrying the projection sub-image are synchronized, and transmitted to the corresponding display screen and camera. The image displayed on the light transmission area of the display screen corresponding to the camera and the image displayed on the non-light transmission corresponding the display area of the display screen are completely synchronized, thereby the poor user experience caused by the image display being out of sync is avoided.

    [0075] In order to implement the above embodiments, the present disclosure also proposes an electronic device including a memory, the processor, and a computer program stored in the memory and executable in the processor, and the image processing method as described in the foregoing method embodiments is implemented when the processor executes the computer program.

    [0076] In order to implement the above embodiments, the present disclosure also proposes a computer readable storage medium, stored a computer program, and the image processing method as described in the foregoing method embodiment is implemented when the program is executed by the processor.

    [0077] In the description of the present specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" and the like means a specific feature, a structure, material or characteristic included in at least one embodiment or example of the disclosure described in connection with the embodiment or example. In the present specification, the schematic representation of the above terms is not necessarily directed to the same embodiment or example. Furthermore, the specific feature, the structure, the material, or the characteristic described may be combined in a suitable manner in any one or more embodiments or examples. In addition, various embodiments or examples and features of various embodiments or examples described in the specification may be combined without contradicting each other.

    [0078] Moreover, the terms "first" and "second" are used for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, features defining "first" and "second" may include at least one of the features, either explicitly or implicitly. In the description of the present disclosure, the meaning of "a plurality" is at least two, such as two, three, etc., unless specifically defined otherwise.

    [0079] Any process or method description in the flowcharts or otherwise described herein may be understood to represent comprising one or more module, segment or portion of code executable instructions for implementing the steps or processes of a custom logic function. And the scope of the preferred embodiments of the present disclosure includes additional implementations, which may not be in the order shown or discussed, and may include performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved. This should be understood by those skilled in the art to which the embodiments of the present disclosure pertain.

    [0080] The logic and/or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer readable medium, used in an instruction execution system, apparatus, or device (eg, a computer-based system, a system including a processor, or other system that can fetch instructions and execute instructions from an instruction execution system, apparatus, or device), or used in conjunction with such instruction execution systems, devices, or devices. For the purposes of this specification, the "computer readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport a program for use in an instruction execution system, an apparatus, or a device, or in conjunction with the instruction execution system, the apparatus, or the device. More specific examples (non-exhaustive list) of computer readable medium include the following: electrical connections (electronic devices) having one or more wires, portable computer disk cartridges (magnetic devices), random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optic devices, and portable compact disk read only memory (CDROM). In addition, the computer readable medium may even be a paper or other suitable medium on which the program can be printed, as the paper or other suitable medium may be optically scanned, processed to obtain the program electronically by editing, interpretation or, if appropriate, other suitable method, and then the program can be stored in a computer memory.

    [0081] It should be understood that portions of the disclosure can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods may be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware and as in another embodiment, it can be implemented by any one or combination of the following techniques well known in the art: discrete logic circuits with logic gates for implementing logic functions on data signals, specific integrated circuits with suitable combinational logic gates, programmable gate array (PGA), field programmable gate arrays (FPGA), and the like.

    [0082] Persons with ordinary skill in the art can understand that all or part of the steps carried by the method of implementing the above embodiments can be completed by a program to instruct related hardware, and the program can be stored in a computer readable storage medium, and when the program executed, the program may include one or a combination of the steps of the method embodiments.

    [0083] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. The integrated modules, if implemented in the form of software functional modules and sold or used as stand-alone products, may also be stored in a computer readable storage medium.

    [0084] The storage medium above mentioned may be a read only memory, a magnetic disk or an optical disk or the like. While the embodiments of the present disclosure have been shown and described above, it is understood that the above-described embodiments are illustrative and are not to be construed as limiting the scope of the present disclosure. Variations, modifications, alterations and variations of the above-described embodiments may be made by those skilled in the art within the scope of the present disclosure. disclosure as defined by the appended claims.


    Claims

    1. An electronic device, comprising a processing unit (10), a camera (20), and a display screen (30) covering the camera (20);

    wherein the display screen (30) is electrically connected to the processing unit (10), the display screen (30) comprises a light transmission area (301) and a non-light transmission area (302), and is configured to display a first image on the non-light transmission area (302) under the control of the processing unit (10);
    characterized in that

    the camera (20) is disposed under the light transmission area (301) and comprises a plurality of arrayed camera units (210), each camera unit (210) comprises at least one projection assembly (202) and at least one adjacent imaging assembly (201), each projection assembly (202) and each imaging assembly (201) are electrically connected to the processing unit (10), each projection assembly (202) is configured to project one pixel of a second image on the light transmission area (301) of the display screen (30) thereby the projection assemblies (202) of the plurality of arrayed camera units (210) project the second image on the light transmission area (301) of the display screen (30) under the control of the processing unit (10), and each imaging assembly (201) is configured to capture one pixel of a third image through the light transmission area (301) of the display screen (30), thereby the imaging assemblies (201) of the plurality of arrayed camera units (210) capture the third image through the light transmission area (301) of the display screen (30) under the control of the processing unit (10); and

    wherein the processing unit (10) is configured to control the display screen (30) to display the first image, and is configured to control the camera (20) to project the second image in a display state and to capture the third image in an imaging state.


     
    2. The electronic device of claim 1, wherein the processing unit (10) comprises an image buffer (101), a processor (102), a first communication interface (103) electrically connected to the camera (20), and a second communication interface (104) electrically connected to the display screen (30);

    wherein the image buffer (101) is configured to separately buffer a display sub-image and a projection sub-image obtained by dividing a fourth image;

    wherein the processor (102) is electrically connected to the image buffer (101), and is configured to read the image buffer (101) to obtain the display sub-image and the projection sub-image, and transmit the projection sub-image to the first communication interface (103) and transmit the display sub-image to the second communication interface (104);

    wherein the first communication interface (103) is electrically connected to the processor (102) and configured to transmit the projection sub-image as the second image to the camera (20); and

    wherein the second communication interface (104) is electrically connected to the processor (102) and configured to transmit the display sub-image as the first image to the display screen (30); wherein preferably

    the image buffer (101) comprises a first buffer (1011) and a second buffer (1012),

    wherein the first buffer (1011) is configured to store the projection sub-image, and the second buffer (1012) is configured to store the display sub-image; and/or

    the first communication interface (103) and the second communication interface (104) are display serial interfaces (DSIs) for synchronous transmission.


     
    3. The electronic device of claim 1 or 2, wherein the projection assembly (202) comprises a color film layer (2021), a metal wiring layer (2022), a light emitting element (2023), and a driving element (2024);

    the driving element (2024) is disposed on a surface of a substrate (2025) and configured to drive the light emitting element (2023) to emit light;

    the metal wiring layer (2022) is disposed on a surface of the driving element (2024), and is electrically connected to the driving element (2024) and the light emitting element (2023);

    the light emitting element (2023) is disposed on a surface of the metal wiring layer (2022), and is configured to emit light under the driving of the driving element (2024); and

    the color film layer (2021) is disposed on a surface of the light emitting element (2023) and configured to filter lights.


     
    4. The electronic device of claim 3, wherein the imaging assembly (201) and the projection assembly (202) share the substrate (2025).
     
    5. The electronic device of claim 3 or 4, wherein the imaging assembly (201) comprises a photosensitive element (2011), the metal wiring layer (2022) shared with the projection assembly (202), and the color film layer (2021) shared with the projection assembly (202);

    the photosensitive element (2011) is disposed on the surface of the substrate (2025), and configured to detect the light filtered by the color film layer (2021); and

    the metal wiring layer (2022) is electrically connected to the photosensitive element (2011).


     
    6. The electronic device of any one of claims 3-5, wherein light emitting element (2023) comprises:

    a metal anode (20231) disposed on the surface of the metal wiring layer (2022);

    an organic polymer light emitting layer (20232) disposed on a surface of the metal anode (20231); and

    a transparent cathode (20233) disposed on a surface of the organic polymer light emitting layer (20232).


     
    7. The electronic device of any one of claims 1-6, wherein each projection assembly (202) corresponds to one pixel of the second image, and the pixel of the second image is determined according to the second image, and
    each imaging assembly (201) corresponds to one pixel of the third image, and the pixel of the third image is configured to generate the third image.
     
    8. An image processing method, characterized by comprising:

    controlling a camera (20) in a display state or in an imaging state, wherein the camera (20) comprises a plurality of arrayed camera units (210), each camera unit (210) comprises at least one projection assembly (202) and at least one adjacent imaging assembly (201);

    controlling the camera (20) to project a second image when the camera (20) is in the display state, wherein each projection assembly (202) corresponds to one pixel of the second image, and is configured to project the pixel of the second image on a display screen (30) covering the camera (20), thereby the projection assemblies (202) of the plurality of arrayed camera units (210) project the second image on the display screen (30); and

    controlling the camera (20) to capture a third image when the camera (20) is in the imaging state, wherein each imaging assembly (201) corresponds to one pixel of the third image, and is configured to capture the pixel of the third image through the display screen (30), thereby the imaging assemblies (201) of the plurality of arrayed camera units (210) capture the third image through the display screen (30).


     
    9. The method of claim 8, wherein before controlling the camera (20) to project a second image, the method further comprises:

    obtaining a fourth image;

    dividing the fourth image into a display sub-image and a projection sub-image; and

    transmitting the display sub-image as a first image to the display screen (30) and transmitting the projection sub-image as the second image to the camera (20); wherein preferably

    before transmitting the display sub-image as a first image to the display screen (30) and transmitting the projection sub-image as the second image to the camera (20), the method further comprises: storing the projection sub-image in a first buffer (1011), and storing the display sub-image in a second buffer (1012); reading the projection sub-image from the first buffer (1011), and reading the display sub-image from the second buffer (1012); and/or

    transmitting the display sub-image as a first image to the display screen (30) and transmitting the projection sub-image as the second image to the camera (20) comprises:

    synchronizing a first transmission signal of the display screen (30) and the second transmission signal of the camera (20), wherein the first transmission signal carries the display sub-image, and the second transmission signal carries the projection sub-image;

    transmitting the synchronized first transmission signal to the display screen (30) and transmitting the synchronized second transmission signal to the camera (20); and/or

    the processing unit (10) comprises a first communication interface (103) electrically connected to the camera (20), a second communication interface (104) electrically connected to the display screen (30), and the first communication interface (103) and the second communication interface (104) are display serial interfaces (DSIs) for synchronous transmission,

    wherein transmitting the display sub-image as a first image to the display screen (30) and transmitting the projection sub-image as the second image to the camera (20) comprises:

    transmitting the projection sub-image as the second image to the camera (20) by the first communication interface (103), and

    transmitting the display sub-image as the first image to the display screen (30) by the second communication interface (104).


     
    10. The method of claim 9, wherein dividing the fourth image into a display sub-image and a projection sub-image comprises:

    determining the display sub-image from the fourth image according to image pixels corresponding to the non-light transmission area (302) of the display screen (30); and

    determining the projection sub-image from the fourth image according to image pixels corresponding to the light transmission area (301) projected by the camera (20).


     
    11. The method of any one of claims 8-10, wherein the method is executable for a processing unit (10) of an electronic device, wherein the electronic device comprises the processing unit (10), the camera (20), a display screen (30) covering the camera (20), the camera (20) and the display screen (30) are electrically connected to the processing unit (10), the display screen (30) comprises a light transmission area (301) and a non-light transmission area (302), and the camera (20) is disposed under the light transmission area (301) and comprises a projection assembly (202) and an imaging assembly (201) both electrically connected to the processing unit (10),
    wherein controlling the camera (20) to project a second image when the camera (20) is in the display state, and controlling the camera (20) to capture a third image when the camera (20) is in the imaging state, comprise:

    controlling the projection assembly (202) to project the second image thereby to display the second image on the light transmission area (301), when the camera (20) is in the display state; and

    controlling the imaging assembly (201) to capture a third image through the light transmission area (301) when the camera (20) is in the imaging state.


     
    12. The method of claim 11, wherein the projection assembly (202) comprises a color film layer (2021), a light emitting element (2023), and a driving element (2024),
    wherein controlling the projection assembly (202) to project the second image comprises:

    driving the light emitting element (2023) to light by applying a voltage to the light emitting element (2023) under a driving signal of the driving element (2024);

    filtering the light to generate three primary colors of red, green, and blue when the light passes through the color film layer (2021), to realize a display of the second image according to a color setting of the filter of the color film layer (2021).


     
    13. The method of claim 12, wherein the light emitting element (2023) comprises a metal anode (20231), an organic polymer light emitting layer (20232), and a transparent cathode (20233),
    wherein the driving the light emitting element (2023) to light by applying a voltage to the light emitting element (2023) under a driving signal of the driving element (2024), comprises:

    applying the voltage to the light emitting element (2023) under the driving signal of the driving element (2024);

    exciting organic materials of the organic polymer light emitting layer (20232) to emit when a current flows from the transparent cathode (20233) to the metal anode (20231) and passes through the organic polymer light emitting layer (20232).


     
    14. The method of any one of claims 11-13, wherein the imaging assembly (201) comprises a photosensitive element (2011) and a color film layer (2021),
    wherein controlling the imaging assembly (201) to capture a third image comprises:

    filtering a light when the light passes through the color film layer (2021);

    detecting the light filtered by the color film layer (2021) by the photosensitive element (2011);

    determining R, G, and B value of a pixel of the third image corresponding to the imaging assembly (201) by transferring the detected light from optical information to digital signal by the photosensitive element (2011), to determine the third image.


     
    15. A computer readable storage medium, with a plurality of instructions stored therein, characterized in that an image processing method of any one of claims 8-14 is implemented when executing the plurality of instructions by a processor (102).
     


    Ansprüche

    1. Elektronische Vorrichtung, die eine Verarbeitungseinheit (10), eine Kamera (20) und einen die Kamera (20) abdeckenden Anzeigebildschirm (30) umfasst;

    wobei der Anzeigebildschirm (30) elektrisch mit der Verarbeitungseinheit (10) verbunden ist, der Anzeigebildschirm (30) einen Lichttransmissionsbereich (301) und einen Nicht-Lichttransmissionsbereich (302) umfasst und so ausgebildet ist, dass er ein erstes Bild auf dem Nicht-Lichttransmissionsbereich (302) unter der Steuerung der Verarbeitungseinheit (10) anzeigt;
    dadurch gekennzeichnet, dass,

    die Kamera (20) unter dem Lichttransmissionsbereich (301) angeordnet ist und eine Vielzahl von angeordneten Kameraeinheiten (210) umfasst,

    jede Kameraeinheit (210) mindestens eine Projektionsanordnung (202) und mindestens eine benachbarte Abbildungsanordnung (201) umfasst,

    jede Projektionsanordnung (202) und jede Abbildungsanordnung (201) elektrisch mit der Verarbeitungseinheit (10) verbunden ist,

    jede Projektionsanordnung (202) so ausgebildet ist, dass sie ein Pixel eines zweiten Bildes auf den Lichttransmissionsbereich (301) des Anzeigebildschirms (30) projiziert, wodurch die Projektionsanordnungen (202) der Vielzahl von angeordneten Kameraeinheiten (210) das zweite Bild auf den Lichttransmissionsbereich (301) des Anzeigebildschirms (30) unter der Steuerung der Verarbeitungseinheit (10) projizieren, und

    jede Abbildungsanordnung (201) so ausgebildet ist, dass sie ein Pixel eines dritten Bildes durch den Lichttransmissionsbereich (301) des Anzeigebildschirms (30) erfasst, wodurch die Abbildungsanordnungen (201) der mehreren angeordneten Kameraeinheiten (210) das dritte Bild durch den Lichttransmissionsbereich (301) des Anzeigebildschirms (30) unter der Steuerung der Verarbeitungseinheit (10) erfassen; und

    wobei die Verarbeitungseinheit (10) so ausgebildet ist, dass sie den Anzeigebildschirm (30) so steuert, dass er das erste Bild anzeigt, und so ausgebildet ist, dass sie die Kamera (20) so steuert, dass sie das zweite Bild in einem Anzeigezustand projiziert und das dritte Bild in einem Abbildungszustand erfasst.


     
    2. Elektronische Vorrichtung nach Anspruch 1, wobei die Verarbeitungseinheit (10) einen Bildpuffer (101), einen Prozessor (102), eine erste Kommunikationsschnittstelle (103), die elektrisch mit der Kamera (20) verbunden ist, und eine zweite Kommunikationsschnittstelle (104), die elektrisch mit dem Anzeigebildschirm (30) verbunden ist, umfasst; wobei der Bildpuffer (101) so ausgebildet ist, dass er ein Anzeige-Unterbild und ein Projektions-Unterbild, das durch Aufteilen eines vierten Bildes erhalten wird, getrennt puffert;

    wobei der Prozessor (102) elektrisch mit dem Bildpuffer (101) verbunden und so ausgebildet ist, dass er den Bildpuffer (101) liest, um das Anzeige-Unterbild und das Projektions-Unterbild zu erhalten, und das Projektions-Unterbild an die erste Kommunikationsschnittstelle (103) überträgt und das Anzeige-Unterbild an die zweite Kommunikationsschnittstelle (104) überträgt;

    wobei die erste Kommunikationsschnittstelle (103) elektrisch mit dem Prozessor (102) verbunden und so ausgebildet ist, dass sie das Projektions-Unterbild als zweites Bild an die Kamera (20) überträgt; und

    wobei die zweite Kommunikationsschnittstelle (104) elektrisch mit dem Prozessor (102) verbunden und so ausgebildet ist, dass sie das Anzeige-Unterbild als das erste Bild an den Anzeigebildschirm (30) überträgt; wobei der Bildpuffer (101) vorzugsweise einen ersten Puffer (1011) und einen zweiten Puffer (1012) umfasst, wobei der erste Puffer (1011) so ausgebildet ist, dass er das Projektions-Unterbild speichert, und der zweite Puffer (1012) so ausgebildet ist, dass er das Anzeige-Unterbild speichert; und/oder

    die erste Kommunikationsschnittstelle (103) und die zweite Kommunikationsschnittstelle (104) serielle Anzeigeschnittstellen (DSIs) für synchrone Übertragung sind.


     
    3. Elektronische Vorrichtung nach Anspruch 1 oder 2, wobei die Projektionsanordnung (202) eine Farbfilmschicht (2021), eine Metallverdrahtungsschicht (2022), ein lichtemittierendes Element (2023) und ein Ansteuerelement (2024) umfasst;

    das Ansteuerelement (2024) auf einer Oberfläche eines Substrats (2025) angeordnet und so ausgebildet ist, dass es das lichtemittierende Element (2023) ansteuert, um Licht zu emittieren;

    die Metallverdrahtungsschicht (2022) auf einer Oberfläche des Ansteuerelements (2024) angeordnet ist und elektrisch mit dem Ansteuerelement (2024) und dem lichtemittierenden Element (2023) verbunden ist;

    das lichtemittierende Element (2023) auf einer Oberfläche der Metallverdrahtungsschicht (2022) angeordnet und so ausgebildet ist, dass es unter der Ansteuerung des Ansteuerelements (2024) Licht emittiert; und

    die Farbfilmschicht (2021) auf einer Oberfläche des lichtemittierenden Elements (2023) angeordnet und so ausgebildet ist, dass sie Licht filtert.


     
    4. Elektronische Vorrichtung nach Anspruch 3, wobei die Abbildungsanordnung (201) und die Projektionsanordnung (202) das Substrat (2025) gemeinsam nutzen.
     
    5. Elektronische Vorrichtung nach Anspruch 3 oder 4, wobei die Abbildungsanordnung (201) ein lichtempfindliches Element (2011), die mit der Projektionsanordnung (202) geteilte Metallverdrahtungsschicht (2022) und die mit der Projektionsanordnung (202) geteilte Farbfilmschicht (2021) umfasst;

    das lichtempfindliche Element (2011) auf der Oberfläche des Substrats (2025) angeordnet und so ausgebildet ist, dass es das von der Farbfilmschicht (2021) gefilterte Licht erfasst; und

    die Metallverdrahtungsschicht (2022) elektrisch mit dem lichtempfindlichen Element (2011) verbunden ist.


     
    6. Elektronische Vorrichtung nach einem der Ansprüche 3-5, wobei das lichtemittierende Element (2023) Folgendes umfasst:

    eine Metallanode (20231), die auf der Oberfläche der Metallverdrahtungsschicht (2022) angeordnet ist;

    eine lichtemittierende Schicht aus organischem Polymer (20232), die auf einer Oberfläche der Metallanode (20231) angeordnet ist; und

    eine transparente Kathode (20233), die auf einer Oberfläche der lichtemittierenden Schicht aus organischem Polymer (20232) angeordnet ist.


     
    7. Elektronische Vorrichtung nach einem der Ansprüche 1-6, wobei jede Projektionsanordnung (202) einem Pixel des zweiten Bildes entspricht und das Pixel des zweiten Bildes entsprechend dem zweiten Bild bestimmt wird, und jede Abbildungsanordnung (201) einem Pixel des dritten Bildes entspricht und das Pixel des dritten Bildes so ausgebildet ist, dass es das dritte Bild erzeugt.
     
    8. Bildverarbeitungsverfahren, dadurch gekennzeichnet, dass es Folgendes umfasst:

    Steuern einer Kamera (20) in einem Anzeigezustand oder in einem Abbildungszustand, wobei die Kamera (20) eine Vielzahl von angeordneten Kameraeinheiten (210) umfasst, wobei jede Kameraeinheit (210) mindestens eine Projektionsanordnung (202) und mindestens eine benachbarte Abbildungsanordnung (201) umfasst;

    Steuern der Kamera (20), um ein zweites Bild zu projizieren, wenn sich die Kamera (20) im Anzeigezustand befindet, wobei jede Projektionsanordnung (202) einem Pixel des zweiten Bildes entspricht und so ausgebildet ist, dass sie das Pixel des zweiten Bildes auf einen die Kamera (20) abdeckenden Anzeigebildschirm (30) projiziert, wodurch die Projektionsanordnungen (202) der Vielzahl von angeordneten Kameraeinheiten (210) das zweite Bild auf den Anzeigebildschirm (30) projizieren; und

    Steuern der Kamera (20), um ein drittes Bild aufzunehmen, wenn sich die Kamera (20) im Abbildungszustand befindet, wobei jede Abbildungsanordnung (201) einem Pixel des dritten Bildes entspricht und so ausgebildet ist, dass sie das Pixel des dritten Bildes durch den Anzeigebildschirm (30) aufnimmt, wodurch die Abbildungsanordnungen (201) der Vielzahl der angeordneten Kameraeinheiten (210) das dritte Bild durch den Anzeigebildschirm (30) aufnehmen.


     
    9. Verfahren nach Anspruch 8, wobei das Verfahren vor der Steuerung der Kamera (20) zur Projektion eines zweiten Bildes ferner Folgendes umfasst:

    Erhalten eines vierten Bildes;

    Aufteilen des vierten Bildes in ein Anzeige-Unterbild und ein Projektions-Unterbild; und

    Übertragen des Anzeige-Unterbildes als erstes Bild an den Anzeigebildschirm (30) und Übertragen des Projektions-Unterbildes als zweites Bild an die Kamera (20);

    wobei das Verfahren vorzugsweise vor der Übertragung des Anzeige-Unterbildes als ein erstes Bild an den Anzeigebildschirm (30) und der Übertragung des Projektions-Unterbildes als das zweite Bild an die Kamera (20), ferner Folgendes umfasst:

    Speichern des Projektions-Unterbildes in einem ersten Puffer (1011), und Speichern des Anzeige-Unterbildes in einem zweiten Puffer (1012);

    Lesen des Projektions-Unterbildes aus dem ersten Puffer (1011) und Lesen des Anzeige-Unterbildes aus dem zweiten Puffer (1012); und/oder

    wobei das Übertragen des Anzeige-Unterbildes als ein erstes Bild an den Anzeigebildschirm (30) und das Übertragen des Projektions-Unterbildes als das zweite Bild an die Kamera (20) Folgendes umfasst:

    Synchronisieren eines ersten Übertragungssignals des Anzeigebildschirms (30) und des zweiten Übertragungssignals der Kamera (20), wobei das erste Übertragungssignal das Anzeige-Unterbild und das zweite Übertragungssignal das Projektions-Unterbild überträgt;

    Übertragen des synchronisierten ersten Übertragungssignals an den Anzeigebildschirm (30) und Übertragen des synchronisierten zweiten Übertragungssignals an die Kamera (20); und/oder wobei die Verarbeitungseinheit (10) eine erste Kommunikationsschnittstelle (103) umfasst, die elektrisch mit der Kamera (20) verbunden ist, eine zweite Kommunikationsschnittstelle (104), die elektrisch mit dem Anzeigebildschirm (30) verbunden ist, und die erste Kommunikationsschnittstelle (103) und die zweite Kommunikationsschnittstelle (104) serielle Anzeigeschnittstellen (DSI) für synchrone Übertragung sind,

    wobei das Übertragen des Anzeige-Unterbildes als ein erstes Bild an den Anzeigebildschirm (30) und das Übertragen des Projektions-Unterbildes als das zweite Bild an die Kamera (20) Folgendes umfasst:

    Übertragen des Projektions-Unterbildes als zweites Bild an die Kamera (20) über die erste Kommunikationsschnittstelle (103), und

    Übertragen des Anzeige-Unterbildes als erstes Bild an den Anzeigebildschirm (30) über die zweite Kommunikationsschnittstelle (104).


     
    10. Verfahren nach Anspruch 9, wobei das Aufteilen des vierten Bildes in ein Anzeige-Unterbild und ein Projektions-Unterbild Folgendes umfasst:

    Bestimmen des Anzeige-Unterbildes aus dem vierten Bild entsprechend den Bildpixeln, die dem Nicht-Lichttransmissionsbereich (302) des Anzeigebildschirms (30) entsprechen; und

    Bestimmen des Projektions-Unterbildes aus dem vierten Bild entsprechend den Bildpixeln, die dem von der Kamera (20) projizierten Lichttransmissionsbereich (301) entsprechen.


     
    11. Verfahren nach einem der Ansprüche 8-10, wobei das Verfahren für eine Verarbeitungseinheit (10) einer elektronischen Vorrichtung ausführbar ist,

    wobei die elektronische Vorrichtung die Verarbeitungseinheit (10), die Kamera (20), einen Anzeigebildschirm (30), der die Kamera (20) abdeckt, umfasst, die Kamera (20) und der Anzeigebildschirm (30) elektrisch mit der Verarbeitungseinheit (10) verbunden sind, der Anzeigebildschirm (30) einen Lichttransmissionsbereich (301) und einen Nicht-Lichttransmissionsbereich (302) umfasst, und die Kamera (20) unter dem Lichttransmissionsbereich (301) angeordnet ist und eine Projektionsanordnung (202) und eine Abbildungsanordnung (201) umfasst, die beide elektrisch mit der Verarbeitungseinheit (10) verbunden sind,

    wobei das Steuern der Kamera (20) zum Projizieren eines zweiten Bildes, wenn sich die Kamera (20) im Anzeigezustand befindet, und das Steuern der Kamera (20) zum Aufnehmen eines dritten Bildes, wenn sich die Kamera (20) im Abbildungszustand befindet, Folgendes umfassen:

    Steuern der Projektionsanordnung (202), um das zweite Bild zu projizieren und dadurch das zweite Bild auf dem Lichttransmissionsbereich (301) anzuzeigen, wenn sich die Kamera (20) im Anzeigezustand befindet; und

    Steuern der Abbildungsanordnung (201), um ein drittes Bild durch den Lichttransmissionsbereich (301) aufzunehmen, wenn sich die Kamera (20) im Abbildungszustand befindet.


     
    12. Verfahren nach Anspruch 11, wobei die Projektionsanordnung (202) eine Farbfilmschicht (2021), ein lichtemittierendes Element (2023) und ein Ansteuerelement (2024) umfasst,
    wobei das Steuern der Projektionsanordnung (202) zum Projizieren des zweiten Bildes Folgendes umfasst:

    Ansteuern des lichtemittierenden Elements (2023) zum Leuchten durch Anlegen einer Spannung an das lichtemittierende Element (2023) unter einem Ansteuersignal des Ansteuerelements (2024);

    Filtern des Lichts, um die drei Primärfarben Rot, Grün und Blau zu erzeugen, wenn das Licht die Farbfilmschicht (2021) durchläuft, um eine Anzeige des zweiten Bildes gemäß einer Farbeinstellung des Filters der Farbfilmschicht (2021) zu realisieren.


     
    13. Verfahren nach Anspruch 12, wobei das lichtemittierende Element (2023) eine Metallanode (20231), eine lichtemittierende Schicht aus organischem Polymer (20232) und eine transparente Kathode (20233) umfasst,
    wobei das Ansteuern des lichtemittierenden Elements (2023) zum Leuchten durch Anlegen einer Spannung an das lichtemittierende Element (2023) unter einem Ansteuersignal des Ansteuerelements (2024) Folgendes umfasst:

    Anlegen der Spannung an das lichtemittierende Element (2023) unter dem Ansteuersignal des Ansteuerelements (2024) ;

    Erregen von organischen Materialien der lichtemittierenden Schicht (20232) aus organischem Polymer zum Emittieren, wenn ein Strom von der transparenten Kathode (20233) zu der Metallanode (20231) fließt und durch die lichtemittierende Schicht (20232) aus organischem Polymer hindurchgeht.


     
    14. Verfahren nach einem der Ansprüche 11-13, wobei die Abbildungsanordnung (201) ein lichtempfindliches Element (2011) und eine Farbfilmschicht (2021) umfasst, wobei das Steuern der Abbildungsanordnung (201) zur Aufnahme eines dritten Bildes Folgendes umfasst:

    Filtern des Lichts, wenn das Licht die Farbfilmschicht durchdringt (2021);

    Erfassen des von der Farbfilmschicht (2021) gefilterten Lichts durch das lichtempfindliche Element (2011);

    Bestimmen des R-, G- und B-Wertes eines Pixels des dritten Bildes, das der Abbildungsanordnung (201) entspricht, durch Übertragen des erfassten Lichts von optischer Information in ein digitales Signal durch das lichtempfindliche Element (2011), um das dritte Bild zu bestimmen.


     
    15. Computerlesbares Speichermedium mit einer Vielzahl von darin gespeicherten Anweisungen, dadurch gekennzeichnet, dass ein Bildverarbeitungsverfahren nach einem der Ansprüche 8-14 implementiert wird, wenn die Vielzahl von Anweisungen durch einen Prozessor (102) ausgeführt wird.
     


    Revendications

    1. Dispositif électronique, comprenant une unité de traitement (10), une caméra (20), et un écran d'affichage (30) recouvrant la caméra (20) ;

    dans lequel l'écran d'affichage (30) est électriquement connecté à l'unité de traitement (10), l'écran d'affichage (30) comprend une zone de transmission de lumière (301) et une zone de non-transmission de lumière (302), et est configuré pour afficher une première image sur la zone de non-transmission de lumière (302) sous le contrôle de l'unité de traitement (10) ;
    caractérisé en ce que

    la caméra (20) est disposée en-dessous de la zone de transmission de lumière (301) et comprend une pluralité d'unités de caméra (210) disposées en réseau, chaque unité de caméra (210) comprend au moins un ensemble de projection (202) et au moins un ensemble de formation d'image (201) adjacent, chaque ensemble de projection (202) et chaque ensemble de formation d'image (201) sont électriquement connectés à l'unité de traitement (10), chaque ensemble de projection (202) est configuré pour projeter un pixel d'une deuxième image sur la zone de transmission de lumière (301) de l'écran d'affichage (30), de sorte que les ensembles de projection (202) de la pluralité d'unités de caméra disposées en réseau (210) projettent la deuxième image sur la zone de transmission de lumière (301) de l'écran d'affichage (30) sous le contrôle de l'unité de traitement (10), et chaque ensemble de formation d'image (201) est configuré pour capturer un pixel d'une troisième image à travers la zone de transmission de lumière (301) de l'écran d'affichage (30), de sorte que les ensembles de formation d'image (201) de la pluralité d'unités de caméra disposées en réseau (210) capturent la troisième image à travers la zone de transmission de lumière (301) de l'écran d'affichage (30) sous le contrôle de l'unité de traitement (10) ; et

    dans lequel l'unité de traitement (10) est configurée pour commander l'écran d'affichage (30) afin d'afficher la première image, et est configurée pour commander la caméra (20) afin de projeter la deuxième image dans un état d'affichage et de capturer la troisième image dans un état de formation d'image.


     
    2. Dispositif électronique selon la revendication 1, dans lequel l'unité de traitement (10) comprend un tampon d'images (101), un processeur (102), une première interface de communication (103) électriquement connectée à la caméra (20), et une deuxième interface de communication (104) électriquement connectée à l'écran d'affichage (30) ;

    dans lequel le tampon d'images (101) est configuré pour mettre en tampon séparément une sous-image d'affichage et une sous-image de projection obtenues en divisant une quatrième image ;

    dans lequel le processeur (102) est électriquement connecté au tampon d'images (101), et est configuré pour lire le tampon d'images (101) afin d'obtenir la sous-image d'affichage et la sous-image de projection, et pour transmettre la sous-image de projection à la première interface de communication (103) et transmettre la sous-image d'affichage à la deuxième interface de communication (104) ;

    dans lequel la première interface de communication (103) est électriquement connectée au processeur (102) et configurée pour transmettre la sous-image de projection en tant que deuxième image à la caméra (20) ; et

    dans lequel la deuxième interface de communication (104) est électriquement connectée au processeur (102) et configurée pour transmettre la sous-image d'affichage en tant que première image à l'écran d'affichage (30) ; dans lequel le tampon d'images (101) comprend de préférence un premier tampon (1011) et un deuxième tampon (1012),

    dans lequel le premier tampon (1011) est configuré pour stocker la sous-image de projection, et le deuxième tampon (1012) est configuré pour stocker la sous-image d'affichage ; et/ou

    la première interface de communication (103) et la deuxième interface de communication (104) sont des interfaces série d'affichage (DSIs) pour une transmission synchrone.


     
    3. Dispositif électronique selon la revendication 1 ou 2, dans lequel l'ensemble de projection (202) comprend une couche de film coloré (2021), une couche de câblage métallique (2022), un élément émetteur de lumière (2023) et un élément d'attaque (2024) ;

    l'élément d'attaque (2024) est disposé sur une surface d'un substrat (2025) et configuré pour attaquer l'élément émetteur de lumière (2023) pour émettre de la lumière ; la couche de câblage métallique (2022) est disposée sur une surface de l'élément d'attaque (2024), et est électriquement connectée à l'élément d'attaque (2024) et à l'élément émetteur de lumière (2023) ;

    l'élément émetteur de lumière (2023) est disposé sur une surface de la couche de câblage métallique (2022), et est configuré pour émettre de la lumière lorsqu'il est attaqué par l'élément d'attaque (2024) ; et

    la couche de film coloré (2021) est disposée sur une surface de l'élément émetteur de lumière (2023) et configurée pour filtrer des lumières.


     
    4. Dispositif électronique selon la revendication 3, dans lequel l'ensemble de formation d'image (201) et l'ensemble de projection (202) partagent le substrat (2025) .
     
    5. Dispositif électronique selon la revendication 3 ou 4, dans lequel l'ensemble de formation d'image (201) comprend un élément photosensible (2011), la couche de câblage métallique (2022) partagée avec l'ensemble de projection (202), et la couche de film coloré (2021) partagée avec l'ensemble de projection (202) ;

    l'élément photosensible (2011) est disposé sur la surface du substrat (2025), et configuré pour détecter la lumière filtrée par la couche de film coloré (2021) ; et

    la couche de câblage métallique (2022) est électriquement connectée à l'élément photosensible (2011).


     
    6. Dispositif électronique selon l'une quelconque des revendications 3-5, dans lequel l'élément émetteur de lumière (2023) comprend :

    une anode métallique (20231) disposée sur la surface de la couche de câblage métallique (2022) ;

    une couche émettrice de lumière en polymère organique (20232) disposée sur une surface de l'anode métallique (20231) ; et

    une cathode transparente (20233) disposée sur une surface de la couche émettrice de lumière en polymère organique (20232) .


     
    7. Dispositif électronique selon l'une quelconque des revendications 1-6, dans lequel chaque ensemble de projection (202) correspond à un pixel de la deuxième image, et le pixel de la deuxième image est déterminé conformément à la deuxième image, et
    chaque ensemble de formation d'image (201) correspond à un pixel de la troisième image, et le pixel de la troisième image est configuré pour générer la troisième image.
     
    8. Procédé de traitement d'images, caractérisé en ce qu'il comprend :

    la commande d'une caméra (20) dans un état d'affichage ou dans un état de formation d'image, dans lequel la caméra (20) comprend une pluralité d'unités de caméra disposées en réseau (210), chaque unité de caméra (210) comprend au moins un ensemble de projection (202) et au moins un ensemble de formation d'image (201) adjacent ;

    la commande de la caméra (20) pour projeter une deuxième image lorsque la caméra (20) est dans l'état d'affichage, dans lequel chaque ensemble de projection (202) correspond à un pixel de la deuxième image, et est configuré pour projeter le pixel de la deuxième image sur un écran d'affichage (30) recouvrant la caméra (20), de sorte que les ensembles de projection (202) de la pluralité d'unités de caméra disposées en réseau (210) projettent la deuxième image sur l'écran d'affichage (30) ; et

    la commande de la caméra (20) pour capturer une troisième image lorsque la caméra (20) est dans l'état de formation d'image, dans lequel chaque ensemble de formation d'image (201) correspond à un pixel de la troisième image, et est configuré pour capturer le pixel de la troisième image à travers l'écran d'affichage (30), de sorte que les ensembles de formation d'image (201) de la pluralité d'unités de caméra disposées en réseau (210) capturent la troisième image à travers l'écran d'affichage (30).


     
    9. Procédé selon la revendication 8, dans lequel, avant la commande de la caméra (20) pour projeter une deuxième image, le procédé comprend en outre :

    l'obtention d'une quatrième image ;

    la division de la quatrième image en une sous-image d'affichage et une sous-image de projection ; et

    la transmission de la sous-image d'affichage en tant que première image à l'écran d'affichage (30) et la transmission de la sous-image de projection en tant que deuxième image à la caméra (20) ; dans lequel, de préférence,

    avant la transmission de la sous-image d'affichage en tant que première image à l'écran d'affichage (30) et la transmission de la sous-image de projection en tant que deuxième image à la caméra (20), le procédé comprend en outre : le stockage de la sous-image de projection dans un premier tampon (1011), et le stockage de la sous-image d'affichage dans un deuxième tampon (1012) ; la lecture de la sous-image de projection depuis le premier tampon (1011), et la lecture de la sous-image d'affichage depuis le deuxième tampon (1012) ; et/ou

    la transmission de la sous-image d'affichage en tant que première image à l'écran d'affichage (30) et la transmission de la sous-image de projection en tant que deuxième image à la caméra (20) comprennent :

    la synchronisation d'un premier signal de transmission de l'écran d'affichage (30) et du deuxième signal de transmission de la caméra (20), dans lequel le premier signal de transmission transporte la sous-image d'affichage, et le deuxième signal de transmission transporte la sous-image de projection ;

    la transmission du premier signal de transmission synchronisé à l'écran d'affichage (30) et la transmission du deuxième signal de transmission synchronisé à la caméra (20) ; et/ou

    l'unité de traitement (10) comprenant une première interface de communication (103) électriquement connectée à la caméra (20), une deuxième interface de communication (104) électriquement connectée à l'écran d'affichage (30), et la première interface de communication (103) et la deuxième interface de communication (104) étant des interfaces série d'affichage (DSIs) pour une transmission synchrone,

    dans lequel la transmission de la sous-image d'affichage en tant que première image à l'écran d'affichage (30) et la transmission de la sous-image de projection en tant que deuxième image à la caméra (20) comprennent :

    la transmission de la sous-image de projection en tant que deuxième image à la caméra (20) par la première interface de communication (103), et

    la transmission de la sous-image d'affichage en tant que première image à l'écran d'affichage (30) par la deuxième interface de communication (104) .


     
    10. Procédé selon la revendication 9, dans lequel la division de la quatrième image en une sous-image d'affichage et une sous-image de projection comprend :

    la détermination de la sous-image d'affichage à partir de la quatrième image conformément à des pixels d'image correspondant à la zone de non-transmission de lumière (302) de l'écran d'affichage (30) ; et

    la détermination de la sous-image de projection à partir de la quatrième image conformément à des pixels d'image correspondant à la zone de transmission de lumière (301) projetée par la caméra (20).


     
    11. Procédé selon l'une quelconque des revendications 8-10, dans lequel le procédé est exécutable pour une unité de traitement (10) d'un dispositif électronique, dans lequel le dispositif électronique comprend l'unité de traitement (10), la caméra (20), un écran d'affichage (30) recouvrant la caméra (20), la caméra (20) et l'écran d'affichage (30) sont électriquement connectés à l'unité de traitement (10), l'écran d'affichage (30) comprend une zone de transmission de lumière (301) et une zone de non-transmission de lumière (302), et la caméra (20) est disposée en dessous de la zone de transmission de lumière (301) et comprend un ensemble de projection (202) et un ensemble de formation d'image (201) tous deux électriquement connectés à l'unité de traitement (10), dans lequel la commande de la caméra (20) pour projeter une deuxième image lorsque la caméra (20) est dans l'état d'affichage, et la commande de la caméra (20) pour capturer une troisième image lorsque la caméra (20) est dans l'état de formation d'image, comprennent :

    la commande de l'ensemble de projection (202) pour projeter la deuxième image afin d'afficher la deuxième image sur la zone de transmission de lumière (301), lorsque la caméra (20) est dans l'état d'affichage ; et

    la commande de l'ensemble de formation d'image (201) pour capturer une troisième image à travers la zone de transmission de lumière (301) lorsque la caméra (20) est dans l'état de formation d'image.


     
    12. Procédé selon la revendication 11, dans lequel l'ensemble de projection (202) comprend une couche de film coloré (2021), un élément émetteur de lumière (2023), et un élément d'attaque (2024),
    dans lequel la commande de l'ensemble de projection (202) pour projeter la deuxième image comprend :

    l'attaque de l'élément émetteur de lumière (2023) pour l'éclairer en appliquant une tension à l'élément émetteur de lumière (2023) avec un signal d'attaque de l'élément d'attaque (2024) ;

    le filtrage de la lumière pour générer trois couleurs primaires à savoir rouge, vert et bleu lorsque la lumière passe à travers la couche de film coloré (2021), pour réaliser un affichage de la deuxième image conformément à un réglage de couleur du filtre de la couche de film coloré (2021).


     
    13. Procédé selon la revendication 12, dans lequel l'élément émetteur de lumière (2023) comprend une anode métallique (20231), une couche émettrice de lumière en polymère organique (20232), et une cathode transparente (20233),
    dans lequel l'attaque de l'élément émetteur de lumière (2023) pour l'éclairer en appliquant une tension à l'élément émetteur de lumière (2023) avec un signal d'attaque de l'élément d'attaque (2024), comprend :

    l'application de la tension à l'élément émetteur de lumière (2023) avec le signal d'attaque de l'élément d'attaque (2024) ;

    l'excitation de matériaux organiques de la couche émettrice de lumière en polymère organique (20232) pour qu'ils émettent lorsqu'un courant circule de la cathode transparente (20233) à l'anode métallique (20231) et passe à travers la couche émettrice de lumière en polymère organique (20232).


     
    14. Procédé selon l'une quelconque des revendications 11-13, dans lequel l'ensemble de formation d'image (201) comprend un élément photosensible (2011) et une couche de film coloré (2021),
    dans lequel la commande de l'ensemble de formation d'image (201) pour capturer une troisième image comprend :

    le filtrage d'une lumière lorsque la lumière passe à travers la couche de film coloré (2021) ;

    la détection de la lumière filtrée par la couche de film coloré (2021) par l'élément photosensible (2011) ;

    la détermination de la valeur R, G et B d'un pixel de la troisième image correspondant à l'ensemble de formation d'image (201) par conversion de la lumière détectée d'informations optiques en un signal numérique par l'élément photosensible (2011), pour déterminer la troisième image.


     
    15. Support de stockage lisible par ordinateur, dans lequel sont stockées une pluralité d'instructions, caractérisé en ce qu'un procédé de traitement d'images selon l'une quelconque des revendications 8-14 est mis en Ĺ“uvre lors de l'exécution de la pluralité d'instructions par un processeur (102).
     




    Drawing




















    Cited references

    REFERENCES CITED IN THE DESCRIPTION



    This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

    Patent documents cited in the description