(19)
(11)EP 3 520 074 B1

(12)EUROPEAN PATENT SPECIFICATION

(45)Mention of the grant of the patent:
04.11.2020 Bulletin 2020/45

(21)Application number: 17778356.0

(22)Date of filing:  15.09.2017
(51)International Patent Classification (IPC): 
G06T 5/50(2006.01)
(86)International application number:
PCT/IB2017/055604
(87)International publication number:
WO 2018/060806 (05.04.2018 Gazette  2018/14)

(54)

METHOD FOR THE ANALYSIS OF SPATIAL AND TEMPORAL INFORMATION OF SAMPLES BY MEANS OF OPTICAL MICROSCOPY

VERFAHREN ZUR ANALYSE VON RÄUMLICHEN UND ZEITLICHEN INFORMATIONEN VON PROBEN MITTELS OPTISCHER MIKROSKOPIE

PROCÉDÉ D'ANALYSE D'INFORMATIONS SPATIALES ET TEMPORELLES D'ÉCHANTILLONS AU MOYEN D'UNE MICROSCOPIE OPTIQUE


(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

(30)Priority: 29.09.2016 IT 201600097811

(43)Date of publication of application:
07.08.2019 Bulletin 2019/32

(73)Proprietor: Fondazione Istituto Italiano di Tecnologia
16163 Genova (IT)

(72)Inventors:
  • DIFATO, Francesco
    16153 Genova (IT)
  • MORONI, Monica
    21053 Castellanza (VA) (IT)
  • PALAZZOLO, Gemma
    16164 Genova (IT)
  • SOLOPERTO, Alessandro
    74028 Sava (TA) (IT)

(74)Representative: Bottino, Giovanni 
ARBO Srl Via Colombo, 11/29
16121 Genova
16121 Genova (IT)


(56)References cited: : 
WO-A1-2009/023635
DE-A1-102008 049 878
  
  • FLORIAN O. FAHRBACH ET AL: "Rapid 3D light-sheet microscopy with a tunable lens", OPTICS EXPRESS, vol. 21, no. 18, 30 August 2013 (2013-08-30), page 21010, XP055089699, ISSN: 1094-4087, DOI: 10.1364/OE.21.021010 cited in the application
  • SEAN QUIRIN ET AL: "Instantaneous three-dimensional sensing using spatial light modulator illumination with extended depth of field imaging", OPTICS EXPRESS, vol. 21, no. 13, 27 June 2013 (2013-06-27) , page 16007, XP055373955, DOI: 10.1364/OE.21.016007 cited in the application
  
Note: Within nine months from the publication of the mention of the grant of the European patent, any person may give notice to the European Patent Office of opposition to the European patent granted. Notice of opposition shall be filed in a written reasoned statement. It shall not be deemed to have been filed until the opposition fee has been paid. (Art. 99(1) European Patent Convention).


Description


[0001] The present invention relates to a method for the analysis of spatial and temporal information of samples by means of optical microscopy.

[0002] In optical microscopy field the need of investigating samples, even of substantial thickness, is known, such as for example biological samples. Regions of interest are typically present inside the biological sample, such as for example specific cells. Such regions of interest can exhibit, above all in case of live biological samples, changes over time of their own optical features, corresponding to specific behaviors of the sample, whose detection is of interest for a biological analysis of the sample. In these cases, optical microscopy has to be able to detect both spatial information of regions of interests, namely where they are arranged inside the three-dimensional space under examination of the sample, and temporal information, namely the trend over time of optical features of the regions of interest.

[0003] In this technical field the document "Instantaneous three-dimensional sensing using spatial light modulator illumination with extended depth of field imaging" (Sean Quirin, Darcy S. Peterka, and Rafael Yuste - Optics express 2013) discloses the sensing of spatial and temporal information about the sample by extended depth of field imaging. However spatial information is encoded on the basis of the particular configuration of the excitation path, that uses a two-photon system. Therefore spatial information gets lost when using other excitation techniques. The document discloses the use of a micro-machined phase mask that provides a fixed phase modulation of the sensing path to have an extended depth of field. This means that the extended depth of field cannot be modified in amplitude, once the phase mask has been made and inserted. That is to say, this configuration does not allow the extension of the depth of field to be selected. The phase mask further induces aberrations in images that require deconvolution techniques to eliminate artifacts. Finally the extended depth-of-field acquisition method used does not include gathering spatial information of the sample in axial direction.

[0004] In the same technical field also the document "Rapid 3D light-sheet microscopy with a tunable lens" (Fahrbach et al. Optics express 2013) describes an analysis of spatial and temporal information of samples by optical microscopy. The document discloses a method using an electrically focus tunable liquid lens to perform a dynamic phase modulation in the reception path. Such lenses are currently known and available on the market and allow, by applying an electrical control signal, the curvature of the liquid to be controlled and therefore the focal length to be changed. This allows an arbitrary plane to be focused within a three-dimensional sample without moving any mechanical part of the system, and therefore allows a very rapid axial scanning of the detected in-focus plane to be performed without inertia. However the document does not disclose the possibility of performing an extended depth of field imaging and therefore the method is limited in scan rate.

[0005] An electrically tunable lens is also used in document WO2009/02363A1 which describes forward-imaging optical coherence tomography (OCT) techniques using a swept laser or a tunable laser source and using at least one tunable lens exemplarily made of a deformable base material e.g. a gel, or a fluid, liquid or gas, the lens deformation being caused by application of an electric or, preferably, magnetic field.

[0006] Document DE102008049878A1 describes microscopic imaging of fluorescent samples wherein the sample is sequentially illuminated in a plurality of phases by structured laser light pulsed at varying frequencies, such that recording of individual images is optimized for achieving the best possible resolution e.g. by depopulating a triplet state of the fluorescent dye, which reduces bleaching. This document also mentions that intermediate images may be integrated to form one individual image.

[0007] From the above it is clear that there is an unsatisfied need in prior art known methods for an optical microscopy method able to effectively obtain and analyse spatial and temporal information of samples, with high scan rates, with the possibility of setting the depth of investigation in the sample, and that does not require changes to the architecture of the used optical system.

[0008] These objects are pursued by the present invention that relates to a method for the analysis of spatial and temporal information of samples by optical microscopy comprising the following steps:
  1. a) choosing a field of view in a sample;
  2. b) scanning the sample in the axial direction by applying an electrical control signal having a first frequency to an electrically tunable liquid lens placed in the detection path, performing a sequential acquisition, at a first acquisition frequency which is higher than the first frequency of the electrical signal, of a stack of images placed on different in-focus planes;
  3. c) processing the stack of images for identifying the position of one or more regions of interest in the three-dimensional space under examination of the sample;
  4. d) scanning the sample in the axial direction by further applying the electrical signal having a second frequency to the tunable lens, performing a sequential acquisition, at a second acquisition frequency which is lower than the second frequency of the electrical signal, of a temporal series of images with extended depth of field;
  5. e) calculating the mean intensity of each region of interest for each image of the temporal series.


[0009] The method therefore provides to use an electrically tunable liquid lens inserted in the reception optical configuration of a microscope, to perform without inertia a very fast axial scanning of the in-focus plane, obtaining speeds that cannot be reached by methods providing a motorized movement of components of the microscope such as for example the objective lens or the stage of the microscope. The method allows the speed and depth of the axial scanning to be controlled by the electrical control signal. This is carried out by a modulation of the focal length of the lens and therefore with a phase modulation of the reception optical path.

[0010] Once the selection of the field of view in the sample is carried out as in step a), step b) provides a slow axial scanning, where an image for each focused plane is acquired by a photosensitive detector, thus obtaining an axial stack, called as z-stack, of images of the sample. The stack of images is therefore processed in step c) to extract spatial information of the sample. Steps b) and c) thus allow the spatial configuration of the sample to be monitored and the three-dimensional position of the features within the sample to be extracted.

[0011] After obtaining spatial information, the method provides at step d) a fast axial scanning, such to gather optical signals from all the scanned planes, projecting them in a single image. Thus an extended depth of field of the objective of the microscope is obtained, gathering signals from all the scanned planes, and thus generating an axial projection of the sample, to acquire in each single image of the series all the features comprised within the axial portion under examination of the sample. The calculation of the mean of the intensity of each region of interest for each image of the temporal series as in step e) allows a time-lapse to be generated to follow signal fluctuations associated with features arranged in the three-dimensional space within the acquired extended depth of field.

[0012] The method has many technical advantages. The speed for axially scanning the sample can be set by dynamic phase modulation. This allows two possible configurations to be used: slow axial scanning and fast axial scanning. The terms slow and fast are referred to the acquisition frequency of the photosensitive detector. The two configurations work in synergy to detect both spatial and temporal information of the sample. Moreover the sample portion involved by the scanning can be arbitrarily defined, as well as the extension of the depth of field, by means of the dynamic phase modulation of the optical detection path carried out by the tunable lens and a synchronized image acquisition by the photosensitive detector. Finally it is not necessary to change the system architecture to pass from one to the other configuration, by means of the dynamic phase modulation allowed by the tunable lens.

[0013] In one embodiment step c) comprises the following steps:

c1) projecting the stack of images into a single combined image;

c2) identifying in the combined image one or more regions of interest;

c3) calculating the mean intensity in each region of interest in each image of the stack and identifying the maximum value of mean intensity for each region of interest.



[0014] This allows firstly the position of the regions of interest to be defined in directions orthogonal to the axial direction and secondly their location in axial direction to be defined.

[0015] According to one embodiment, step b) is preceded by a calibration step comprising the following steps:

a1) applying the electrical signal to the tunable lens, which electrical signal is such that a scan of a sequence of in-focus planes is performed;

a2) visualizing by a user the sequence of in-focus planes;

a3) choosing by the user the limit in-focus planes of the scan;

a4) setting the corresponding voltage limit values of the electrical signal.



[0016] Calibration step allows the portion of the sample under examination to be set in axial direction. This is obtained by changing the electrical control signal applied to the tunable lens and contemporaneously visualizing the sample, such that it is possible to select and set the maximum and minimum voltage to be applied to the tunable lens, and therefore the limit in-focus planes of the scan.

[0017] According to an improvement there is provided the generation of a conversion factor from the voltage values of the electrical signal to the corresponding in-focus planes of the sequence, such that step c3) is followed by the following steps:

c4) associating for each region of interest the maximum value of mean intensity to a corresponding voltage value of the electrical signal;

c5) identifying for each region of interest the in-focus plane corresponding to the maximum value of mean intensity by applying the conversion factor to the associated voltage value of the electrical signal.



[0018] Preferably the generation of the conversion factor is carried out in the calibration step.

[0019] Thus it is possible to identify the voltage value of the electrical signal corresponding to the scan plane where the region of interest has the maximum mean intensity, and to convert such voltage value into the axial spatial coordinate by multiplying by the calculated conversion factor.

[0020] According to one embodiment in step b) the distance between in-focus planes corresponding to the images of the stack is defined by the examination depth of the sample, and by the number of acquisitions made during a whole sample scanning, which number of acquisitions is defined by setting the frequency of the electrical signal and the acquisition frequency respectively.

[0021] According to a further embodiment, the projection of the stack of images in a single combined image occurs by processing by software the stack of images.

[0022] According to a further embodiment one or more regions of interest in the combined image are identified by the binarization of the combined image according to a predetermined threshold.

[0023] The analysis of the image combined at step c1) and binarized according to the predetermined threshold therefore makes it possible to identify spatial coordinates perpendicular to the axial direction of the regions of interest. This, in combination with identifying the spatial coordinates along the axial direction described above, allows the position of each region of interest in the three-dimensional space under examination to be accurately identified.

[0024] In a further embodiment, in step d) the acquisition frequency is higher or equal to twice the frequency of the electrical signal.

[0025] Thus the in-focus plane moves between the limit planes at a frequency higher than the acquisition frequency such that the photosensitive detector substantially integrates in a single image information of all planes, in the one that can be described as a projection of the peaks of the three-dimensional portion under examination of the sample performed by hardware.

[0026] In one embodiment the duration of the acquisition at step d) is set by the following steps:

d1) setting the frequency of the electrical signal;

d2) setting the acquisition frequency;

d3) defining the number of images with extended depth of field of the temporal series.



[0027] Thus it is possible to decide the number of images to be acquired to compose the temporal series to define the duration of the time-lapse acquisition.

[0028] According to one embodiment, the optical microscopy procedure is of fluorescence type.

[0029] Optical fluorescence microscopy uses the fluorescence phenomenon, that is the ability of a material to emit light after absorbing light, as an alternative or in combination with reflection and absorption phenomena, to study properties of the material under examination. However the method can be applied to other types of optical microscopy, for example transmitted light microscopy.

[0030] According to one embodiment the optical microscopy procedure is of widefield type. In this type of procedure all the sample is contemporaneously illuminated in the three-dimensional space. In this case the method allows the configuration of the excitation path of the widefield system to be maintained unchanged, and it requires only small changes to the reception path. However the method can be applied to further configurations such as standard or two-photon illumination confocal microscopy implemented by phase masks or light spatial modulators, single plane illumination or structured illumination microscopy, oblique illumination microscopy.

[0031] These and other characteristics and advantages of the present invention will be more clear from the following description of some non limitative embodiments shown in annexed drawings where:

Fig.1 is slow scanning;

Fig.2 is fast scanning;

Fig.3 is the dependence of in-focus plane from electrical control signal;

Fig.4 is the electrical control signal;

Fig.5 is a stack of images obtained by slow scanning;

Fig. 6 is the identification of the position of the regions of interest in axial direction;

Fig.7 is the identified position of the regions of interest in the three-dimensional space under examination of the sample;

Fig.8 is the trend over time of mean intensity of two distinct regions of interest;

Fig.9 is a possible device for carrying out the method.



[0032] The method of the present invention uses an electrically focus tunable liquid lens to perform a dynamic phase modulation in the reception path of an optical microscopy device, preferably by widefield architecture.

[0033] A preferred embodiment of the device is shown in figure 9. The device in figure is a widefield inverted fluorescence microscope, but the method can be implemented also with upright microscope. The device comprises an excitation light source 34, preferably a xenon lamp, whose excitation light is gathered by a lens 33 and reflected towards an objective 31 by a dichroic mirror 32. The objective 31 receives fluorescence light generated by the sample 4, which is separated from the excitation light by the dichroic mirror 32. Therefore fluorescence light is sent to the photosensitive detector 39 by a telescope composed of two simple convex lenses 35 and 37, called as relay lenses, the tunable lens 36 being placed in the common in-focus plane of relay lenses 35 and 37. The presence of relay lenses 35 and 37 allows image deformations not to be introduced as the in-focus plane changes. Thus the system magnification does not change as the imaged plane changes. It is possible to provide a configuration without relay lenses 35 and 37, wherein however a following image processing by software is necessary. An emission filter 38 is provided, which eliminates light reflected from the sample, interposed between telescope and detector 39. The device further comprises a control unit for the tunable lens, not shown in figure, which control unit is configured such that it generates an electrical signal with a frequency smaller than acquisition frequency, for a sequential acquisition of a stack of images placed on different in-focus planes, or it generates an electrical signal with a frequency higher than the acquisition frequency, for the sequential acquisition of a temporal series of images with extended depth of field.

[0034] In such preferred embodiment, the tunable lens has a focal length ranging from 40 to 140 mm and it can be modulated by an electrical control signal with temporal bandwidth of 1 KHz.

[0035] By changing the focal length of the tunable lens 36, the optical transfer function of the reception path changes and therefore it is possible to scan the in-focus plane of the objective 31 in the axial direction of the sample 4, such as shown by the several light beams shown in figure 9.

[0036] There is a linear relationship between the electrical control signal applied and the shift of the in-focus plane inside the sample 4, such as shown in figure 3, wherein abscissa denotes voltage intensity of the electrical signal in Volt and ordinate denotes the shift of the in-focus plane in mm.

[0037] It is possible to control the speed of the shift of the in-focus plane by changing the frequency of the control signal of the tunable lens, and the extension of the overall shift by setting limit voltage values that can be taken by the electrical signal:

where p0 and p1 are the start and end points of the shift of the in-focus plane, which can be selected arbitrarily by setting the limit values of maximum and minimum voltage Vp0 and Vp1 of the electrical control signal. The electrical control signal is visible in figure 4, wherein a graph is shown with time as abscissa and signal voltage as ordinate, and preferably it is a sawtooth wave ranging between the limit voltage values Vp0 and Vp1. Values p0 and p1 can be arbitrarily selected, for example p0=0 µm and p1=300 µm.

[0038] The method can be performed by a microscopy device comprising a photosensitive detector. Such detector preferably is a two-dimensional detector configured for receiving a pixel array, such as a CCD 39.

[0039] Image acquisition by the photosensitive detector 39 is synchronized with the axial shift produced by the tunable lens 36, such to operate in two conditions.

[0040] In the first condition the focal length of the tunable lens is slowly changed to gather a plurality of images during the slow axial imaging of sample 4, and to obtain a series of images stacked in a z-stack 5. Such condition is shown in figure 1, wherein the sample 4 under examination has regions of interest, particularly cells 1, in the case of the figure neurons. In this arrangement, the acquisition frequency or frame rate of the photosensitive detector 39 is more rapid than the electrical control signal of the tunable lens 36. By shifting the in-focus plane 2 in axial direction from the axial position p0 corresponding to the limit plane 20 to the axial position p1 corresponding to the limit plane 21 with speed Vshift=(p1-p0)/Fshift, and contemporaneously acquiring images with frequency FCCD, a number of images N = (FCCD/Fshift)/2 is acquired during all the axial travel (between p0 and p1) of the sample 4. Considering the linear relationship between the control signal and the shift of the in-focus plane 2, it is possible to determine the dimension of the axial distance Zstep between images of the acquired stack:Zstep = (p1 - p0)/N.

[0041] During slow imaging the control signal may be composed at most only by a voltage ramp from Vp0 to Vp1. On the contrary, advantageously, the signal is oscillated several times between Vp0 and Vp1 during acquisition, and acquired images related to the same in-focus planes are averaged with one another to have a higher signal-to-noise ratio.

[0042] In the second condition the focal length of the tunable lens 36 is rapidly changed to continuously perform to and fro movements among the limit in-focus planes during the fast axial imaging of the sample 4. In this arrangement the control signal of the tunable lens 36 is much faster than the acquisition rate of the photosensitive detector 39. The axial imaging has to be at least twice faster than the acquisition frequency, such to perform imaging of the whole axial portion of the sample during the acquisition of a single image by the detector, and therefore it allows optical signals coming from scanned planes to be contemporaneously gathered and allows them to be projected in a single acquired image. Thus the depth of field of the microscope is extended. Figure 2 shows such second condition, wherein the sample 4 is rapidly imaged in the axial direction by moving the in-focus plane 2 among the limit planes 20 and 21. The extension of the depth of field can be arbitrarily selected by setting the initial voltage Vp0 and final voltage Vp1 of the signal sent to the tunable lens 36, to which the axial position p0 corresponding to limit plane 20 and the axial position p1 corresponding to the limit plane 21 of the axial imaging correspond respectively:



[0043] In slow imaging as of step b) a first frequency of the electrical signal and a first acquisition frequency are used, while in fast imaging as of step d) a second frequency of the electrical signal and a second acquisition frequency are used. In order to obtain alternately the different slow and fast imaging configurations it is necessary for the first acquisition frequency to be higher than the first frequency of the electrical signal and for the second acquisition frequency to be lower than the second frequency of the electrical signal. In order to obtain the above, it is possible to maintain as constant the acquisition frequency both in slow imaging and in fast imaging and to change only the frequency of the electrical signal in the two configuration or vice versa to maintain as constant the frequency of the electrical signal both in slow imaging and in fast imaging and to change only the acquisition frequency in the two configurations. A preferred embodiment operates in the two configurations on both the frequencies, such that the first and second frequencies of the electrical signal are different from each other as well as the first and second acquisition frequency.

[0044] One embodiment of a protocol for performing the method of the present invention is described in details below:
  1. 1. Placing the sample 4 on the stage of the microscope 30.
  2. 2. Activating the excitation light source 34 to excite the fluorescent molecules inside the sample 4.
  3. 3. Selecting a field of view (see step a) in x, y direction by moving the stage of the microscope 30, with the tunable lens 36 set in a fixed focal position, that is with a electrical control signal set as constant.
  4. 4. Maintaining the x, y position of the sample 4 selected as at step 3, and scanning the sample 4 in axial direction by slowly changing the voltage applied to the tunable lens 36. The voltage to the tunable lens 36 can be changed by any type of waveform, or manually by a voltage generator. In this step, the aim is to monitor the sample 4 in the axial direction, for selecting and setting the maximum voltage Vp1 and minimum voltage Vp0 to be applied to the tunable lens 36 and therefore for imaging the selected portion of the sample 4 in the axial direction (see steps a1, a2, a3, a4).
  5. 5. Once the position of sample 4 (x,y) and axial imaging extension (z) are selected by setting Vp0 and Vp1, acquiring an image stack 5 from CCD (see step b). A periodic waveform voltage is applied to the tunable lens 36, for scanning to and fro the sample 4 between limit positions p0 and p1, that are the positions selected at step 4 and corresponding to Vp0 and Vp1 respectively. In this configuration, the frequency of the signal sent to the tunable lens 36 (Fshift) is smaller than the acquisition frequency of CCD 39. The ratio FCCD/Fshift defines the number N of images acquired in the stack 5 during the axial imaging of the sample 4. The distance between subsequent images of the acquired stack 5 is defined as Zstep = (p1 - p0)/N. During such step the electrical control signal to the tunable lens 37 and image acquisition are synchronized. The acquired image stack 5 is shown in figure 5.
  6. 6. Processing the stack 5 to define the position x, y, z of the features of interest 50 inside the sample portion under examination, such as shown in figure 7.

    6.1 Projecting the stack 5 in a single two-dimensional image by software (see step c). The projection in axial direction can be a projection of peaks, mean, slice-sum or standard deviation. The aim is to obtain the best contrast in the projection image. It is possible to possibly implement a deconvolution of the stack before producing the projection image to enhance the contrast.

    6.2 Binarization of the combined projection image by a threshold, such to automatically identify the regions of interest (see step c), and to extract x, y coordinates of the regions of interest 50 in the field.

    6.3 Calculating the mean intensity in the regions of interest 50 identified at step 6.2 for each image of the stack 5.

    6.4 Identifying, for each region of interest 50, the image of the stack 5 to which the maximum mean value of intensity corresponds (see step c3). A graph of the trend of mean intensity as the axial depth changes is shown in figure 6, where a peak point 60 is shown, corresponding to a region of interest 50.

    6.5 Converting the value of the control signal of the tunable lens identified as corresponding to each image with maximum mean intensity of the region of interest 50 in a space axial coordinate (see steps c1 and c2). The conversion is performed by multiplying the value of the control signal by the conversion factor obtained by the calibration, that binds the voltage signal to the shift of the in-focus plane. Therefore the x, y, z position of each region of interest 50 identified is obtained, such as seen in figure 7.

  7. 7. Observing signal fluctuations of the regions of interest 50, by a time-lapse acquisition of images in extended depth of field configuration. The x, y position defined at point 1 and voltage values Vp0 and Vp1 defined at point 2 are maintained. In this configuration, the frequency Fshift of the signal is set at a value higher than the image acquisition frequency FCCD: Fshift has to be at least twice the FCCD to image the whole axial portion of the sample 4 in a single image acquisition and therefore to integrate signals deriving from the regions of interest in an image of CCD 39 (see step e).
  8. 8. Once the frequency Fshift of the signal is set, setting the number of image to be acquired by the CCD 39 in the temporal sequence to define the time duration of the time-lapse image acquisition.
  9. 9. After acquiring time-lapse images, calculating the mean intensity in the regions of interest 50 for each image of the temporal sequence, to extract signal fluctuations over time 7, such as shown in figure 8.


[0045] The method described up to now therefore aims at defining the extension of the depth of field, at identifying the x, y, z position of the features of interest, and at following signal fluctuations as a function of time of the defined features. Therefore the method allows image acquisitions to be performed with extended depth of field on a volume of the sample arbitrarily defined, with information encoded by the axial position of the features inside the field of view.

[0046] The method can be effectively used to define the position of transfected neuronal cells 1 by a genetically encoded calcium fluorescence sensor (possibly other fluorescence probes can be used, such as for example probes sensitive to cellular membrane potential or to intracellular concentration of other ions. Moreover the probe can be genetically encoded in cells by transfection or viral infection, or synthetic markers introduced in cells can be used). Once the position of cells is defined, it is possible to observe the fluorescence fluctuations associated to changes of the concentration of intracellular calcium of each cell inside the three-dimensional network, which fluctuations are associated to the electrophysiological activity of cells.

[0047] The system can be provided with a user interface for the automatization of image analysis and data extraction.


Claims

1. Method for the analysis of spatial and temporal information of samples by means of optical microscopy,
comprising the following steps:

a) choosing the field of view in a sample;

b) scanning the sample in the axial direction by applying an electrical control signal, such that the electrical signal has a first frequency, to an electrically tunable liquid lens placed in the detection path, performing a sequential acquisition, at a first acquisition frequency which is higher than the first frequency of the electrical signal, of a stack of images placed on different in-focus planes;

c) processing the stack of images for identifying the position of one or more regions of interest in the three-dimensional space under examination of the sample;

d) scanning the sample in the axial direction by further applying the electrical signal, such that the electrical signal has a second frequency, to the tunable lens, performing a sequential acquisition, at a second acquisition frequency which is lower than the second frequency of the electrical signal, of a temporal series of images with extended depth of field;

e) calculating the mean intensity of each region of interest for each image of the temporal series.


 
2. Method according to claim 1, wherein step c) comprises the following steps:

c1) projecting the stack of images into a single combined image;

c2) identifying in the combined image one or more regions of interest;

c3) calculating the mean intensity in each region of interest in each image of the stack and identifying the maximum value of mean intensity for each region of interest.


 
3. Method according to claim 1 or 2, wherein step b) is preceded by a calibration step comprising the following steps:

a1) applying the electrical signal to the tunable lens, which electrical signal is such that a scan of a sequence of in-focus planes is performed;

a2) visualizing by a user the sequence of in-focus planes;

a3) choosing by the user the limit in-focus planes of the scan;

a4) setting the corresponding voltage limit values of the electrical signal.


 
4. Method according to claim 2 or 3, wherein there is provided the generation of a conversion factor from the voltage values of the electrical signal to the corresponding in-focus planes of the sequence, such that step c3) is followed by the following steps:

c4) associating for each region of interest the maximum value of mean intensity to a corresponding voltage value of the electrical signal;

c5) identifying for each region of interest the in-focus plane corresponding to the maximum value of mean intensity by applying the conversion factor to the associated voltage value of the electrical signal.


 
5. Method according one or more of the preceding claims from 2 to 4, wherein the projection of the stack of images in a single combined image occurs by processing by software the stack of images.
 
6. Method according to one or more of the preceding claims from 2 to 5, wherein the identification of one or more regions of interest in the combined image occurs by binarization of the combined image according to a predetermined threshold.
 
7. Method according to one or more of the preceding claims, wherein in step d) the acquisition frequency is higher than or equal to twice the frequency of the electrical signal.
 
8. Method according to one or more of the preceding claims, wherein the duration of the acquisition at step d) is set by means of the following steps:

d1) setting the second frequency of the electrical signal;

d2) setting the second acquisition frequency;

d3) defining the number of images with extended depth of field of the temporal series.


 
9. Method according to one or more of the preceding claims, wherein the optical microscopy procedure is of fluorescence type.
 
10. Method according to one or more of the preceding claims, wherein the optical microscopy procedure is of widefield type.
 


Ansprüche

1. Verfahren zur Analyse von Raum- und Zeitinformationen von Proben mittels optischer Mikroskopie, das die folgenden Schritte umfasst:

a) Auswählen des Sichtfeldes in einer Probe;

b) Abtasten der Probe in einer Axialrichtung durch Anwenden eines elektrischen Steuersignals, derart, dass das elektrische Signal eine erste Frequenz aufweist, auf eine elektrisch abstimmbare flüssige Linse, die im Detektionspfad platziert ist, Durchführen einer sequenziellen Erfassung eines Stapels von Bildern, die in verschiedenen fokussierten Ebenen platziert sind, mit einer ersten Erfassungsfrequenz, die höher ist als die erste Frequenz des elektrischen Signals;

c) Verarbeiten des Stapels von Bildern zum Identifizieren der Position von einer oder mehreren Regionen von Interesse im dreidimensionalen Raum, der in der Probe untersucht wird;

d) Abtasten der Probe in der Axialrichtung durch weiteres Anwenden des elektrischen Signals, derart, dass das elektrische Signal eine zweite Frequenz aufweist, auf die abstimmbare Linse, Durchführen einer sequenziellen Erfassung einer zeitlichen Reihe von Bildern mit erweiterter Tiefenschärfe und einer zweiten Erfassungsfrequenz, die niedriger ist als die zweite Frequenz des elektrischen Signals;

e) Berechnen der mittleren Intensität jeder Region von Interesse für jedes Bild der zeitlichen Reihe.


 
2. Verfahren nach Anspruch 1, wobei Schritt c) die folgenden Schritte umfasst:

c1) Projizieren des Stapels von Bildern zu einem einzigen kombinierten Bild;

c2) Identifizieren von einer oder mehreren Regionen von Interesse im kombinierten Bild;

c3) Berechnen der mittleren Intensität in jeder Region von Interesse in jedem Bild des Stapels und Identifizieren des maximalen Wertes der mittleren Intensität für jede Region von Interesse.


 
3. Verfahren nach Anspruch 1 oder 2, wobei Schritt b) ein Kalibrierungsschritt vorausgeht, der die folgenden Schritte umfasst:

a1) Anwenden des elektrischen Signals auf die abstimmbare Linse, wobei das elektrische Signal derart ist, dass eine Abtastung einer Sequenz von fokussierten Ebenen durchgeführt wird;

a2) Visualisieren der Sequenz von fokussierten Ebenen durch einen Benutzer;

a3) Auswählen der Grenze in fokussierten Ebenen der Abtastung durch den Benutzer;

a4) Einstellen der entsprechenden Spannungsgrenzwerte des elektrischen Signals.


 
4. Verfahren nach Anspruch 2 oder 3, wobei die Erzeugung eines Umrechnungsfaktors von den Spannungswerten des elektrischen Signals in die entsprechenden fokussierten Ebenen der Sequenz bereitgestellt ist, derart, dass Schritt c3) die folgenden Schritte folgen:

c4) Verknüpfen des maximalen Wertes der mittleren Intensität mit einem entsprechenden Spannungswert des elektrischen Signals für jede Region von Interesse;

c5) Identifizieren der fokussierten Ebene, die dem maximalen Wert der mittleren Intensität entspricht, durch Anwenden des Umrechnungsfaktors auf den verknüpften Spannungswert des elektrischen Signals für jede Region von Interesse.


 
5. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche von 2 bis 4, wobei die Projizierung des Stapels von Bildern zu einem einzigen kombinierten Bild durch Verarbeiten des Stapels von Bildern mittels Software erfolgt.
 
6. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche von 2 bis 5, wobei die Identifikation von einer oder mehreren Regionen von Interesse im kombinierten Bild durch Binarisierung des kombinierten Bildes gemäß einem vorbestimmten Schwellwert erfolgt.
 
7. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche, wobei in Schritt d) die Erfassungsfrequenz gleich oder höher ist als die doppelte Frequenz des elektrischen Signals.
 
8. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche, wobei die Dauer der Erfassung in Schritt d) mittels der folgenden Schritte eingestellt wird:

d1) Einstellen der zweiten Frequenz des elektrischen Signals;

d2) Einstellen der zweiten Erfassungsfrequenz;

d3) Definieren der Anzahl von Bildern mit erweiterter Tiefenschärfe der zeitlichen Reihe.


 
9. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche, wobei der optische Mikroskopieprozess von einem Fluoreszenztyp ist.
 
10. Verfahren nach einem oder mehreren der vorhergehenden Ansprüche, wobei der optische Mikroskopieprozess von einem Weitfeldtyp ist.
 


Revendications

1. Procédé d'analyse d'informations spatiales et temporelles d'échantillons au moyen de microscopie optique, comprenant les étapes suivantes consistant à :

a) choisir le champ de vision dans un échantillon ;

b) balayer l'échantillon dans la direction axiale en appliquant un signal de commande électrique, de sorte que le signal électrique a une première fréquence, à une lentille liquide accordable électriquement placée dans la trajectoire de détection, en effectuant une acquisition séquentielle, à une première fréquence d'acquisition qui est supérieure à la première fréquence du signal électrique, d'une pile d'images placées sur différents plans de mise au point ;

c) traiter la pile d'images pour identifier la position d'une ou de plusieurs régions d'intérêt dans l'espace tridimensionnel à l'examen de l'échantillon ;

d) balayer l'échantillon dans la direction axiale en appliquant en outre le signal électrique, de telle sorte que le signal électrique a une seconde fréquence, à la lentille accordable, en effectuant une acquisition séquentielle, à une seconde fréquence d'acquisition qui est inférieure à la seconde fréquence du signal électrique, d'une série temporelle d'images avec une profondeur de champ étendue ;

e) calculer l'intensité moyenne de chaque région d'intérêt pour chaque image de la série temporelle.


 
2. Procédé selon la revendication 1, où l'étape c) comprend les étapes suivantes consistant à :

c1) projeter la pile d'images en une seule image combinée ;

c2) identifier dans l'image combinée une ou plusieurs régions d'intérêt ;

c3) calculer l'intensité moyenne dans chaque région d'intérêt dans chaque image de la pile et identifier la valeur maximale de l'intensité moyenne pour chaque région d'intérêt.


 
3. Procédé selon la revendication 1 ou 2, où l'étape b) est précédée d'une étape d'étalonnage comprenant les étapes suivantes consistant à :

a1) appliquer le signal électrique à la lentille accordable, lequel signal électrique est tel qu'un balayage d'une séquence de plans de mise au point est effectué ;

a2) visualiser par un utilisateur la séquence de plans de mise au point ;

a3) choisir par l'utilisateur les plans de mise au point limites du balayage ;

a4) régler les valeurs limites de tension correspondantes du signal électrique.


 
4. Procédé selon la revendication 2 ou 3, où il est prévu la génération d'un facteur de conversion à partir des valeurs de tension du signal électrique vers les plans de mise au point correspondants de la séquence, de telle sorte que l'étape c3) est suivie des étapes suivantes consistant à :

c4) associer pour chaque région d'intérêt la valeur maximale de l'intensité moyenne à une valeur de tension correspondante du signal électrique ;

c5) identifier pour chaque région d'intérêt le plan de mise au point correspondant à la valeur maximale de l'intensité moyenne en appliquant le facteur de conversion à la valeur de tension associée du signal électrique.


 
5. Procédé selon une ou plusieurs des revendications précédentes de 2 à 4, où la projection de la pile d'images dans une seule image combinée se produit en traitant par logiciel la pile d'images.
 
6. Procédé selon une ou plusieurs des revendications précédentes de 2 à 5, où l'identification d'une ou plusieurs régions d'intérêt dans l'image combinée a lieu par binarisation de l'image combinée selon un seuil prédéterminé.
 
7. Procédé selon une ou plusieurs des revendications précédentes, où, à l'étape d), la fréquence d'acquisition est supérieure ou égale au double de la fréquence du signal électrique.
 
8. Procédé selon une ou plusieurs des revendications précédentes, où la durée de l'acquisition à l'étape d) est réglée au moyen des étapes suivantes consistant à :

d1) régler la seconde fréquence du signal électrique ;

d2) régler la seconde fréquence d'acquisition ;

d3) définir le nombre d'images à profondeur de champ étendue de la série temporelle.


 
9. Procédé selon une ou plusieurs des revendications précédentes, où le procédé de microscopie optique est de type à fluorescence.
 
10. Procédé selon une ou plusieurs des revendications précédentes, où le procédé de microscopie optique est de type à champ large.
 




Drawing




















Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description




Non-patent literature cited in the description