(19)
(11)EP 3 739 531 A1

(12)EUROPEAN PATENT APPLICATION

(43)Date of publication:
18.11.2020 Bulletin 2020/47

(21)Application number: 20174272.3

(22)Date of filing:  12.05.2020
(51)International Patent Classification (IPC): 
G06Q 10/08(2012.01)
(84)Designated Contracting States:
AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR
Designated Extension States:
BA ME
Designated Validation States:
KH MA MD TN

(30)Priority: 14.05.2019 US 201962847439 P

(71)Applicant: Qatar Foundation For Education Science And Community Development
Doha (QA)

(72)Inventors:
  • RIZZO, Stefano
    Doha (QA)
  • LUCAS, Ji
    Doha (QA)
  • KAOUDI, Zoi
    Doha (QA)
  • QUIANE-RUIZ, Jorge-Arnulfo
    Doha (QA)
  • CHAWLA, Sanjay
    Doha (QA)

(74)Representative: K&L Gates LLP 
Karolinen Karree Karlstraße 12
80333 München
80333 München (DE)

  


(54)CARGO BOOKING MANAGEMENT SYSTEMS AND METHODS


(57) The present disclosure provides a cargo revenue management system and method that increases the efficiency of cargo revenue management by increasing the prediction accuracy of cargo volumes that customers will tender in order to generate more efficient decisions to accept or reject cargo bookings. The provided system accomplishes this increased efficiency by identifying cargo volumes that customers arbitrarily book when an actual volume is unknown as disguised missing values and deemphasizing such values in the prediction of a cargo volume that will be received. The provided system additionally utilizes machine-learning models trained on a combination of features to predict a cargo volume that will be received for a particular cargo booking. Based on the predicted cargo volume that will be received, the system generates a decision of whether to accept or reject the cargo booking to maximize revenue generation.




Description

PRIORITY CLAIM



[0001] The present application claims priority to and the benefit of U.S. Provisional Application 62/847,439, filed May 14, 2019, the entirety of which is herein incorporated by reference.

TECHNICAL FIELD



[0002] The present invention generally relates to cargo revenue management. More specifically, the present invention relates to decision generation for cargo revenue management that includes identifying disguised missing values.

BACKGROUND



[0003] The revenue of commercial airlines is primarily derived from sales of passenger tickets and cargo (freight) shipments. While most modern airlines have implemented sophisticated data-driven passenger revenue management systems, particular difficulties arise for doing the same for cargo shipments. The air-cargo ecosystem is complex and involves several players including shippers, freight forwarders, airline customer, and end-customers. Overall, five fundamental differences may be highlighted between passenger and cargo revenue management.

[0004] First, in the case of passenger revenue, the unit of sale is an airline seat, which is static. In the case of cargo, however, there is substantial variability in both volume and weight of cargo shipments. Further, the revenue from a cargo shipment often depends on the nature of the cargo. For example, perishable and non-perishable shipments generate different marginal revenues. Accordingly, the unit of sale in cargo is highly dynamic.

[0005] Second, a large chunk of air cargo capacity is pre-booked by freight forwarders who tend to overbook and release capacity closer to the date of departure. The ecosystem of air-cargo management is such that there is no penalty for overbooking. Additionally, some portion of cargo space is also reserved for mail and passenger baggage. Thus, the effective capacity available for cargo is called the "free sale", which can vary up to departure day.

[0006] Third, for cargo shipments, what matters is the source and destination. How the cargo is routed from source to destination is less of a concern as long as it reaches the destination on time. Rerouting, however, has ancillary costs as the shipment has to be stored in a warehouse.

[0007] Fourth, a unique aspect of the air-cargo ecosystem is that there is often a substantial discrepancy between the space booked by a shipping agent (in terms of volume and weight) for a particular item and the actual quantity that arrives on or just before the departure day. Airlines therefore tend to overbook flights under the assumption that the quantity that will arrive will be less than what was booked. Overbooking often leads to offloading, which increases costs in terms of storage and rerouting. Additionally, it is a conventional practice in the industry that airlines will not charge for the above-described discrepancy, which makes it difficult for airlines to manage their shipment capacity that is perishable in the sense that once a flight takes off no additional cargo may be added. Overall, this aspect of the air-cargo ecosystem creates inefficiencies in the market.

[0008] Fifth, passenger capacity is determined by a quantity of seats whereas cargo capacity is often volume-constrained. Moreover, an aircraft will reach volume capacity before it reaches weight capacity. Volume estimations tend to be less accurate than weight estimations, which adds difficulty to airlines managing their booked capacity prior to departure day.

[0009] Due to these fundamental differences, air-cargo management requires not only accurately predicting the quantity (e.g., weight and volume) of items that will be tendered by departure, but also requires making decisions on whether to accept or reject a particular booking for a flight. Such predictions and decision-making in the air-cargo setting, however, is challenging because data on booked cargo is often unreliable. Employees on cargo revenue teams often use their intuition to decide whether to accept a cargo booking for a flight or to reroute the cargo booking to a different flight. Accordingly, systems and methods for increasing the efficiency of selling cargo space, and therefore increasing revenue, are desired.

SUMMARY



[0010] The present disclosure provides new and innovative systems, methods, and non-transitory, computer-readable mediums that increase the efficiency of cargo revenue management. In an example, a system includes a processor in communication with a memory. The processor is configured to receive a cargo booking including a booked cargo volume and a booked cargo weight. The cargo booking is received a quantity of days until a cargo departure day. The processor is also configured to determine whether the booked cargo volume is a disguised missing value. The processor determines a predicted cargo volume of the cargo booking based on a set of features of the cargo booking. The set of features include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day. The processor is further configured to generate a decision to accept or reject the cargo booking based in part on the predicted cargo volume.

[0011] In an example, a method includes receiving a cargo booking including a booked cargo volume and a booked cargo weight. The cargo booking is received a quantity of days until a cargo departure day. It is then determined whether the booked cargo volume is a disguised missing value. A predicted cargo volume of the cargo booking is determined based on a set of features of the cargo booking. The set of features include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day. A decision is then generated to accept or reject the cargo booking based in part on the predicted cargo volume.

[0012] In an example, a non-transitory, computer-readable medium stores instructions. The instructions, when performed by a processor, cause the processor to receive a cargo booking including a booked cargo volume and a booked cargo weight. The cargo booking is received a quantity of days until a cargo departure day. The instructions further cause the processor to determine whether the booked cargo volume is a disguised missing value. The instructions also cause the processor to determine a predicted cargo volume of the cargo booking based on a set of features of the cargo booking. The set of features include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day. The instructions further cause the processor to generate a decision to accept or reject the cargo booking based in part on the predicted cargo volume.

BRIEF DESCRIPTION OF THE DRAWINGS



[0013] 

FIG. 1 illustrates a graph showing observed discrepancies between booked cargo volume and received cargo volume.

FIG. 2 illustrates a block diagram of a cargo booking system, according to an aspect of the present disclosure.

FIG. 3 illustrates a flow chart of a cargo revenue management method, according to an aspect of the present disclosure.

FIG. 4 illustrates a graph showing identified disguised missing values.

FIG. 5 illustrates a graph showing the importance of features utilized in determining a predicted cargo volume, according to an aspect of the present disclosure.

FIG. 6A illustrates a graph showing comparison data between utilizing booked cargo volumes and utilizing predicted cargo volumes with respect to offloading costs at various capacities.

FIG. 6B illustrates a graph showing comparison data between utilizing booked cargo volumes and utilizing predicted cargo volumes with respect to final revenue at various capacities.

FIG. 7 illustrates a scatter plot of flight data utilizing the presently disclosed system showing error percentage of predicted cargo volumes as compared to received cargo volumes, according to an aspect of the present disclosure.


DETAILED DESCRIPTION



[0014] Cargo booking data that airlines collect is replete with errors and often unreliable, thus making it difficult for airlines to efficiently sell their cargo space without under- or over-booking. Under-booking leads to a loss of revenue that could have otherwise been had, whereas over-booking results in off-loading costs that decrease revenue. For instance, capacity is a perishable quantity for any flight. Once the flight takes off, the capacity is lost. However, flight capacity is also a finite quantity. Once the capacity of the flight is reached, no more cargo can be added to the flight. Therefore, an airline wants to accept bookings that will maximize revenue.

[0015] One source of error is that shippers coordinate with freight forwarders and airlines using various communication means including email, phone calls, instant messages, or SMS messages, which can increase the possibility of data entry errors. For instance, a received cargo booking may be transferred incorrectly somewhere along the chain from a customer to an intermediary freight forwarder to an airline and into a central system using the various communication means. In another instance, a cargo booking may inadvertently be duplicated by being communicated through more than one communication means. There may also be a lag time between a cargo booking being received and the cargo booking being entered into the central system, which can affect intervening orders during that lag time (e.g., seemingly available space is booked that is not actually available).

[0016] Another source of error is that customers often book arbitrary but fixed values of cargo volume when the customers do not know the actual cargo volume that they will deliver, instead of indicating that the volume is unknown (e.g., NaN value). These arbitrary values have no bearing on the cargo volume that will be received, but will be misinterpreted as valid booked values if there is nothing to indicate to the airline that the values should not be treated as such. Accordingly, airlines may make predictions of how much cargo volume is left for sale on a flight based on values that do not reflect how much volume will be delivered for a given order. FIG. 1 illustrates a graph 100 showing discrepancies the inventors have observed between booked cargo volume and received cargo volume when analyzing airline cargo data. For instance, the graph illustrates how many data points deviate from the line 102 that indicates when a booked volume equaled a received volume.

[0017] As used herein, "disguised missing values" (DMVs) refer to unknown, inapplicable, or otherwise nonspecified data entries that are misinterpreted as valid data values. In the context of cargo booking, as stated above, DMVs may arise in cargo booking data when a customer does not know a cargo volume that the customer needs at the time of booking and instead submits an arbitrary cargo volume. For example, consider a set of six cargo bookings each having a booked volume of 10.23, but a received volume of 5.1, 2.8, 13.3, 26.4, 26.4, and 2.8, respectively. Unless other features can explain the diverse range of values for the received volume, the value 10.23 is very likely to be a DMV. One having skill in the art will appreciate the substantial effect that DMVs can have on data analysis outcomes, such as predictive machine-learning models.

[0018] For instance, the effect a DMV can have on a prediction may be seen by considering a linear regression model in a single dimension. Assuming a linear model y = wx is learnt from examples

then the Equation 1 below is well known.



[0019] Then, assuming xdmv and an associated set of values {y1, y2, ..., ym} are added to the training set, a new updated parameter wnew may be obtained, which is shown as Equation 2 below.



[0020] Depending upon the value of

the model might or might not be impacted by xdmv. For instance, if

then substituting in the above equation shows that wnew = w and the DMV has no impact on the model. However, if

deviates significantly from the straight line y = wx, the impact of xdmv can be large.

[0021] The presently disclosed cargo revenue management system and method increase the efficiency of cargo revenue management by increasing the prediction accuracy of cargo volumes that customers will tender in order to generate more efficient decisions to accept or reject cargo bookings. The provided system accomplishes this increased efficiency by identifying the above-described arbitrary, fixed booked values that customers book as disguised missing values and deemphasizing such values in the prediction of a cargo volume that will be received. The provided system additionally utilizes machine-learning models trained on a combination of features to predict a cargo volume that will be received for a particular cargo booking. Based on the predicted cargo volume that will be received, the system generates a decision of whether to accept or reject the cargo booking depending on whether the revenue from the booking and the value of accepting the booking outweigh the value of rejecting the booking. Accordingly, the presently disclosed system via the prediction models and decision generation helps increase the efficiency of the cargo revenue management process.

[0022] While the present disclosure is described in terms of air cargo, it should be appreciated that the provided systems and methods are also applicable, in some instances, to other forms of cargo transportation, such as ground transportation and sea transportation.

System Embodiment



[0023] FIG. 2 illustrates a block diagram of an example cargo booking system 200. The cargo booking system 200 includes an example cargo revenue management system 202. One or more devices 240, 242, 244 may communicate with the cargo revenue management system 202 to submit cargo bookings. For example, a customer may submit a cargo booking to the cargo revenue management system 202 via the laptop 240 or the smartphone 242 over the network 206. The network 206 can include, for example, the Internet or some other data network, including, but not limited to, any suitable wide area network or local area network. In another example, a customer may communicate with a cargo management employee (e.g., over a phone call, email, text message, etc.) and the cargo management employee may submit the customer's cargo booking to the cargo revenue management system 202 via the laptop 244 over a wired connection. The laptop 244 may additionally or alternatively communicate with the cargo revenue management system 202 over the network 206.

[0024] The cargo revenue management system 202 includes a processor in communication with a memory 210. In some aspects, the cargo revenue management system 202 may include a display 216 so that cargo bookings and other information may be viewed on the display 216. In other examples, the components of the cargo revenue management system 202 may be combined, rearranged, removed, or provided on a separate device or server. The processor may be a CPU 204, an ASIC, or any other similar device. The memory 210 stores a DMV database 212 that includes identified DMVs. The DMV database 212 may be continually updated as new DMVs are identified based on received bookings. The memory 210 may also store booking data 214 that includes data on all cargo bookings received by the cargo revenue management system 202. For instance, the booking data 214 includes cargo volumes that have been received each time a particular booked volume has appearing in a cargo booking. This data may be used to identify DMVs as described below. In various instances, some or all of the booking data 214 may be stored on one or more separate servers in communication with the cargo revenue management system 202. The booking data 214 may be used to continually train the algorithms and models of the cargo revenue management system 202 described herein.

[0025] A DMV detector 220 of the cargo revenue management system 202 may be programmed to identify whether a received cargo volume booking is a DMV. For instance, the DMV detector 220 may compare the received cargo volume booking with identified DMVs in the DMV database 212. The DMV detector 220 may also be programmed to continually identify new DMVs from received cargo bookings and from the booking data 214. The DMV detector 220 may update the DMV database 212 with a newly identified DMV that is not already in the DMV database 212. The DMV detector 220 may be implemented by software executed by the CPU 204.

[0026] The cargo revenue management system 202 may also include a cargo predictor 222 that is programmed to predict a cargo volume that will be received for a particular cargo booking. The predicted cargo volume may be determined via a machine-learning model trained on various features, as will be described in more detail in connection with FIG. 3. The machine-learning model of the cargo predictor 222 may be continually trained via the booking data 214. The cargo predictor 222 may be implemented by software executed by the CPU 204.

[0027] A decision generator 224 of the cargo revenue management system 202 may be programmed to generate a decision of whether to accept or reject a received cargo booking. The generated decision is based on the predicted cargo volume. The generated decision may also be based on an expected revenue of accepting the cargo booking, a value of accepting the cargo booking, and a value of rejecting the cargo booking. This will be described in more detail in connection with FIG. 3. In some aspects, the decision generator 224 may be programmed to generate an actionable decision that is displayed on the display 216. Stated differently, the actionable decision is a suggestion for an operator to accept or reject a particular cargo booking, which the operator may do by interacting with the actionable decision. In other aspects, the decision generator 224 may be programmed to automatically accept or reject a particular cargo booking based on the decision that is generated. The decision generator 224 may be implemented by software executed by the CPU 204.

Method Embodiment



[0028] FIG. 3 illustrates a flow chart of a cargo revenue management method 300, according to an aspect of the present disclosure. Although the example method 300 is described with reference to the flowchart illustrated in Fig. 3, it will be appreciated that many other methods of performing the acts associated with the method 300 may be used. For example, the order of some of the blocks may be changed, certain blocks may be combined with other blocks, and some of the blocks described are optional. The method 300 may be performed by processing logic that may comprise hardware (circuitry, dedicated logic, etc.), software, or a combination of both.

[0029] The method 300 may include receiving a cargo booking (block 302). For example, the DMV detector 220 may receive the cargo booking. The cargo booking is received a quantity of days until a cargo departure day. The cargo departure day is when a cargo shipment departs its origin for its destination. For example, the day a flight takes off, a truck leaves, or a ship departs its dock. No additional cargo may therefore be added to a shipment after the cargo departure day. The cargo booking includes information pertaining to the booking. For instance, the cargo booking includes a booked volume (e.g., m3 or ft3) of the cargo and a booked weight (e.g., lbs or kgs) of the cargo. The cargo booking may also include a quantity of items in the booked cargo, a shipment code, a cargo type, a cargo destination, and/or a cargo origin. In an example, the DMV detector 220 receives a cargo booking three days from the departure day that includes a booked volume (e.g., 10m3) and a booked weight (e.g., 50kg).

[0030] It may then be determined whether the booked cargo volume is a DMV (block 304). For example, the DMV detector 220 may compare the booked volume (e.g., 10m3) of the received cargo booking with the identified DMVs in the DMV database 212. If the value of the booked volume matches one of the identified DMVs in the DMV database 212, then the DMV detector 220 flags the booked volume as a DMV. If the booked volume does not match one of the identified DMVs, then it may be determined whether the booked volume is a newly identified DMV.

[0031] One method to identify whether a booked volume is a DMV is to consider a deviation of an average received volume for a particular booked volume from the particular booked volume. Stated differently, each cargo booking includes a cargo volume that was booked and a cargo volume that was received, and multiple cargo bookings over time may have the same particular booked cargo volume. Analyzing how an average of the received cargo volumes deviates from that particular booked cargo volume may help identify that the booked cargo volume is an arbitrary number that customers enter (e.g., a DMV) if the deviation is large enough.

[0032] For instance, returning to the example including a set of six cargo bookings each having a booked volume of 10.23, but a received volume of 5.1, 2.8, 13.3, 26.4, 26.4, and 2.8, respectively, the average of the received volumes is 12.8. The deviation g1 may then be determined according to Equation 3 below to obtain 6.60. In Equation 3, ui is the particular booked cargo volume, vi,k is a respective received cargo volume for the particular booked cargo volume, and ni is a quantity of received cargo volumes for the particular booked cargo volume. The determined deviation may then be compared to a predetermined threshold to determine whether the deviation is large enough to identify a DMV.



[0033] Another method to identify whether a booked volume is a DMV is to consider an entropy of the received cargo volumes for a particular booked volume. The entropy of the values is a measure of the order or predictability of the values. For instance, the higher the entropy of the received cargo volumes, the greater the unpredictability of the received cargo volumes, and the more likely it is that the particular booked volume is a DMV. In some examples, the entropy may be normalized, such as to upperbound the entropy at one. The normalized entropy g2 may be determined according to Equation 4 below, in which Vi consists of K distinct elements and Vi,k is the the k-th bucket of Vi. Moreover, ui is the particular booked cargo volume, Vi is a set of received cargo volumes for the particular booked cargo volume, K is a quantity of distinct elements of Vi, and ni is a quantity of received cargo volumes for the particular booked cargo volume. The determined entropy may then be compared to a predetermined threshold to determine whether the entropy is sufficient to identify a DMV.



[0034] In various aspects, a booked value is identified as a DMV only upon both g1, exceeding its predetermined threshold and g2 exceeding its predetermined threshold. In other aspects, a booked value is identified as a DMV upon g1 or g2 exceeding its predetermined threshold. It should be noted that prior cargo booking data (e.g., included in the booking data 214) may be analyzed according to the same methods to identify DMVs and populate the DMV database 212 with identified DMVs (e.g., FIG. 4). Once a booked volume is identified as a DMV, the value of the booked volume may be deemphasized when predicting a volume that will be received in subsequent instances in which the booked volume is received.

[0035] A predicted cargo volume of the cargo booking may then be determined based on a set of features of the cargo booking (block 306). For example, the cargo predictor 222 may determine a predicted cargo volume (e.g., 8m3) for the booked cargo volume (e.g., 10m3) that was received. The predicted cargo volume may be determined based on a machine-learning regression model fθ: X → R+, in which X is feature set, each booking i is mapped as an element xi ∈ X, and each received booking volume is mapped as yi ∈ R+. In some instances, the model is built using gradient boosting machines. In other instances, the model is built using random forests. Gradient boosting machines are ensemble methods and are known to perform well "out of the box". They can also easily handle a mixture of datatypes including numeric and categorical data. Random forests may reduce error by reducing variance as they combine independently generated deep trees on bootstrapped samples. Gradient boosting machines, however, reduce the bias by building shallow trees in a sequential manner, where each subsequent tree is trained by using the dependent variable as the residuals of the previous one.

[0036] The inventors have found that building the prediction model using gradient boosting machines may produce better predictions for purposes of the cargo revenue management method 300. For example, even though the prediction model is trained to make predictions at the booking level, the primary interest is in making flight level predictions (e.g., the total cargo volume of the flight is maximally utilized to maximum revenue), which are an aggregation of booking level predictions. Therefore, while gradient boosting machines predictions fluctuate more and individual predictions are further from the actual value, the differences cancel each other out at the flight level. Stated differently, the aggregation of bookings at the flight level will automatically result in variance reduction. The inventors have found in one evaluation that, on the booking level, the variance in gradient boosting machine predictions was more than five times higher than random forest predictions. At the flight level, however, the mean absolute error of random forest predictions was 87.1% higher than the gradient boosting machine prediction error. For these reasons, building the prediction model using gradient boosting machines may be preferable in some instances, though not limiting. In other instances, the prediction model may be built on other suitable machine-learning models.

[0037] The prediction model may be built having various features of a cargo booking in the feature set. In one example, the feature set includes whether the booked cargo volume is a DMV, the booked cargo weight, and the quantity of days until the cargo departure day. If a booked cargo volume is flagged as a DMV, then the prediction model deemphasizes the value of the booked cargo volume. Contrary to booked cargo volume, which usually tends to be a DMV, booked cargo weight can be valuable information. This is because shipping agents have more accurate information with respect to cargo weight since they have access to high quality weighing machines. Conversely, instruments for accurately measuring cargo volume are not readily available for many shipping agents. Thus, because it is easier to measure cargo weight, the booked cargo weight is on average more precise than the booked cargo volume.

[0038] The quantity of days until the cargo departure day may be included because the inventors have found that bookings closer to the departure day tend to be more accurate. In fact, the inventors have found that bookings that are timestamped several days before departure day tend to show a clear pattern of overbooking from the customer side (e.g., using DMVs). Therefore, the quantity of days until the cargo departure day may be the most important feature for predicting cargo volumes that will be received. One explanation is that it is natural for shippers to overbook since in the air-cargo business there is no penalty for overbooking, and shippers would rather have booked too much space than not enough.

[0039] In other aspects, the feature set may include one or more other features, including but not limited to a quantity of items in the booked cargo volume, the booked cargo volume, a shipment code, a cargo type, a cargo destination, and a cargo origin. The quantity of items in the booked cargo volume may be valuable to include in the prediction model because received cargo volumes tend to differ from booked cargo volumes not because a volume of an item was different, but rather because the quantity of items received is more or less than what was booked. Thus, the quantity of items in the booked cargo volume may be useful for predicting possible outcomes of cargo volumes that will be received. For example, if two pieces were booked having a volume of 12m3, with a volume for each piece of 6m3, it is unlikely that a single piece will be split and the received volume will become 4m3. It is in fact much more likely that the received cargo volume may be 6m3, 18m3, or 24m3.

[0040] The booked cargo volume, while in many instances a DMV, can still be valuable to include in the prediction model. This is because, when the booked cargo volume is not a DMV, the booked cargo volume tends to be precise. A cargo type refers to a type of product that the cargo is, such as fresh food, pharmaceuticals, electronics, etc. Various patterns with respect to particular cargo types may be observed and included in the prediction model. A shipment code is one or more codes that instruct how a shipment must be handled, e.g., live animals or perishables. The shipment code feature may replace or complement the product type feature. The shipment code may be encoded as a binary vector with one element for each shipment code (e.g., one-hot encoding).

[0041] A cargo destination (e.g., where the cargo is going) may be included in the prediction model because, in some instances, while the cargo destination is a weak predictor for a volume that will be received, the cargo destination may help elicit subtypes within cargo types, which may help reduce variance in predictions. Similarly, the cargo origin (e.g., where the cargo is leaving from) is a weak predictor for a volume that will be received. The cargo origin, however, may help to capture the average behavior of booking agents from a particular origin, which may help reduce variance in predictions. Accordingly, it may be valuable to include the cargo origin and destination as features in the prediction model.

[0042] The features included in the predication model's feature set may each be weighted differently with respect to their respective effects on the predicted cargo volume. For instance, each feature may be weighted according to the feature's importance (e.g., FIG. 5) in determining a predicted cargo volume.

[0043] A decision to accept or reject the cargo booking is then generated based on the predicted cargo volume of the cargo booking (block 308). For example, the decision generator 224 may generate a decision to accept or reject the cargo booking based on the cargo volume (e.g., 8m3) predicted by the cargo predictor 222. For any flight, capacity is a perishable quantity (e.g., once the flight takes off, capacity is lost). Therefore, the decision making problem can be seen as a generalization of the classic Knapsack problem with two caveats: (i) cargo bookings appear over time and (ii) the exact volume and weight of the shipment become available only at departure time. Accordingly, the decision to accept or reject a cargo booking may be modeled as a stochastic dynamic program.

[0044] To model an example stochastic dynamic program, a state vector x = (xi, ..., xm) may first be defined. Each xi is the number of items of type i assigned to a flight. As stated above, a cargo type is a predefined category, such as fresh food or pharmaceuticals. The state vector x evolves with time t. A value function VF(x, t) may be defined as the expected revenue from a flight given that at time t the flight is in state x. The departure day may be labeled as time t = 0 and the booking horizon (e.g., a quantity of days until the departure day) extends up to time t = T. Accordingly, time flows backwards in this stochastic dynamic program. In this example, a single flight is modeled with volume capacity kv that is fixed and known. In other examples, a volume capacity of a flight may depend on factors beyond the size of an aircraft, such as for flights that are not strictly cargo flights but carry passengers as well. In such examples, the volume capacity kv varies based on passenger load for a particular flight.

[0045] Returning to the example stochastic dynamic program, time may also be discretized, and in each time bin t the probability of an item i being received for a booking is pi,t. It can be assumed that at each time step only one shipment can arrive for booking. The probability that no booking will show up in time period t may be defined as

In practice, when an agent books an item of type i, it is accompanied by a booked volume bkvoli. When the item finally arrives for shipment, the received volume is rcsvoli. The revenue received from the item i is R(rcsvoli), where R() is typically an increasing and concave function of volume. For instance, the more cargo volume that is received, the more payment that is received from customers, up until a volume capacity of an aircraft is reached. During booking time, the airline is only aware of the booked volume bkvoli and not the received volume rcsvoli. A decision about whether to accept or reject a booking may therefore be based on an average volume of type i, vi The value function VF(x, t) may then be defined as a recursive function (Bellman's Equation) shown in Equation 5 below in order to maximize the overall expected revenue from a flight. The value function at the time of departure VF(x, 0) may be defined as shown in Equation 6 below.





[0046] To further explain the value function, when the state is x at a given time t, then VF(x, t) is the expected revenue over the full time horizon of the booking. At time step t, the probability of a shipment of type i arriving is pi,t. If the booking is accepted, then the state will transition to x + ei, where ei is a one-hot binary vector with a 1 at the i-th location. By accepting the booking, the expected revenue is R(vi). To maximize revenue, however, the booking of item i will only be accepted if the expected revenue plus the value of accepting the booking, e.g., R(vi) + VF(x + ei, t - 1), is greater than the value of not accepting the booking and transitioning one step towards departure day while staying in the same state, e.g., VF(x, t - 1). This revenue maximization decision rule D1V to determine whether to accept or reject an incoming shipment of type i may be defined as shown in Equation 7 below. At time t = 0 and in state x, VF(x, 0) in Equation 6 captures the cost of off-loading, which is proportional (e.g., the proportional value hv) to the total expected volume ∑ixivi minus the capacity kv. For example, if the expected volume is 100 units and the capacity kv is 50 units, then the off-loading cost is -50hv.



[0047] The prediction model described above for predicting a cargo volume that will be received may be integrated into the decision rule D1V to obtain the decision rule D2V shown in Equation 8 below. The prediction model is fθ such that given a booked volume bkvoli of type i, fθ(bkvoli) is the predicted received volume (rcsv̂oli). In some examples, therefore, the decision to accept or reject the cargo booking is generated based on the decision rule D2V. For instance, the decision generator 224 may be programmed to generate a result of the decision rule D2V. If the expected revenue of the predicted cargo volume plus the value of accepting the predicted cargo volume of the booking, e.g., R(fθ(bkvoli)) + VF(x + ei, t - 1), is determined to be greater than the value of not accepting the booking and transitioning one step towards departure day while staying in the same state, e.g., VF(x, t - 1), then a decision is generated to accept the cargo booking. If the opposite is true, then a decision is generated to reject the cargo booking.



[0048] In some aspects, the value function may be modified to help limit or eliminate the effects of the curse of dimensionality of dynamic programming. In such instances, for example, suppose there are m items and the number of time periods is T. Then, the size of the state space is exponential in m. For instance, the size of the state space is S(T, m), Stirling number of second kind. An approximate solution to escape the exponential blow-up is to use aggregate x = Σixivi. This makes the state space one-dimensional scalar-valued, instead of vector-valued, of maximum size. This state space is bounded by M × T, where M is the maximum possible volume booked for any type. In this way, the construction of the value function VF(x, t) becomes simplified. In some examples, therefore, the decision to accept or reject the cargo booking is generated based on the decision rule D2S, which is shown in Equation 9 below. For instance, the decision generator 224 may be programmed to generate a result of the decision rule D2S.



[0049] To help illustrate how dynamic programming is used to form the value equation, an example situation is shown in Table 1 below. It is assumed that there are two cargo types: type 1 and type 2. Both types can arrive for a booking with a probability of 0.4 in any time step and the probability that no shipment will arrive for any booking is 0.2. The revenue for type1 is 1 and for type2 is 2, while the volume for both types is fixed at 1 unit each. Recall that time is labeled in a reverse order, such that departure time is zero and the booking horizon extends up to time t = 4. To compute the value function VF, a procession backwards for each state x may be performed. At this point, the state is a two-dimensional vector x = (x1,x2), where x1 and x2 are the number of bookings of type1 and type2 respectively. In this example, the vector x may be collapsed into x = x1v1 + x2v2 = x1 + x2 since it is assumed that the volume booked has a value of 1. The different values of x are shown as rows in Table 2 below for the value function according to the situation shown in Table 1. An example computation of VF(1,2) is shown below Table 2.
Table 1.
  type1type2
1 volume 1 1
3 revenue (ρ) 1 2
4 prob. arrival in t 0.4 0.4
5 prob. no booking in t 0.2
6 max capacity (kv) 2
Table 2.
 t=0t=1t=2t=3t=4
x=0 0.0 1.2 2.4 3.1 3.6
x=1 0.0 1.2 1.8 2.2 0.0
x=2 0.0 0.4 0.8 0.0 0.0
x=3 -1.0 -0.6 0.0 0.0 0.0
x=4 -2.0 0.0 0.0 0.0 0.0




[0050] In some aspects of the present disclosure, the generated decision to accept or reject a booking may be a suggestion or an actionable decision that an operator (e.g., an airline employee) may consider when the user makes an independent decision of whether to accept or reject the booking. For instance, the decision generator 224 may be programmed to generate a message or an actionable decision that is displayed on the display 216. The message may be text that informs the operator of the generated decision. An actionable decision may inform the operator of the generated decision and also enable the operator to execute the generated decision by interacting with the actionable decision (e.g., selecting a link or computer-generated button). In other aspects of the present disclosure, the generated decision to accept or reject a booking may automatically accept or reject the booking without further input from an operator. For example, the decision generator 224 may be programmed to automatically accept or reject a particular cargo booking based on the decision that is generated. This automatic decision may occur shortly after receiving the particular cargo booking or may occur at a point in time closer to the cargo departure day.

[0051] In at least one example, the decision generation may be adjusted for various business objectives. For example, the predictive modeling and decision generation rule may be designed so that excess overbooking incurs a negative penalty (e.g., offloading), while underbooking is not penalized (e.g., VF(x, 0) is zero when the total volume is less than the flight capacity). In such examples, it may be more beneficial to reduce the risk of offloading by using a predictive model that may over predict cargo volume that will be received, which leads to less shipments getting accepted. In other examples, the predictive model may be designed to under predict cargo volume that will be received so that flights are more often at full capacity than they are not. Other business objectives may result in similarly suitable adjustments to the predictive modeling and decision generation.

[0052] FIG. 4 illustrates a graph 400 showing identified disguised missing values based on applying Equations 3 and 4 above to cargo booking data (e.g., the booking data 214). The graph maps each distinct value from the booking data into a two-dimensional feature space so that predetermined threshold values for g1, and g2 may be visually determined. For instance, the graph 400 may be used to identify DMVs from an initial set of booking data to populate the DMV database 212. Every value that crosses the determined thresholds is identified as a DMV. For example, the values DMV1, DMV2, DMV3, DMV4, and DMV5 are identified as DMVs in the graph 400.

[0053] FIG. 5 illustrates a graph 500 showing the importance the inventors have found of features that may be included in the prediction model described above. DAYS refers to the quantity of days until the cargo departure day. BKWT refers to the booked cargo weight. PIECES refers to the quantity of items in the booked cargo volume. BKVOL refers to the booked cargo volume. SHC refers to the shipment code. PRODUCT refers to the product type. DMV refers to whether a booked cargo volume is flagged as a DMV. DEST refers to the cargo destination. ORIG refers to the cargo origin. For categorical features, such as the product type, the value of the category with the maximum important is illustrated in the graph 500. The importance of each feature may be taken into account when weighting the effect of each feature in the prediction model. As seen in the graph 500, the inventors have found that the quantity of days until the cargo departure day is the most important feature by a significant margin as compared to the differences between the other features.

Validation Data



[0054] To validate the advantages of the presently disclosed cargo revenue management system and method, the inventors tested the system using two years of cargo booking data from an airline company. Each booking record consists of several attributes including: booking date, origin, destination, agent, booking volume (bkvol), cargo type, received date, departure date and time, and received volume (rcsvol). This dataset was used to detect DMVs and build the machine-learning model for predicting rcsvol using all other attributes. Simulated data was used for the revenue and offload cost information to evaluate the decisions generated by the provided system. To create the simulations from the real dataset, probabilities pi of the product types from the real dataset were computed for each type i as shown in Equation 10 below.



[0055] The related probabilities of the ten most frequent cargo types are shown in Table 3 below. It was observed that the cargo type frequencies are skewed, with the most frequent cargo type having a probability of 0.856. The booking horizon was split into sixty equal time steps and the probability of a booking arriving at time step t was computed by Equation 11 below. A single probability for six different intervals of time steps was computed, resulting in ten time steps per interval. For each interval an average of the ten single time steps was taken that belong to this interval. The results are shown in Table 4 below. Given the probability pi of an incoming type i at any time, and the probability pt of getting any type of booking at time t, the probability of getting a booking of product type i at time t is pi,t = pipt.
Table 3.
Cargo TypeType1Type2Type3Type4Type5Type6Type7Type8Type9Type10
pi 0.856 0.042 0.036 0.035 0.012 0.007 0.005 0.003 0.002 0.002


Table 4.
Time Period1-1011-2021-3031-4041-5051-60
pt 0.05 0.03 0.009 0.004 0.003 0.005


[0056] The provided system was first evaluated in terms of revenue benefits and offload costs. The value function VF(x, t) table was computed by using the entire dataset and taking the average booking volume for each time step. Then, two different test cases were considered in order to evaluate the advantage of the provided system that combines predictive modeling with decision making. In the first test case, no predictive modeling was done. The decision of whether to accept or reject an incoming booking by applying D2S was based on the reported booked cargo volume, i.e., fθ (bvi) = bVi. Final offloading cost was then calculated based on the received cargo volume. In the second test case, the received bookings were processed for DMV identification and the presently disclosed predictive modeling generated a predicted cargo volume that could be expected to be received. The decision of whether to accept or reject an incoming booking by applying D2S was based on the predicted cargo volume.

[0057] At each time step, bookings were drawn from the dataset following the probabilities of Table 3 and applying the decision rule D2S. FIGS. 6A and 6B show the results of eleven different flight capacities kv and ten thousand flights each, for a total of 220 thousand flights. In particular, FIG. 6A illustrates a graph 600A showing comparison data between utilizing booked cargo volumes and utilizing predicted cargo volumes with respect to offloading costs at the eleven capacities. The graph 600A shows that for various capacity constraints (kv) the offloading cost is lower almost by a factor of ten and with a much lower standard deviation. This suggests that using the predictive modeling of the provided system instead of booked volume not only reduces the offloading cost but adds substantial amount of certainty into the whole air cargo booking process.

[0058] FIG. 6B illustrates a graph 600B showing comparison data between utilizing booked cargo volumes and utilizing predicted cargo volumes with respect to final revenue at the eleven capacities. It was observed that the revenue increases when using the predictive modeling of the provided system, indicating that the decision function selected better-value shipments during the booking time horizon. Additionally, the standard deviation of the revenue is lower when using the presently disclosed system, thus providing increased certainty to the air cargo booking process. In this example, the predictive model of the system was designed such that it may over predict cargo volumes, leading to less shipments getting accepted.

[0059] The predictive model and the decision generation of the provided system were then evaluated. To evaluate the predictive model, a 3-fold cross-validation was used on the dataset of two years of cargo bookings. A prediction was made on each single booking and the aggregated flight leg predicted volume vs. the flight-leg received volume was evaluated. Cross-validation was implemented so that all the bookings from the same flight leg are kept in the same split. Based on grid-search results, the XGBoost regressor was set with 0.9 subsample ratio of columns for each split, 300 estimators, a maximum tree depth of 20, and a learning rate of 0.05. All other parameters were set as default.

[0060] The mean relative absolute error e shown in Equation 12 below was used on the predictive model of the provided system. The average error on the entire historical data was 7.8%. FIG. 7 illustrates a scatter plot 700 showing error percentage of predicted cargo volumes as compared to received cargo volumes. The scatter plot 700 shows that the prediction error is under 5% in almost half of the flights, while it is under 10% for 74.8% of the flights. The table 702 illustrated in FIG. 7 shows this data. It was observes that by using the provided system's predictive model instead of the actual booked volume values, a greater number of flights have a small error. It is also noteworthy that the prediction error is lower for higher capacity flights, where it has the biggest impact.



[0061] Table 5 below shows the benefits of the provided system's prediction on the shipment level for the ten most frequent cargo types. Specifically, it shows the decrease in percentage of the prediction error (predicted rcsvol vs. actual rcsvol) from the booking error (original booked volume vs. actual rcsvol). The predicted volume decreased the error considerably for nine out of the ten cargo types. The cargo type9 was a rare cargo type that did not have enough data to train the predictive model. Because cargo type9 was rare it did not influence the total flight-leg predicted volume.
Table 5.
Product type% of booking% error decrease from bkvol
Type1 73.3% -33.4%
Type2 10.1% -38.3%
Type3 3.7% -61.9%
Type4 3.5% -66.9%
Type5 3.1% -34.1%
Type6 2.7% -7.2%
Type7 1.4% -43.9%
Type8 0.8% -96.6%
Type9 0.7% +25.8%
Type10 0.1% -30.7%


[0062] To evaluate the decision generation of the provided system isolated from the predictive modeling, predictions were not made, but rather decisions were made based on the mean value vi. Offloading was determined based on the random number generated by a lognormal distribution, which leads to a decision rule of D3S: R(vi) + VF(x + vi, t - 1) > VF(x, t - 1). The decision rule D3S was evaluating by comparing it with a first-come first-served (FCFS) policy. In the FCFS policy, every incoming booking is accepted until the capacity runs out. FCFS is a greedy strategy in the sense that it will accept immediate revenue instead of waiting for a potential booking from which more revenue can be made. In an ideal setting, if the booking value was equal to the received value, FCFS will not incur any offloading costs. However, since the two values are rarely the same, the natural advantage of FCFS is not realized and the experiments reveal this. The decision rule D3S and the FCFS strategy were compared based on a simulated data set created using the mean volume µk for each cargo type k of the 24 categories and the related probability distributions. Simulated data was created for ten thousand different simulations. Mean value is used as booked volume, while the received volume is drawn from a lognormal distribution with mean equal to the booked volume and variance (θµk)2.

[0063] Results in Table 6 below show how the decision generation of the provided system has a benefit not only for the expected revenue but also the final revenue after the offloading cost is deducted. As the variance increases, offloading increases and revenue decreases, which suggests that the provided system can better handling the booking decision making process as compared to the FCFS approach.
Table 6.
 Expected revenueRevenue after offloading
θ D1S FCFS D1S FCFS
0.8 2927.87 2548.95 2584.28 2541.32
1.0 2927.87 2548.95 2553.74 2517.23


[0064] Table 7 below provides the last sixteen steps in the booking horizon for one of the flight simulations in the simulated dataset. In particular, it shows how, in practice, the decision rule D3S keeps overbooking if the revenue rate is advantageous, while rejecting the less profitable shipments once the capacity is reached. This leads to the accumulation of offloading cost with the ultimate goal of maximizing the revenue.
Table 7.
   FCFSD1S
tbkvolRevenue rateLoadDecisionLoadDecision
45 55.0 0.69 2759.0 accepted 2759.0 accepted
46 59.0 0.79 2814.0 accepted 2814.0 accepted
47 52.0 0.72 2873.0 accepted 2873.0 accepted
48 30.0 1.12 2925.0 accepted 2925.0 accepted
49 59.0 0.54 2955.0 rejected 2955.0 rejected
50 119.0 0.98 2955.0 rejected 2955.0 accepted
51 30.0 1.12 2955.0 rejected 3074.0 accepted
52 30.0 1.12 2955.0 rejected 3104.0 accepted
53 30.0 1.12 2955.0 rejected 3134.0 accepted
54 52.0 0.8 2955.0 rejected 3164.0 rejected
55 27.0 0.8 2955.0 rejected 3164.0 rejected
56 52.0 0.8 2955.0 rejected 3164.0 rejected
57 125.0 0.69 2955.0 rejected 3164.0 rejected
58 30.0 1.12 2955.0 rejected 3164.0 accepted
59 30.0 1.12 2955.0 rejected 3194.0 accepted
60 30.0 1.12 2955.0 rejected 3224.0 accepted


[0065] As used herein, "about," "approximately" and "substantially" are understood to refer to numbers in a range of numerals, for example the range of-10% to +10% of the referenced number, preferably -5% to +5% of the referenced number, more preferably -1% to +1% of the referenced number, most preferably -0.1% to +0.1% of the referenced number.

[0066] Furthermore, all numerical ranges herein should be understood to include all integers, whole or fractions, within the range. Moreover, these numerical ranges should be construed as providing support for a claim directed to any number or subset of numbers in that range. For example, a disclosure of from 1 to 10 should be construed as supporting a range of from 1 to 8, from 3 to 7, from 1 to 9, from 3.6 to 4.6, from 3.5 to 9.9, and so forth.

[0067] As used herein and in the appended claims, the singular form of a word includes the plural, unless the context clearly dictates otherwise. Thus, the references "a," "an" and "the" are generally inclusive of the plurals of the respective terms. For example, reference to "an airline" or "a cargo type" includes a plurality of such "airlines" or "cargo types." The term "and/or" used in the context of "X and/or Y" should be interpreted as "X," or "Y," or "X and Y."

[0068] Without further elaboration, it is believed that one skilled in the art can use the preceding description to utilize the claimed inventions to their fullest extent. The examples and aspects disclosed herein are to be construed as merely illustrative and not a limitation of the scope of the present disclosure in any way. It will be apparent to those having skill in the art that changes may be made to the details of the above-described examples without departing from the underlying principles discussed. In other words, various modifications and improvements of the examples specifically disclosed in the description above are within the scope of the appended claims. For instance, any suitable combination of features of the various examples described is contemplated.


Claims

1. A system for managing cargo bookings, the system comprising:

a memory; and

a processor in communication with the memory, the processor configured to:

receive a cargo booking including a booked cargo volume and a booked cargo weight, wherein the cargo booking is received a quantity of days until a cargo departure day,

determine whether the booked cargo volume is a disguised missing value,

determine a predicted cargo volume of the cargo booking based on a set of features of the cargo booking that include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day, and

generate a decision to accept or reject the cargo booking based in part on the predicted cargo volume.


 
2. The system of claim 1, wherein the memory stores a database of identified disguised missing values, and wherein determining whether the booked cargo volume is a disguised missing value includes comparing the booked cargo volume to the database of identified disguised missing values.
 
3. The system of claim 2, wherein the processor is further configured to:

identify new disguised missing values; and

update the database of disguised missing values as new disguised missing values are identified.


 
4. The system of claim 3, wherein new disguised missing values are identified based on at least one of a deviation of an average received cargo volume for a particular booked cargo volume from the particular booked cargo volume exceeding a first predetermined threshold and an entropy of the received cargo volumes for the particular booked cargo volume exceeding a second predetermined threshold.
 
5. The system of claim 4, wherein the deviation g1 is defined by the following equation:

wherein

ui is the particular booked cargo volume,

vi,k is a respective received cargo volume for the particular booked cargo volume, and

ni is a quantity of received cargo volumes for the particular booked cargo volume.


 
6. The system of claim 4, wherein the entropy g2 is defined by the following equation:

wherein

and wherein

ui is the particular booked cargo volume,

Vi is a set of received cargo volumes for the particular booked cargo volume,

K is a quantity of distinct elements of Vi, and

ni is a quantity of received cargo volumes for the particular booked cargo volume.


 
7. The system of claim 1, wherein determining the predicted cargo volume is based on one or more gradient boosting machines.
 
8. A method for managing cargo booking, the method comprising:

receiving a cargo booking including a booked cargo volume and a booked cargo weight, wherein the cargo booking is received a quantity of days until a cargo departure day,

determining whether the booked cargo volume is a disguised missing value,

determining a predicted cargo volume of the cargo booking based on a set of features of the cargo booking that include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day, and

generating a decision to accept or reject the cargo booking based in part on the predicted cargo volume.


 
9. The method of claim 8, wherein the set of features further includes one or more of the group consisting of: a quantity of items in the booked cargo volume, the booked cargo volume, a shipment code, a cargo type, a cargo destination, and a cargo origin, or
wherein each feature of the set of features includes a significance factor for determining the predicted cargo volume, and wherein the quantity of days until the cargo departure day from receiving the cargo booking has the greatest significance factor, preferably wherein
the booked cargo weight has the second greatest significance factor.
 
10. The method of claim 8, wherein generating the decision is based on maximizing an expected revenue of a total cargo volume including a plurality of cargo bookings, preferably wherein
the total cargo volume is a fixed volume.
 
11. The method of claim 10, wherein a decision to accept the cargo booking is generated if a sum, of a value of accepting the predicted cargo volume and the expected revenue by accepting the predicted cargo volume, is greater than a value of rejecting the predicted cargo volume.
 
12. The method of claim 11, wherein the value of accepting or rejecting the predicted cargo volume is defined by the following recursive function:

wherein t = 1,2, ..., T, wherein x is a state vector (xi, ..., xm), and wherein

t is a quantity of days remaining until the cargo departure day,

T is the quantity of days from receiving the cargo booking until the cargo departure day,

i is a quantity of cargo types in the total cargo volume,

xi is each respective cargo type of the quantity of cargo types,

R(vi) is the expected revenue by accepting the predicted cargo volume,

pi,t is a probability of a cargo type i arriving at time t, and

ei is a one-hot binary vector with a 1 at the i-th location.


 
13. The method of claim 12, wherein the decision is generated based on the following relationship:

wherein

R(fθ(bkvoli)) is the expected revenue by accepting the predicted cargo volume,

fθ(bkvoli) is the predicted cargo volume,

VF(x + ei, t - 1) is the value of accepting the predicted cargo volume, and

VF(x, t - 1) is a value of rejecting the predicted cargo volume.


 
14. The method of claim 12, wherein the decision is generated based on the following relationship:

wherein

vi is a respective received cargo volume,

R(fθ(bkvoli)) is the expected revenue by accepting the predicted cargo volume,

fθ(bkvoli) is the predicted cargo volume,

VF(x + ei, t - 1) is the value of accepting the predicted cargo volume, and

VF(x, t - 1) is a value of rejecting the predicted cargo volume.


 
15. A non-transitory, computer-readable medium storing instruction which, when performed by a processor, cause the processor to:

receive a cargo booking including a booked cargo volume and a booked cargo weight, wherein the cargo booking is received a quantity of days until a cargo departure day,

determine whether the booked cargo volume is a disguised missing value,

determine a predicted cargo volume of the cargo booking based on a set of features of the cargo booking that include at least whether the booked cargo volume is a disguised missing value, the booked cargo weight, and the quantity of days until the cargo departure day, and

generate a decision to accept or reject the cargo booking based in part on the predicted cargo volume, preferably wherein

generating the decision includes automatically accepting or rejecting the cargo booking without further input, or
wherein the predicted cargo volume is an output of a regression model including the set of features, and wherein the regression model is continually trained via received cargo bookings and respective received cargo volumes corresponding to each received cargo booking.
 




Drawing

























Search report









Search report




Cited references

REFERENCES CITED IN THE DESCRIPTION



This list of references cited by the applicant is for the reader's convenience only. It does not form part of the European patent document. Even though great care has been taken in compiling the references, errors or omissions cannot be excluded and the EPO disclaims all liability in this regard.

Patent documents cited in the description