Effective optimization: Proposal & Optimizer – the (un)-similar brothers
In this blog we want to have a look into the proposal and what’s it makes different from the „usual“ optimizer. Let’s start why we use the optimizer:
The goal of the optimizer is to find the best plan in term of costs in a given run-time, to transport a set of freight units
The set can contain only very view Freight Units or several thousand. Of course the driving demand must not be a Freight Units only, but also container or rail-cars, which must be transported. In this blog I speak only about the Freight Units to keep it easy. The simple word “best” suggest a lot of expectation. In reality it is only the best plan which can be found in the runtime, which of course should be very near to the really best plan. The costs contain the freight cost and penalty cost for delays or not planned freight as well.
Example of an proposal call for a FU from Shanghai to Walldorf-Germany. We see different alternatives with Air, Ocean and Rail on the Main-leg
Now let’s look to the proposal:
The goal of the proposal is to find several alternatives to plan one single freight unit, whereat
- The alternatives must be different to each other
- The alternatives must be quite good
- The response time must be quite fast
Let’s start with point 3): We have currently two main uses-cases for the proposal: It can be used as manual working step. If a Freight Unit causes some trouble you can use the proposal to get planning alternatives for the complete Freight Unit or for one or some stages of it. In this case you see the different alternatives and you can select one of them. The second use-case is a call inside the Sales-Order-Scheduling. From ERP we receive a call to create an exact planned and checked proposal for this Freight Unit. Again the proposal creates several alternatives and as result the cost optimal solution is selected. In both scenarios it is clear, that the response times must be short, at max a few seconds.
Which alternatives do we want to see as result of a proposal call? The points 1) and 2) together define the goal. It makes no sense to have very similar solutions, even if they are all good. For example you use a schedule for that Freight Unit and you vary the departure times only. So we need as different as possible solutions: different routings through the network, different carriers, different modes of transport and finally different resources and dates.
If this diversity would be the one and only goal, it would be very simple: You just look for very different routings. Let’s have an FU from Hamburg to New-York as example. Of course you would search for a direct transport. Then you look for a second with an intermediate stop in Africa, then with one in Australia and so on. So we end up with very different alternatives to transport the FU.
But of course you are not interested in that. You want to have meaningful and good alternatives. Because of that we need the requirement 2). Now it is not that clear, what we mean, when we say “good”. For the optimizer we said, “good” means minimal coast. This makes sense for the optimizer, as we changed and consolidate all FU’s to get a good foundation to calculate the cost of the plan. If we do the same for the one new FU in the proposal, we have two drawbacks: We cannot estimate, which FU’s will come next, so we cannot really consolidate like in the optimization case. If we look at the manual use-case you often will call the proposal if you have problems, perhaps because of delays in execution. In these cases you may be much more interested in the fastest transport, to reduce the remaining delay as much as possible. The pure cost may become less relevant.
You can define the behavior of the proposal in planning Profile on the optimization tab. In the default view you see mainly the possibility to define numbers of alternatives you want to see.
If you press the button for advances setting at the top, you get the detailed configuration.
In the two fields “Time Relevance” and “Cost Relevance” you define, how important the different criteria of “good proposal” in point 2) are. You can set the focus on cost or on times and delays or you use the balanced view of the default settings.
In the three variation-fields above, you can define, which kind of difference in the alternatives are important to you. For air or ocean scenarios it might be very important to vary the different carriers, whereas a road scenario wants to vary the routing via different hubs in the different alternatives.
I recommend to start with the default settings always. If you then want to see more different carriers, or more different something else, you can increase this specific value a little bit. Or -even simpler- you can increase the number of alternatives in the standard settings.
How Is the Proposal working internally?
As you may already know the optimizer works with a search meta-heuristic. It creates and modifies plans to improve them incrementally. The proposal works different. We want to use the very short runtime as efficient as possible and don’t want to wait for a search heuristic. So we construct the different alternatives in a very controlled manner to get the wanted variations and “good”-definitions as soon as possible.
In a first step the proposal determines the possible routings to transport the freight. A routings means the stages to connect the start location with the end location of the proposal call. It contains all the hub and stages between. In this step the proposal considers already the incompatibilities or other constraints, which can eliminate a stage or route, so that already a lot of routs can be sorted out. This step is common with the preprocessing in the optimizer. It considers also default tours.
Then the proposal estimates the quality of the routings. A routing is good, if it contains no or few hubs, if it has low cost and fits to the dates like described in the relevance-fields and if it increases the variability defined in the variance-fields.
After that we start to create concrete solutions along the best routings. If we focus only on the best routings or if we consider more or all routings depends on the field “Route Variation” in the profile. If such a routing contains several stages, we start with the longest stage and from that one we go to the stages before and after to plan the other stages without unnecessary waiting-times. For planning such routes the proposal reuses the services of the optimizer.
Of course this is only high level, but I want to keep the details like the assessment ofthe routings or the variation method secret. The important point is: It is a special algorithm to construct the alternatives, it is not the search of the optimization process.
Several FU’s in the proposal?
So far I described the proposal as a service for one single Freight Unit. But you can even call it, with several Freight Units. What’s happening?
In general it’s doing a proposal call for every Freight Unit separately. Then it’s combining the different alternatives for the different FU’s to several alternatives for the complete set of FU’s. At the end you receive alternatives, which all contain the complete set of Freight Units.
Often the user will not like these results: If you put several FU’s into a call of the proposal, you will expect a good consolidation. But as every FU is planned separately, you will end up in the wrong decisions to get a good consolidation in the merge at the end. If consolidation is needed, use the optimizer!
The proposal is a tool designed for one single Freight Unit, not for several ones.
The proposal as debugger for the optimization
On one side the proposal shares a lot of code with the optimizer. Especially, the feasibility-checks, and the detailed construction heuristics are common. On the other side it is doing a much smaller job and you can easily influence and control it in the proposal screen.
Together this is a good opportunity to answer the most frequent question about the optimizer: ”Why is my freight unit not planned in the optimization run?” If there is no direct answer in the explanation tool of the optimizer, call the proposal for that FU.
If the proposal plans, it, but not the optimizer, then most probably the runtime for the optimizer was too short to plan all FU’s. But in nearly all other cases the proposal will not plan it,too.
Then the proposal can help you to find the reason for the problem in planning: You have a solution in mind for your FU. Let’s start and add part’s of your solution as additional preferences in the proposal screen. This can help the proposal to find the correct solution. In this case we have to think about the heuristics in optimizer and proposal why they overlooked the solution without the preferences. Perhaps it’s because of the planning cost, which you can see in the proposal results. Or we have to improve the heuristics and it’s time to open a ticket for a correction.
But if the proposal still cannot find a solution or not the expected one, you might get much more detailed information in the protocol or the explanation tool as possible in an optimization run. Often this is the easiest way to find defect in the data model, for example in the transportation network.
Remarks – What’s else to say about the proposal?
- Usually the proposal creates new freight orders and is not consolidation to existing freight orders. You can change that if you activate incremental planning. Then the FU of the proposal can be added to existing FOR’s. this requires, that all these FOR’s are transferred to the proposal, this can increase the runtime of a proposal-call in a significant manner.
- The proposal is build in the same technology like the optimizer, but it is a different approach. Because of that you see it as separate service with a separate rfc-connection and a separate entry in the RCC-Framework (rcc_custom).
Today the service is implemented in the same executable like the optimization process. So the Rfc-connections of the optimization and the proposal direct usually to the same executable.
- In a good system setup an average proposal call is somewhere in the range of 1-2 seconds. Given such short run times, it is important to install the executable of the proposal on a local disk. I saw several installations, with the proposal installed on a network drive. Then for every call the executable has to be transferred to the executing server and in most cases the antivirus software had to check it before we could start. That lost unnecessary 0.5-2 seconds.
- Often I heard the question, if it is better to install the proposal on the application server. If you have only one Application Server this may help a very little. But the effect is much smaller than the local installation vs. network drive.
- When the proposal is called it is handled like any optimization call, we select a server and we register it in the RCC-Framework. Here is a little serial bottleneck. If you plan to do around or more than 100 Proposal-Calls per seconds (perhaps triggered by SOS), then you have to carefully set up your system to get the required through put in the administration tables of RCC.
- As the runtime is short and the resource consumption not very high, it is common usage to have no load balancing on the proposal, but maintain one RCC-connection with an unlimited number of slots.
Finally, my proposal to use the proposal:
- Use it with ONE Freight Unit only.
- Start with the default settings
- If you miss the wished alternative, first increase the number of requested alternatives.
- If you still miss it, try to add to additional constraint and stop, to see if the proposal finds it now and if it is possible at all.
- If it was found with these additional constraints steps, adapt the expert settings for relevance and variation in small steps to archive
the requested behavior.