Dynamic Programming Essay

Dynamic Programming is a mathematical technique dealing with the optimization of multistage decision processes. In this technique, decisions regarding a certain problem are typically optimized in stages rather than simultaneously. This generally signifies that the original decision problem is divided into small sub-problem (stages) which can then be handled more efficiently from the computational view point. Basic Elements of Dynamic Programming To apply Dynamic Programming, we have to pay special attention to the three basic elements of the DP Model. They are: 1. Definition of the stages. 2. Definition of the alternatives at each stage. 3. Definition of the states for each stage. Definition of the states varies depending on the situation being modeled. Nevertheless, as we investigate each application, we will find it helpful to consider the following questions: 1. What relationships bind the stages together? 2. What information is needed to make feasible decisions at the current stage without reexamining the decisions made at previous stages? Application of the Dynamic Programming in the Business World We will try to present three application models and finally a worked out implementation of Dynamic Programming showing the superiority of DP over the usual or straight forward method of solution. 1. Work Force Model: In some construction projects, hiring and firing are exercised to maintain a labour force that meets the needs of the project. Given that the activities of hiring and firing both incur additional costs. In such cases, through the implementation of DP Model, we can get the optimum result regarding how the labor force should be maintained throughout the life of the project. For example: A construction contractor estimates that the size of the work force needed over the next 5 weeks is to be 5, 7, 8, 4 and 6 workers respectively. Excess labor kept on the force will cost $300 per week and new hiring in any week will incur a fixed cost of $400 plus $200 per worker per week. The elements of this DP model are: 1. Stage i Such problem can optimally be solved through DP Model. Equipment Replacement Model: The longer a machine stays in service, the higher is its maintenance cost, and the lower its productivity. When a machine reaches a certain age, it may be more economical to replace it. The problem thus turns into determining the most economical age of a machine. Suppose that we are studying the machine replacement problem over a span of n years. At the start of each year, we decide whether to keep the machine in service an extra year or to replace it with a new one. For example: Shajib Farms wants to develop a replacement policy for its 2-year-old tractor over the next 5 years. A tractor must be kept in service for at least 3 years, but must be disposed of after 5 years. The current purchase price of a tractor is $40,000 and increases by 10% a year. The salvage value of a 1-year-old tractor is $30,000 and decreases by 10% a year. The current annual operating cost of the tractor is $1,300 but is expected to increase by 10% a year. Such problem can optimally be solved easily by applying DP Model. Investment Model We commonly assume that an investor wants to maximize â€Å"Total Return†. Suppose that Mr. Jamal wants to invest Tk. 4,000,000 (4 Million) now and 2,000,00 (2 Million) at the starts of years 2 to 4. The interest rate offered by NCC Bank is 8% compounded annually and the bonuses over the next 4 years are 1.8%, 1.7%, 2.1% and 2.5% respectively. The annual interest rate offered by Eastern Bank is 2% lower than that of NCC Bank, but its bonus is .5% higher. The objective is to maximize the accumulated capital at the end of 4 years. Such problem can also optimally be solved easily by applying DP Model. A company is selecting the advertising for its productand the frequency of advertising by each material are shown in the following table: |Frequency per week |Expected Sales (In Tk. 1,000) | | |Television |Radio |Newspaper | |0 |0 |0 |0 | |1 |25 |20 |33 | |2 |42 |38 |43 | |3 |55 |54 |47 | |4 |63 |65 |50 | We have to determine the optimum combination of advertising frequency and sales. Solution: States: Let X1= The frequency of advertisement at stage-1 (0~6) X2= The frequency of advertisement at stage-2 (0~6) X3= The frequency of advertisement at stage-3 (=6) S= Total Frequncy Stage-1 |Total Frequency (S) |Frequency at |Expected Sales | | |Stage-1(X1) | | |0 |0 |0 | |1 |1 |25 | |2 |2 |42 | |3 |3 |55 | |4 |4 |63 | Stage-2 | X2 |f 2(S, X2)=R2(X2)+ f 1*(S-X2) | | | | | |f2*(S) |X2* | |S | | | | | |0 |1 |2 |3 |4 | | | |0 |0+0=0 | | | | |0 |0 | |1 |0+25=25 |20+0=20 | | | |25 |0 | |2 |0+42=42 |20+25=45 |38+0=38 | | |45 |1 | |3 |0+55=55 |20+42=62 |38+25=63 |54+0=54 | |63 |2 | |4 |0+63=63 |20+55=75 |38+42=80 |54+25=79 |65+0=65 |80 |2 | Stage-3 | X2 |f 3(S, X3)=R3(X3)+ f 2*(S-X3) | | | | | |f3*(S) |X3* | |S | | | | | |0 |1 |2 |3 |4 | | | |4 |0+80=80 |33+63=96 |43+45=88 |47+25=72 |50+0=50 |96 |1 | Now we can derive the optimal values: X1=1 X2=2 X3=1 Expected Sales= 96,000 Usual or Straight forward method of solution √ Circle indicates alternative plans at each stage & √ Arrows represent the decision. The features of the above exhaustive enumeration scheme are: 1. All the decisions of any combination must specified before a combination can be evaluated. Here during solution, we have to make 64 alternative plans first. 2. An optimum policy cannot be determined until all combinations have been evaluated. This method is inefficient because some of the combination may not be feasible. 3. In other cases the number of combination may be too large to allow exhaustive listing. The Dynamic Programming approach avoids the above mentioned difficulties by first breaking up the problem into smaller sub-problems which are called stages in DP. A stage here signifies a portion of the problem for which a separate decision can be made.