Dynamic Programming and Reinforcement Learning

Question 1. We have a tree farm. At any time, the size s of a tree is within six categories: 0, 1, …, 5, where 0 means that the tree has died, category 4 is about the average size of a mature tree and 5 indicated above average size of a mature tree. We need to decide when to harvest a given tree. Each year it costs about $ 10+s to maintain a tree, and $ 30+5s to harvest a tree. The sales price of a tree of each size is as follows:

The transition probability matrix for the size of the tree is as follows:

Don't use plagiarized sources. Get Your Custom Essay on
Dynamic Programming and Reinforcement Learning
Just from $13/Page
Order Essay


sizes 0 1 2 3 4 5
0 1 0 0 0 0 0
1 0.05 0.15 0.7 0.1 0 0
2 0.05 0 0.2 0.6 0.1 0.05
3 0.05 0 0 0.5 0.4 0.05
4 0.1 0 0 0 0.85 0.05
5 0.1 0 0 0 0 0.9

(Q1.a) Describe a dynamic programming problem to determine an optimal harvesting policy. (Q1.b) Solve the problem numerically. What numerical methods are applicable to this problem and why.

Question 2. Each quarter the marketing manager of a retail store divides the customers into two groups based on their purchase behavior in the previous quarter. The classes are denoted by L and H . The manager wishes to determine to which group of customers he should sent a catalog. The cost of sending a catalog is $ 15 per customer. If a customer from group L receives a catalog, then the expected purchase in the current quarter is $ 20, otherwise it is $ 10. If a customer from group H receives a catalog, then the expected purchase in the current quarter


is $ 50, otherwise it is $ 25. Furthermore, if a customer from group L receives a catalog, then the probability that he will stay in group L for the next quarter is 0.3, otherwise, it is 0.5. If a customer from group H receives a catalog, then the probability that s/he will stay in group H for the next quarter is 0.8, otherwise, it is 0.4.

(Q2.a) Formulate an average reward problem to help the manager. (Q2.b) Determine an optimal policy using policy iteration method. (Q2.c) Solve the problem using linear programming.

(Q2.d) For what discount factor is the discounted infinite-horizon problem equivalent to the average reward problem in this context?





























The Homework Labs
Calculate your paper price
Pages (550 words)
Approximate price: -

Our Advantages

Plagiarism Free Papers

We ensure that all our papers are written from scratch. We deliver original plagiarism-free work. To guarantee this, we submit all work alongside a plagiarism report.

Free Revisions

All our papers are completed and submitted before the deadline. We ensure this to provide you with enough time to go through the work and point out any sections or topics that may need revision or polishing. We provide unlimited revision services for free.


All papers have a title page providing your personal and institutional information. We do not charge you for this title page.


All papers have a bibliography or references page. This page is a requirement for academic and professional documents. We provide this page at no cost for all our papers.

Originality & Security

At Thehomeworklabs, we guarantee the confidentiality and security of your information. We value our clients and take confidentiality seriously. All personal information is treated with confidentiality and stored safely to ensure that no third parties gain access to it. We also provide original work and attach an originality/plagiarism report alongside all papers.

24/7 Customer Support

Our customer support team is available 24/7 to provide you with any necessary assistance when you need it. You can contact us at any time, day or night, via email or through the live chat button.

Try it now!

Calculate the price of your order

Total price:

How it works?

Follow these simple steps to get your paper done

Place your order

Fill in the order form and provide all details of your assignment.

Proceed with the payment

Choose the payment system that suits you most.

Receive the final file

Once your paper is ready, we will email it to you.

Our Services

We provide our customers with the best experience in the academic and business writing field.


Flexible Pricing

We provide the best quality of service at affordable prices. We also allow our clients to make partial payments for their orders. You can also contact our customer support team in case you need to discuss a different payment plan.


Admission help & Client-Writer Contact

We realize that sometimes clarification is necessary to ensure that quality work is done. Therefore, we provide a button for clients and writers to communicate in case some clarification is needed.


Paper Submission

We ensure that we submit all papers ahead of their respective deadlines. This allows you to go through the documents and request any revision, corrections, or polishing before the paper is due.


Customer Feedback

We encourage customer feedback, positive or negative. We can identify the various areas that we need to improve to provide even better services through your feedback. Please feel free to give us feedback.