Student: Stanley

Markov Decision Processes

IE 515 Course Semester Project 2 Guideline The goal of this project is to give you an opportunity to explore real-world applications of stochastic processes. Each student or a team may select one of the following two types of projects. Option A: Application Project (Teams of three students are allowed. Fine to do by yourself or with a partner. More depth will be expected of a three-student project) You need to describe the problem considered for your project and propose a model and/or solution approach(es) for solving the problem using Markov chain modeling and/or Markov decision processes . Ideally, the project could be based on an industrial application. If you do not have an industrial project, you may want to select a problem from a published journal. Industrial application project: the emphasis should be given on the problem statement and a Markov decision model developed, as those might be relatively difficult to accomplish. Also, make an attempt to find the optimal policy, and use visualization tools to demonstrate the results. Selected problem from a published journal: you are expected to present the formulation of the problem as a Markov decision process and a solution procedure for finding an optimal stationary policy. Also, make an attempt to consider alternatives while proposing design solutions, show the benefits and pitfalls of each alternative, and use visualization tools to demonstrate the results. Option B: Research Paper (This option is the alternative, but it is not recommended unless you are unable to do Option A. Teaming is not allowed and the maximum possible credit you can get with this option is 9 of 10) You may choose a specific topic in the area of stochastic process and explore it in greater detail. This type of project should be based on at least five journal articles to survey the existing literature, identify a problem (or a gap) to be researched, present existing methods for solving the problem, and in some cases develop an improved solution approach (e.g., an algorithm). (Of course, improvements to the existing algorithms will be an asset!) A review paper is a good example of this type of project. A list of journals you may want to consider for your project includes Interfaces, Expert Systems with Applications, Computers & Industrial Engineering, International Journal of Industrial Engineering, IIE Transactions, Journal of Manufacturing Systems, International Journal of Production Research, Journal of Intelligent Manufacturing, Computers & Operations Research, European Journal of Operational Research, etc. However, conference proceedings, magazines, and technical reports are not recommended as source materials. Note that NMSU library database would be a good place to search these resources for your reference. You may be required to register to request journal articles through NMSU Interlibrary loan at https://illiad.nmsu.edu/logon.html. Requesting a journal article through NMSU library For example, Interfaces are freely accessible if you follow the steps below: (1) Go to http://lib.nmsu.edu/index.html IE 515 Spring 2020 page 1 of 6 (2) Choose Search & Find (3) Type "interfaces" in the search box and press Search button (4) Optionally, sign in using your NMSU username and password (5) Choose “Interfaces (Providence, R.I.)" (6) Choose “Interfaces (Providence, R.I.: Online)" (7) Choose "EBSCOhost Applied Science & technology Source" or other options, IE 515 Spring 2020 page 2 of 6 (8) Click on Search within this publication. (9) Search for articles. For example, type “Markov decision process”, choose “TI Title” from pull-down menu, and then press Search button (see below) Note: According to my past experience, some students couldn’t get the pdf file of the article since they didn’t choose the publication year and month correctly. For example, if you selected “Gale Cengage Academic OneFile” in Step #7, choose the article which is published between 1999 and 5 years ago. However, if “EBSCOhost Applied Science & Technology Source” is selected in Step #7, you have to choose the article published from 1970 to 5 years ago. IE 515 Spring 2020 page 3 of 6 SEMESTER PROJECT REPORT FORMAT The project report should be prepared on a word processor and should contain figures and tables that are necessary to make the report easy to read. Be concise in your writing and consult technical writing references as needed. Your report will be formatted with the following traits: The title page should include the title of the project, student name, course title, and the date. The title page should be followed by an abstract (about 100 words). Include also a table of contents, a list of figures, and a list of tables. All references must be included at the end of the paper, followed by appendices, if any. There is no page limit but the project report should be at least 10 pages long (excluding title page, table of contents, appendix, references), and single-spaced throughout. Use a standard font (Times New Roman 12). Use 1 inch margins for top, bottom, left, and right. Use proper punctuation, spelling, and grammar. All pages (with the exception of the title page) should be numbered. The main body of the semester project report should include the sections outlined as follows: Option A: Application Project 1. Introduction (including project goals) 2. Statement of the problem 3. Markov chain and/or Markov decision model formulation 4. Results and discussion 5. Conclusions and future research 6. References 7. Appendix Note that the results has to be essential for discussion, and thus, provides an evidence to reinforce your conclusion later. In the discussion section, you need to respond to what the results mean. You have to be careful since, if the discussion is not sufficient, you cannot objectively demonstrate the significance of your research work. A large number of the submitted articles are rejected because of weak discussion. Thus, I would say that this section is one of the most important section. While writing the discussion section, think about how the results are related to the original question or goals (and/or objectives) outlined in the Introduction. Are your results consistent with what other investigators have reported? Discuss any weaknesses and discrepancies. You may want to think about if there is another way to interpret your results, too. Also, think about what further research would be necessary to answer the questions raised by your results. After that, you will be able to write a clear conclusion. Without the clear conclusion, readers will find it difficult to judge your research work. In the conclusion, instead of just repeating or listing the results, you have to provide a clear scientific justification for your research, and suggest future research works, as well. You might want to present global and specific conclusions, which are related to the objectives included in the Introduction Option B: Research Paper 1. Introduction (including project goals and some research issues) IE 515 Spring 2020 page 4 of 6 2. Literature review 3. Statement of the problem 4. Existing models and solution approaches (Option) 5. Proposed model and/or solution approach (Option) 6. Examples (Option) 7. Conclusions and future research 8. References 9. Appendix While writing the report, you have to think about the following questions: What is the goal of the project and why is it important? What research has been done? How was it presented in the written report? (Describe the mathematical model, in terms that your classmate could understand) Did the results support the main idea of the project? Was the argument convincing, or did more work need to be done? Think about what some aspects of the real-world behavior that the model describes are and in what ways the model is unrealistic or realistic, what assumptions the authors made, and so on. Who is the intended audience and how could the results of your project be used for them? PROJECT REQUIREMENTS Project written report submission (10 points) Each team should submit the following items (one per project): A MS word file with project report and software developed (when applicable) (5 points) A set of pdf files with references used (5 points) Compress all files with Winzip if possible, and submit it through the class Canvas system. Do not email me the electronic files of your project report. PROJECT DUE DATES Project written report is due on Sunday, May 10 (5pm US Mountain time) IMPORTANT NOTES The project 2 score will contribute 50% to your final grade. Your project report has to be submitted on time. Late submission will be penalized with 10% of the total points available for each day late. The following item will be considered while scoring written report: o Organization (information is presented in a logical, interesting way, which is easy to follow. Also, points are logically ordered) o Quality of information (supporting details specific to subject, clarity of writing) o Literacy (virtually no grammatical or syntactical errors). This is a threshold category, and you can be marked down significantly if your report does not meet a minimum standard. o Length of report For each category, students will obtain 0 point (unacceptable), 1.5 points (acceptable), 2 points (satisfactory), or 2.5 points (Exemplary). IE 515 Spring 2020 page 5 of 6 Also, keep in mind the following: For the application project (i.e., Option A), more depth will be expected for a three-student project. For the research paper (i.e., Option B), teaming is not allowed and the maximum possible credit you can get with this option is 9 of 10. IE 515 Spring 2020 page 6 of 6

Budget: $10.00

Due on: April 30, 2020 00:00

Posted: 6 months ago.

Answers (0)