Design of Experiments for Reinforcement Learning [Paperback]

This thesis takes an empirical approach to understanding of the behavior and interactions between the two main components of reinforcement learning: the learning algorithm and the functional representation of learned knowledge.?The author approaches these entities using design of experiments not commonly employed to study machine learning methods.?The results outlined in this work provide insight as to what enables and what has an effect on successful reinforcement learning implementations so that this learning method can be applied to more challenging problems.
GLOSSARYACKNOWLEDGMENTFOREWARD1. INTRODUCTION2. REINFORCEMENT LEARNING2.1 Applications of reinforcement learning2.1.1 Benchmark problems2.1.2 Games2.1.3 Real-world applications2.1.4 Generalized domains2.2 Components of reinforcement learning2.2.1 Domains2.2.2 Representations2.2.3 Learning algorithms2.3 Heuristics and performance effectors3. DESIGN OF EXPERIMENTS3.1 Classical design of experiments3.2 Contemporary design of experiments3.3 Design of experiments for empirical algorithm analysis4. METHODOLOGY4.1 Sequential CART4.1.1 CART modeling4.1.2 Sequential CART modeling4.1.3 Analysis of sequential CART4.1.4 Empirical convergence criteria4.1.5 Example: 2-D 6-hump camelback function4.2 Kriging metamodeling4.2.1 Kriging4.2.2 Deterministic kriging4.2.3 Stochastic kriging4.2.4 Covariance function4.2.5 Implementation4.2.6 Analysis of kriging metamodels5. THE MOUNTAIN CAR PROBLEM5.1 Reinforcement learning implementation5.2 Sequential CART5.3 Response surface metamodeling5.4 Discussion6. THE TRUCK BACKER-UPPER PROBLEM6.1 Reinforcement learning implementation6.2 Sequential CART6.3 Response surface metamodeling6.4 Discussion7. THE TANDEM TRUCK BACKER-UPPER PROBLEM7.1 Reinforcement learning implementation7.2 Sequential CART7.3 Discussion8. DISCUSSION8.1 Reinforcement learning8.2 Experimentation8.3 Innovations8.4 Future workAPPENDICESA. Parameter effects in the game of Chung ToiB. Design of experiments for thels