Home / About Us
News
People
Research
Publications
Teaching
Seminars
Links
Contacts
Sponsors
Admin Login

View Publication Details
Title: 
Evaluation of Qlearning for search and inspect missions using underwater vehicles 
Author: 
Frost, G. W. & Lane, D. M. 
Year: 
2014 
Book / Journal:  IEEE International Conference Oceans 2014 St Johns 
Abstract: 
An application for offline Reinforcement Learning in the underwater domain is proposed. We present and evaluate the integration of the Qlearning algorithm into an Autonomous Underwater Vehicle (AUV) for learning the actionvalue function in simulation. Three separate experiments are presented. The first compares two search policies: the ε — least visited, and random action, with respect to convergence time. The second experiment presents the effect of the learning discount factor, gamma, on the convergence time of the ε — least visited search policy. The final experiment is to validate the use of a policy learnt offline on a real AUV. This learning phase occurs offline within the continuous simulation environment which had been discretized into a gridworld learning problem. Presented results show the system's convergence to a global optimal solution whilst following both suboptimal policies during simulation. Future work is introduced, after discussion of our results, to enable the system to be used in a real world application. The results presented, therefore, form the basis for future comparative analysis of the necessary improvements such as function approximation of the state space. 
Keywords: 
, 
BibTex: 

