This paper aims to accelerate processes
of actor-critic method, which is one of major
reinforcement learning algorithms, by a transfer
learning. In general, reinforcement learning is used
to solve optimization problems. Learning agents
acquire a policy to accomplish the target task autonomously.
To solve the problems, agents require
long learning processes for trial and error. Transfer
learning is one of effective methods to accelerate
learning processes of machine learning algorithms.
It accelerates learning processes by using
prior knowledge from a policy for a source task. We
propose an effective transfer learning algorithm for
actor-critic method. Two basic issues for the transfer
learning are method to select an effective source
policy and method to reuse without negative transfer.
In this paper, we mainly discuss the latter. We proposed
the reuse method which based on the selection
method that uses the forbidden rule set. Forbidden
rule set is the set of rules that cause immediate failure
of tasks. It is used to foresee similarity between
a source policy and the target policy. Agents should
not transfer the inappropriate rules in the selected
policy. In actor-critic, a policy is constructed by two
parameter sets: action preferences and state values.
To avoid inappropriate rules, agents reuse only reliable
action preferences and state values that imply
preferred actions. We perform simple experiments
to show the effectiveness of the proposed method. In
conclusion, the proposed method accelerates learning
processes for the target tasks.
雑誌名
Proceedings of the Second International Workshop on Regional Innovation Studies : (IWRIS2010)
号
2
ページ
55 - 58
発行年
2011-10-01
フォーマット
application/pdf
著者版フラグ
publisher
出版者
Graduate School of Regional Innovation Studies, Mie University
Accelerate Learning Processes by Avoiding Inappropriate Rules in Transfer Learning for Actor-Critic
60C15235.pdf
(106.12KB)
