Abstraction plays an important role in the gener- alisation of knowledge and skills and is key to sample efficient learning. In this work, we study joint temporal and state abstraction in reinforce- ment learning, where temporally-extended actions in the form of options induce temporal abstractions, while aggregation of similar states with respect to abstract options induces state abstractions. Many existing abstraction schemes ignore the interplay of state and temporal abstraction. Consequently, the considered option policies often cannot be directly transferred to new environments due to changes in the state space and transition dynamics. To address this issue, we propose a novel abstraction scheme building on successor features. This includes an al- gorithm for transferring abstract options across dif- ferent environments and a state abstraction mech- anism that allows us to perform efficient planning with the transferred options.

Top