HEXQ

HEXQ is a reinforcement learning algorithm created by Bernhard Hengst, which attempts to solve a Markov Decision Process by decomposing it hierarchically. [http://citeseer.ist.psu.edu/hengst02discovering.html]

cite paper
author = Bernhard Hengst
title = Discovering Hierarchy in Reinforcement Learning with HEXQ
date = 2002
url =http://citeseer.ist.psu.edu/hengst02discovering.html