In Reinforcement Learning, one solves optimal control problems without knowledge of the underlying system's dynamics from the following perspective: An agent, who is aware of the current state of his environment, decides in favour of a particular action. The action is performed resulting in a change of the agent's environment. The agent notices the new state, receives a reward and decides again. This process repeats over and over and may be terminated by reaching a terminal state. In the course of time the agent learns from his experience by developing a strategy which maximizes his estimated total reward.The overall research in Reinforcement Learning concentrates on discrete sets of actions, but for real world problems it would be nice to have methods which are able to find good strategies using actions drawn from continuous sets, e.g. when you have to decide for a spatial direction in order to reach a distant point by going a minimal number of steps.We're using Python for searching and comparing strategies by evaluating combinations of different Reinforcement Learning algorithms, control tasks and requirements. In this talk, we give an overview of our implementation pointing out the contexts in which SciPy and other Python packages are applied.