eHive production system
A big fraction of Ensembl's computation pipelines are run using eHive (Ensembl Hive) distributed scheduling system.
The main goal behind creating it was to enable efficient parallel computation on Unix clusters using a simple Perl API.
Recently eHive code was detached from Ensembl codebase and can now be used as a separate product
by those who need to run computational pipelines.
Code and documentation
The eHive system is hosted on GitHub and has a copmrehensive user manual.
guiHive is a Web app that helps both to visualize the progress of an eHive pipeline and to tune some parameters of the already running computations. It is also hosted on GitHub and has its own documentation (in README.md).