Ensembl Variant Effect Predictor Download and install

Download

Download ensembl-vep package (see below the different ways to download it) and then follow the installation instructions.

Using Git

Clone the Git repository

Use git to download the ensembl-vep package:

git clone https://github.com/Ensembl/ensembl-vep.git
cd ensembl-vep

Update to a newer version

To update from a previous version:

cd ensembl-vep
git pull
git checkout release/115
perl INSTALL.pl

Use an older version

To use an older version (this example shows how to set up release 87):

cd ensembl-vep
git checkout release/87
perl INSTALL.pl

Download the Zipped package file

Users without the git utility installed may download a zip file from GitHub, though we would always recommend using git if possible.

curl -L -O https://github.com/Ensembl/ensembl-vep/archive/release/115.zip
unzip 115.zip
cd ensembl-vep-release-115/

Previous versions (ensembl-tools)

Previously, Ensembl VEP was available as part of the ensembl-tools package (see the Ensembl archive site for documentation). The following downloads are available for archival purposes. Show versions

What's new?

Previous version history - from version 88: Show

Older versions (ensembl-tools) - until version 87: Show

Requirements

• gcc, g++ and make
• Perl version 5.22 or above recommended (tested on 5.22, 5.26, 5.32, 5.34, 5.38)
• Perl packages:
  • Archive::Zip
  • DBD::mysql (version <=4.050)
  • DBI
  See this guide for more information on how to install perl modules.
  Additional libraries can be installed for extra features and enhancements but they are not required to run Ensembl VEP in most of the use cases.

Ensembl VEP's INSTALL.pl script will install required components of Ensembl API for you, but Ensembl VEP may also be used with any pre-existing API installations you have, provided their versions match the version of VEP you are using.

Ensembl VEP is available in the following platforms:

• Linux (e.g., Ubuntu, Debian, Mint)
• macOS
• Windows (requires a more involved installation process)

Ensembl VEP is also available as Docker and Singularity images, allowing to skip the complex installation steps.

Installation

Ensembl VEP's INSTALL.pl makes it easy to set up your environment for using the Ensembl VEP. It will download and configure a minimal set of the Ensembl API for use by the Ensembl VEP, and can also download cache files, FASTA files and plugins.

Run the following, and follow any prompts as they appear:

perl INSTALL.pl

Additional non-essential components and enhancements must be installed manually.

Software components installed

• BioPerl
• ensembl
• ensembl-io
• ensembl-variation
• ensembl-funcgen
• Bio::DB::HTS

If you already have the latest version of the API installed you do not need to run the installer, although it can be used to simply update your API version (with post-release patches applied), and retrieve cache and FASTA files. The installer downloads the API within the Ensembl VEP directory and will not affect any other Ensembl API installations.

The script will also attempt to install a Perl::XS module, Bio::DB::HTS, for rapid access to bgzipped FASTA files. If this fails, you may add the --NO_HTSLIB flag when running the installer; Ensembl VEP will fall back to using Bio::DB::Fasta for this functionality (more details).

Running the installer

The installer is run on the command line as follows:

 perl INSTALL.pl [options] 

Follow on-screen prompts and note warnings of any files which will be deleted/overwritten

You should not need to add any options, but configuration of the installer is possible with the flags below. Options can also be set by exporting environment variables prefixed with VEP_ before running the installer (for instance, export VEP_NO_HTSLIB=1 and export VEP_DIR_PLUGINS="/plugins").

--ASSEMBLY

-y

--AUTO

-a

perl INSTALL.pl --AUTO ac

--CACHE_VERSION [version]

--CACHEDIR [dir]

-c

./vep --dir_cache [dir]

--DESTDIR [dir]

-d

perl -I [dir] vep

--NO_HTSLIB

-l

--NO_TEST

--NO_UPDATE

-n

--PLUGINS

-g

# List the available plugins:
perl INSTALL.pl -a p --PLUGINS list
# Download/install all the available plugins:
perl INSTALL.pl -a p --PLUGINS all
# Download/install a defined list of plugins, e.g.:
perl INSTALL.pl -a p --PLUGINS dbNSFP,CADD,G2P

--PLUGINSDIR [dir]

-r

./vep --dir_plugins [dir]

--PREFER_BIN

-p

--SPECIES

-s

--USE_HTTPS_PROTO

--GITHUBTOKEN

--QUIET

-q

Additional components

INSTALL.pl will set up the minimum requirements Ensembl VEP. Some features and enhancements, however, require the installation of additional components. Most are perl modules that are easily installed using cpanm; see this guide for more information on how to install perl modules.

Typically, you will use cpanm to install modules locally in your home directories; this shows how to set up a path for perl modules and install one there:

mkdir -p $HOME/cpanm
export PERL5LIB=$PERL5LIB:$HOME/cpanm/lib/perl5
cpanm -l $HOME/cpanm Set::IntervalTree

To make the change to PERL5LIB permanent, it is recommended to add the export line to your $HOME/.bashrc or $HOME/.profile.

• Additional features
  • JSON - required to produce JSON format output
  • Set::IntervalTree - used to find overlaps between entities in coordinate space. Required to use --nearest
  • Bio::DB::BigFile - required to use bigWig format custom annotation files. See Bio::DB::BigFile instructions.
• Speed enhancements - these modules can improve Ensembl VEP runtime
  • PerlIO::gzip - marginal gains in compressed file parsing as used by the Ensembl VEP cache
  • ensembl-xs - provides pre-compiled replacements for frequently used routines in Ensembl VEP. Requires manual installation, see README for details

Bio::DB::BigFile

In order for Ensembl VEP to be able to access bigWig format custom annotation files, the Bio::DB::BigFile perl module is required. Installation involves downloading and compiling the kent source tree. The current version of the kent source tree does not work correctly with Bio::DB::BigFile, so it is necessary to install an archive version known to work (v335).

1. Download and unpack the kent source tree

   wget https://github.com/ucscGenomeBrowser/kent/archive/v335_base.tar.gz
   tar xzf v335_base.tar.gz

2. Set up some environment variables; these are required only temporarily for this installation process

   export KENT_SRC=$PWD/kent-335_base/src
   export MACHTYPE=$(uname -m)
   export CFLAGS="-fPIC"
   export MYSQLINC=`mysql_config --include | sed -e 's/^-I//g'`
   export MYSQLLIBS=`mysql_config --libs`

3. Modify kent build parameters

   cd $KENT_SRC/lib
   echo 'CFLAGS="-fPIC"' > ../inc/localEnvironment.mk

4. Build kent source

   make clean && make
   cd ../jkOwnLib
   make clean && make

   If either of these steps fail, you may have some missing dependencies. Known common missing dependencies are libpng and libssl; these may be installed, for example, with apt-get on Ubuntu. If you do not have sudo access you may have to ask your sysadmin to install any missing dependencies.

   sudo apt-get install libpng-dev libssl-dev

   On macOS you may use brew; the openssl libraries also need to be symbolically linked to a different path:

   brew install libpng openssl
   cd /usr/local/include
   ln -s ../opt/openssl/include/openssl .
   cd -

5. On some systems (e.g. macOS), a compiled file is placed in a path that Bio::DB::BigFile cannot find. You can correct this with:

   ln -s $KENT_SRC/lib/x86_64/* $KENT_SRC/lib/

6. We'll now use cpanm to install the perl module for Bio::DB::BigFile itself. See above for guidance on this. In this example we're going to install the module to a path within your home directory. In order to do this we must modify the paths that perl looks in to find modules by adding to the PERL5LIB environment module. To make this change permanent you must add the export line to your $HOME/.bashrc or $HOME/.profile.

   mkdir -p $HOME/cpanm
   export PERL5LIB=$PERL5LIB:$HOME/cpanm/lib/perl5
   cpanm -l $HOME/cpanm Bio::DB::BigFile

   If you are prompted for the path to the kent source tree, that means something didn't go right in the compilation above. Double check that $KENT_SRC/lib/jkweb.a exists and is not found instead at e.g. $KENT_SRC/lib/x86_64/jkweb.a. You may copy or link the file (and the other files in that directory) to the former path.

   ln -s $KENT_SRC/lib/x86_64/* $KENT_SRC/lib/

7. You should now be able to successfully run the appropriate test in the Ensembl VEP package:

   perl -Imodules t/AnnotationSource_File_BigWig.t

Using Ensembl VEP in macOS

Installing Ensembl VEP on macOS is slightly trickier than other Linux-based systems, and will require additional dependancies.
These instructions will guide you through the setup of Perlbrew, Homebrew, MySQL and other dependancies that will allow for a clean installation of Ensembl VEP on your macOS system.

These instructions have been tested on macOS High Sierra (10.13) and macOS Sierra (10.12).
Older versions may require additional tweaks, however we shall endeavouXcoder to keep these instructions up to date for future versions of MacOS.

Prerequisite Setup

List of prerequisites: Xcode, GCC, Perlbrew, Cpanm, Homebrew, mysql, DBI, DBD::mysql (version <=4.050)

Xcode and GCC

Ensembl VEP requires Xcode and GCC for installation purposes. Fortunately, recent versions of macOS will look for (and attempt to install if required) both of these when you run the following command:

gcc -v

Perlbrew

We recommend using Perlbrew to install a new version of Perl on your mac, to prevent messing with the vendor perl too much. This can be done with the following command:

curl -L http://install.perlbrew.pl | bash

echo 'source $HOME/perl5/perlbrew/etc/bashrc' >> ~/.bash_profile

At this point, PLEASE RESTART YOUR TERMINAL WINDOW to allow for the perlbrew changes to take effect.

We recommend installing Perl version 5.26.2 to run Ensembl VEP, and installing cpanm to handle the installation of perl modules.
These steps can be completed with the commands:

perlbrew install -j 5 --as 5.26.2 --thread --64all -Duseshrplib perl-5.26.2 --notest
perlbrew switch 5.26.2
perlbrew install-cpanm

Homebrew

This package management system for macOS would make the installation of the next prerequisite (i.e. xs) easier.

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

xz

Ensembl VEP requires the installation of xz, a data-compression utility. The easiest way to install the xz package is through homebrew:

brew install xz

MySQL

In order to connect to the Ensembl databases, a collection of MySQL related dependancies are required. Fortunately, these can be installed neatly with Homebrew and Cpanm:

brew install mysql
cpanm DBI
cpanm DBD::mysql@4.050

Installing BioPerl

On some versions of macOS, the Ensembl VEP installer fails to cleanly install BioPerl, so a manual install will prevent issues:

curl -O https://cpan.metacpan.org/authors/id/C/CJ/CJFIELDS/BioPerl-1.6.924.tar.gz
tar zxvf BioPerl-1.6.924.tar.gz
echo 'export PERL5LIB=${PERL5LIB}:##PATH_TO##/bioperl-1.6.924' >> ~/.bash_profile

where ##PATH_TO##/bioperl-1.6.924 refers to the location of the newly unzipped BioPerl directory.

Final Dependancies

Installing the following Perl modules with cpanm will allow for full Ensembl VEP functionality:

cpanm Test::Differences Test::Exception Test::Perl::Critic Archive::Zip PadWalker Error Devel::Cycle Role::Tiny::With Module::Build LWP List::MoreUtils

export DYLD_LIBRARY_PATH=/usr/local/mysql/lib/:$DYLD_LIBRARY_PATH

Installing Ensembl VEP

And that should be that! You should now be able to install Ensembl VEP using the installer:

git clone https://github.com/ensembl/ensembl-vep
cd ensembl-vep
perl INSTALL.pl --NO_TEST

Using Ensembl VEP in Windows

Ensembl VEP was developed as a command-line tool, and as a Perl script its natural environment is a Linux system. However, there are several ways you can use Ensembl VEP on a Windows machine.

You may also consider using Ensembl VEP's web or REST interfaces.

Virtual machines

Using a virtual machine you can run a virtual Linux system in a window on your machine. There are two ways to do this:

1. Use the Ensembl virtual machine image
2. Use Docker

Perl

If Perl is installed on Windows, Ensembl VEP can be setup. However this may require installation of dependent modules. We recommend using Docker to run Ensembl VEP on Windows.

1. Check Perl is installed
2. Download and unpack the zip of the ensembl-vep package
3. Open a Command Prompt (search for Command Prompt in the Start Menu)

4. Navigate to the directory where you unpacked the Ensembl VEP package, e.g.

   cd Downloads/ensembl-vep-release-115

5. Run INSTALL.pl with --NO_HTSLIB and --NO_TEST; you will see some warnings about the "which" command not being available (these will also appear when running Ensembl VEP and can be ignored).

   perl INSTALL.pl --NO_HTSLIB --NO_TEST

Docker

Docker allows running applications in virtualised containers. The Ensembl VEP Docker image is available from DockerHub: Ensembl VEP in DockerHub

After installing Docker, download the Ensembl VEP Docker image:

docker pull ensemblorg/ensembl-vep

To download cache files and other data with Ensembl VEP Docker, we recommend mounting a directory from your local (host) machine to folder /data from the Docker image. For instance:

mkdir $HOME/vep_data
docker run -t -i -v $HOME/vep_data:/data ensemblorg/ensembl-vep

In the example above, data in $HOME/vep_data will be accessible by both the local machine and Ensembl VEP Docker. The Ensembl VEP API, plugins maintained by Ensembl VEP and their dependencies (e.g. Perl APIs, Bio::DB::HTS, htslib, ...) are already installed in the image.

sudo docker run -t -i -v $HOME/vep_data:/data:Z ensemblorg/ensembl-vep

chmod -R a+rwx $HOME/vep_data

Cache and FASTA files installation

You can run the INSTALL.pl script to install the cache and FASTA files:

docker run -t -i -v $HOME/vep_data:/data ensemblorg/ensembl-vep INSTALL.pl

• You will be asked to install cache data. Type the comma-separated numbers for the species/assembly of interest and press enter. Your data will download and unpack; this may take a while.
• If you wish to retrieve HGVS annotations, please download the FASTA files for your species. To do this, at the next prompt type 0 and press enter.

The above process may also be performed in one command; for example, to set up the cache and corresponding FASTA for human GRCh38:

docker run -t -i -v $HOME/vep_data:/data ensemblorg/ensembl-vep INSTALL.pl -a cf -s homo_sapiens -y GRCh38

The installer downloads Ensembl VEP data to the mounted directory (e.g., $HOME/vep_data). The downloaded data will be automatically detected as long as its folder is mounted when running VEP:

docker run -v $HOME/vep_data:/data ensemblorg/ensembl-vep vep -i examples/homo_sapiens_GRCh38.vcf --cache

Running Ensembl VEP with data from local folder

Here is an example on running Ensembl VEP with data from folder $HOME/vep_data in the local machine (provided that the cache has been downloaded to that folder):

docker run -v $HOME/vep_data:/data ensemblorg/ensembl-vep \
  vep --cache --offline --format vcf --vcf --force_overwrite \
      --input_file input/my_input.vcf \
      --output_file output/my_output.vcf \
      --custom file=custom/my_extra_data.bed,short_name=BED_DATA,format=bed,type=exact,coords=1 \
      --plugin NMD

Please avoid using absolute paths to data as the paths inside the container differ from your local machine.

Update from a previous version

1. Update your Docker container

   docker pull ensemblorg/ensembl-vep

2. Update your cache

   # Install the new cache through the Ensembl VEP INSTALL.pl script (see "Cache installation" section above)
   docker run -t -i -v $HOME/vep_data:/data ensemblorg/ensembl-vep INSTALL.pl -a c
   
   # Or install the cache manually
   cd $HOME/vep_data
   curl -O http://ftp.ensemblgenomes.org/pub/protists/release-115/variation/vep/homo_sapiens_vep_115_GRCh38.tar.gz
   tar xzf homo_sapiens_vep_115_GRCh38.tar.gz