DAPPER is a set of templates for benchmarking the performance of data assimilation (DA) methods. The tests provide experimental support and guidance for new developments in DA. The typical set-up is a synthetic (twin) experiment, where you specify a dynamic model and an observational model, and use these to generate a synthetic truth (multivariate time series), and then estimate that truth given the models and noisy observations.
read, run and try to understand
Some of the examples can also be opened in Jupyter, and thereby run in the cloud
(i.e. without installation, but requiring Google login): .
provides an introduction.
includes general guidelines and the API,
but for any serious use you will want to read and adapt the code yourself.
If you use it in a publication, please cite, e.g.,
The experiments used (inspiration from) DAPPER [ref], version 1.2.1,
where [ref] points to .
Lastly, for an introduction to DA theory also using Python,
see these tutorials.
DAPPER enables the numerical investigation of DA methods
through a variety of typical test cases and statistics. It
(a) reproduces numerical benchmarks results reported in the literature, and
(b) facilitates comparative studies, thus promoting the
(a) reliability and
(b) relevance of the results.
For example, this figure is generated by
examples/basic_3.py and is a
reproduction of figure 5.7 of these lecture notes.
DAPPER is (c) open source, written in Python, and (d) focuses on readability; this promotes the (c) reproduction and (d) dissemination of the underlying science, and makes it easy to adapt and extend. It also comes with a battery of diagnostics and statistics, and live plotting (on-line with the assimilation) facilities, including pause/inspect options, as illustrated below
In summary, it is well suited for teaching and fundamental DA research. Also see its drawbacks.
Works on Linux/Windows/Mac.
conda create --yes --name dapper-env python=3.8 conda activate dapper-env python -c 'import sys; print("Version:", sys.version.split())'
Ensure the output at the end gives a version bigger than 3.7.
Keep using the same terminal for the commands below.
Do you want the DAPPER code available to play around with? Then
git clone) DAPPER.
cdinto it (ensure you're in the folder with a
pip install -e .[dev]
[dev]if you don't need to do serious development.
Do you just want to run a script that requires DAPPER? Then
requirements.txtfile, then do
pip install -r path/to/requirements.txt.
pip install dapper==1.0.0to get version
1.2.3(as an example).
You should now be able to do run your script with
For example, if you are in the DAPPER dir,
PS: If you closed the terminal (or shut down your computer),
you'll first need to run
conda activate dapper-env
|EnKF 1||Sakov08, Hoteit15, Grudzien2020|
|iEnKS / iEnKF / EnRML / ES-MDA 2||Sakov12, Bocquet12, Bocquet14|
|LETKF, local & serial EAKF||Bocquet11|
|Sqrt. model noise methods||Raanes2014|
|Particle filter (bootstrap) 3||Bocquet10|
|Optimal/implicit Particle filter 3||Bocquet10|
|Rank histogram filter (RHF)||Anderson10|
1: Stochastic, DEnKF (i.e. half-update), ETKF (i.e. sym. sqrt.).
Serial forms are also available.
Tuned with inflation and "random, orthogonal rotations".
2: Also supports the bundle version, and "EnKF-N"-type inflation.
3: Resampling: multinomial (including systematic/universal and residual).
The particle filter is tuned with "effective-N monitoring", "regularization/jittering" strength, and more.
For a list of ready-made experiments with suitable,
tuned settings for a given method (e.g. the
grep -r "xp.*iEnKS" dapper/mods
|Linear Advect. (LA)||Yes||Yes||Yes||1d||1000 *||51||Evensen/Raanes|
|LorenzUV||No||Yes||No||2x 1d||256 + 8 *||≈60||Raanes|
|Vissio-Lucarini 20||No||Yes||No||1d||36 *||10||Yumeng|
*: Flexible; set as necessary
**: Tangent Linear Model included?
The models are found as subdirectories within
A model should be defined in a file named
and illustrated by a file named
Most other files within a model subdirectory
are usually named
authorYEAR.py and define a
which holds the settings of a specific twin experiment,
using that model,
as detailed in the corresponding author/year's paper.
A list of these files can be obtained using
find dapper/mods -iname '[a-z]*[0-9]*.py'
Some files contain settings used by several papers.
Moreover, at the bottom of each such file should be (in comments)
a list of suitable, tuned settings for various DA methods,
along with their expected, average
rmse.a score for that experiment.
As mentioned above, DAPPER reproduces literature results.
You will also find results that were not reproduced by DAPPER.
DAPPER is aimed at research and teaching (see discussion up top). Example of limitations:
The scope of DAPPER is restricted because
Moreover, even straying beyond basic configurability appears unrewarding when already building on a high-level language such as Python. Indeed, you may freely fork and modify the code of DAPPER, which should be seen as a set of templates, and not a framework.
Also, DAPPER comes with no guarantees/support. Therefore, if you have an operational or real-world application, such as WRF, you should look into one of the alternatives, sorted by approximate project size.
|JEDI||JCSDA (NOAA, NASA, ++)||General|
|ERT||Statoil||History matching (Petroleum DA)|
|PIPT||CIPR||History matching (Petroleum DA)|
|PyOSSE||Edinburgh, Reading||Earth-observation DA|
Below is a list of projects with a purpose more similar to DAPPER's (research in DA, and not so much using DA):
|DAPPER||Raanes, Chen, Grudzien||Python|
|hIPPYlib||Villa, Petra, Ghattas||Python, adjoint-based PDE methods|
|FilterPy||R. Labbe||Python. Engineering oriented.|
|DASoftware||Yue Li, Stanford||Matlab. Large inverse probs.|
|Pomp||U of Michigan||R|
|EnKF-C||Sakov||C. Light-weight, off-line DA|
IEnKS codes have been inspirational
in the development of DAPPER.
Patrick N. Raanes, Yumeng Chen, Colin Grudzien, Maxime Tondeur, Remy Dubois
DAPPER is developed and maintained at NORCE (Norwegian Research Institute) and the Nansen Environmental and Remote Sensing Center (NERSC), in collaboration with the University of Reading, the UK National Centre for Earth Observation (NCEO), and the University of Nevada, Reno.