Implicit equal‐weights particle filter

• Published in: Quarterly journal of the Royal Meteorological Society Vol. 142; no. 698; pp. 1904 - 1919
• Main Authors: Zhu, Mengbin, van Leeuwen, Peter Jan, Amezcua, Javier
• Format: Journal Article
• Language: English
• Published: Chichester, UK John Wiley & Sons, Ltd 01-07-2016; Wiley Subscription Services, Inc
• Subjects: Filters; implicit sampling; particle filter; non‐degeneracy; targeted weights
• ISSN: 0035-9009; 1477-870X
• DOI: 10.1002/qj.2784

Filter degeneracy is the main obstacle for the implementation of particle filters in nonlinear high‐dimensional models. A new scheme, the implicit equal‐weights particle filter (IEWPF), is introduced, in which samples are drawn implicitly from proposal densities with a different covariance for each particle, such that all particle weights are equal by construction.
We test and explore the properties of the new scheme using a 1000 dimensional simple linear model and the 1000 dimensional nonlinear Lorenz96 model and compare the performance of the scheme with that of a local ensemble transformed Kalman filter (LETKF). The new scheme is never degenerate and shows good and consistent performance in all experiments. The LETKF has lower root‐mean‐square errors at observed grid points, but its ensemble spread is too low at unobserved grid points, where the IEWPF performs better. Furthermore, the IEWPF has a consistent spread in all experiments.
This new filter opens up a new class of particle filters that, by construction, do not suffer from the curse of dimensionality.
The physics related to the formation of clouds like the one shown in the image is highly complex and nonlinear. Standard data‐assimilation techniques rely on linearisations and yield low‐quality results when applied to phenomena like this. Fully nonlinear data‐assimilation methods do exist but are typically too computationally expensive for high‐dimensional applications. The implicit equal‐weights particle filter developed here is fully nonlinear and very efficient in systems of any dimension, without artificial tricks like localisation. As such it forms an important step in solving this problem.