Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Interesting, totally new concept for me: Where can I read more about EMA filter coefficient in Keras? My Google-fu is failing.


I think they're referring to the momentum parameter at [1]. The exponential moving average (EMA) of the batch mean/variance is used in the batch normalizing transform (Algorithm 1 in [2]).

The momentum ranges from 0 to 1. If it's close to 1, which the default of 0.99 is, the EMA of the batch mean/variance will change slowly across batches. If it's close to 0, the EMA will be close to the mean/variance of the current batch.

The EMA acts as a low-pass filter. With a momentum close to 1, the EMA changes slowly, filtering out high frequencies and leaving only frequencies close to DC. Note that this is opposite to what grandparent says: 0.99 has a lower frequency cutoff than 0.6 does. So I'm not really sure what they're getting at there.

[1] https://keras.io/layers/normalization/#batchnormalization

[2] https://arxiv.org/abs/1502.03167




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: