Hi all,

IN SHORT: mlp_bunch_size > 256 does not train properly

I have been using for years qnstrn/qnmultitrn with a bunch size of
256, which trains much faster than when updating on every sample. It
gives reasonable accuracy. Now I tried to set mlp_bunch_size to 1024
and I am no more able to train the MLP - the train/CV scores stay very
low till the end of the training.

I hoped it was due to a very small train corpus (10^4 frames for some
10^6 MLP parameters), but the same holds for the full-scale training
(details below). Does anyone have a guess what am I doing wrong?

Small experiment:
- MLP size 826x3500x39x210 (826 inputs, 210 phoneme-state outputs)
- 30665 train frames, 3354 CV frames
- 4-threads training with Atlas, @Xeon (64bit)
* mlp_bunch_size=256 -> 53/43% Acc (train/CV), 10 epochs
* mlp_bunch_size=512 -> 15/16% Acc (train/CV), 6 epochs
* mlp_bunch_size=1024-> 5/0.00% Acc (train/CV), 2 epochs

Normal size experiment:
- MLP size 351x3500x39x210 (351 inputs, 210 phoneme-state outputs)
- 20,382,407 train + 2,206,465 CV frames
* mlp_bunch_size=256 -> 52/50% Acc (train/CV), 7 epochs
* mlp_bunch_size=1024 -> 4.7/5.0% Acc (train/CV), 4 epochs

Petr.