We explore neural higher-order input-dependent factors inlinear-chain conditional random fields (LC-CRFs) for sequencelabeling. It is a fusion of two powerful models as higher-orderLC-CRFs with linear factors are well-established for sequencelabeling tasks, but they lack to model non-linear dependencies.Therefore, we present neural higher-order input-dependent fac-tors which map sub-sequences of inputs to sub-sequences ofoutputs using distinct multilayer perceptron sub-networks. Thisis important in many tasks, in particular, for phoneme classifi-cation where the phone representation strongly depends on thecontext phonemes. Experimental results for phoneme classifi-cation with LC-CRFs and neural higher-order factors confirmthis fact and we achieve the best ever reported phoneme clas-sification performance on TIMIT, i.e. a phoneme error rate of15:8%. Furthermore, we show that the success is not obviousas linear high-order factors degrade phoneme classification per-formance on TIMIT.

Top