Although discriminative learning in graphicalmodels generally improves classification re-sults, the generative semantics of the modelare compromised. In this paper, we intro-duce a novel approach of hybrid generative-discriminative learning for Bayesian net-works. We use an SVM-type large marginformulation for discriminative training, in-troducing a likelihood-weightedℓ1-norm forthe SVM-norm-penalization. This simultane-ously optimizes the data likelihood and there-fore partly maintains the generative charac-ter of the model. For many network struc-tures, our method can be formulated as a con-vex problem, guaranteeing a globally optimalsolution. In terms of classification, the result-ing models outperform state-of-the art gen-erative and discriminative learning methodsfor Bayesian networks, and are comparablewith linear and kernelized SVMs. Further-more, the models achieve likelihoods close tothe maximum likelihood solution and showrobust behavior in classification experimentswith missing features.

Top