NeurIPS - A few other MRF tools we built

Paper1: Learning the Dependency Structure of Latent Factors

• Y. He, Y. Qi, K. Kavukcuoglu, H. Park (2012) NeurIPS
• PDF

• Talk: Slide

• Abstract In this paper, we study latent factor models with the dependency structure in the latent space. We propose a general learning framework which induces sparsity on the undirected graphical model imposed on the vector of latent factors. A novel latent factor model SLFA is then proposed as a matrix factorization problem with a special regularization term that encourages collaborative reconstruction. The main benefit (novelty) of the model is that we can simultaneously learn the lower-dimensional representation for data and model the pairwise relationships between latent factors explicitly. An on-line learning algorithm is devised to make the model feasible for large-scale learning problems. Experimental results on two synthetic data and two real-world data sets demonstrate that pairwise relationships and latent factors learned by our model provide a more structured way of exploring high-dimensional data, and the learned representations achieve the state-of-the-art classification performance.

Citations

@INPROCEEDINGS{yhe12NIPS,
  title={Learning the Dependency Structure of Latent Factors},
  author={Y. He and Y. Qi and K. Kavukcuoglu and H. Park},
  booktitle={Proceedings of Advances in Neural Information Processing Systems (NIPS)},
  year={2012},
  note="{\\Acceptance rate = 25\% (370/1467)}"
}

Paper2: Sparse higher-order Markov random field

• PDF

• Abstract Systems and methods are provided for identifying combinatorial feature interactions, including capturing statistical dependencies between categorical variables, with the statistical dependencies being stored in a computer readable storage medium. A model is selected based on the statistical dependencies using a neighborhood estimation strategy, with the neighborhood estimation strategy including generating sets of arbitrarily high-order feature interactions using at least one rule forest and optimizing one or more likelihood functions. A damped mean-field approach is applied to the model to obtain parameters of a Markov random field (MRF); a sparse high-order semi-restricted MRF is produced by adding a hidden layer to the MRF; indirect long-range dependencies between feature groups are modeled using the sparse high-order semi-restricted MRF; and a combinatorial dependency structure between variables is output.

Citations

@misc{min2015sparse,
  title={Sparse higher-order Markov random field},
  author={Min, Renqiang and Qi, Yanjun},
  year={2015},
  month=nov # "~10",
  publisher={Google Patents},
  note={US Patent 9,183,503}
}

Support or Contact

Having trouble with our tools? Please contact Yanjun Qi and we’ll help you sort it out.