Time filter

Source Type

Ridgway J.,University of Paris Dauphine | Alquier P.,CREST ENSAE | Chopin N.,CREST ENSAE | Liang F.,University of Illinois at Urbana - Champaign
Advances in Neural Information Processing Systems | Year: 2014

We develop a scoring and classification procedure based on the PAC-Bayesian approach and the AUC (Area Under Curve) criterion. We focus initially on the class of linear score functions. We derive PAC-Bayesian non-asymptotic bounds for two types of prior for the score parameters: a Gaussian prior, and a spike-and-slab prior; the latter makes it possible to perform feature selection. One important advantage of our approach is that it is amenable to powerful Bayesian computational tools. We derive in particular a Sequential Monte Carlo algorithm, as an efficient method which may be used as a gold standard, and an Expectation-Propagation algorithm, as a much faster but approximate method. We also extend our method to a class of non-linear score functions, essentially leading to a nonparametric procedure, by considering a Gaussian process prior.

Harrison R.,Institute for Fiscal Studies | Harrison R.,University College London | Jaumandreu J.,Boston University | Mairesse J.,CREST ENSAE | And 3 more authors.
International Journal of Industrial Organization | Year: 2014

We study the impact of process and product innovations introduced by firms on employment growth with random samples of manufacturing and services from France, Germany, Spain and the UK for 1998-2000, totaling about 20,000 companies. We develop and estimate a model relating firms' and industry's employment to innovation, that leads us to the conclusions that follow. Trend increases in productivity reinforced by process innovation are an important source of reduction of employment requirements for a given output, but the growth of demand for the old products tends to overcompensate these displacement effects. The switch of production towards new products does not reduce employment requirements, and the growth of the demand for the new products is the strongest force behind employment creation. Reallocation due to business stealing is estimated at a maximum of one third of the net employment created by product innovators. The growth of employment originated from the market expansion induced by the new products can be as important as another third. © 2014 Elsevier B.V.

Pezzoni M.,University of Nice Sophia Antipolis | Pezzoni M.,Bocconi University | Mairesse J.,CREST ENSAE | Mairesse J.,Maastricht University | And 5 more authors.
PLoS ONE | Year: 2016

We examine gender differences among the six PhD student cohorts 2004-2009 at the California Institute of Technology using a new dataset that includes information on trainees and their advisors and enables us to construct detailed measures of teams at the advisor level. We focus on the relationship between graduate student publications and: (1) their gender; (2) the gender of the advisor, (3) the gender pairing between the advisor and the student and (4) the gender composition of the team. We find that female graduate students coauthor on average 8.5% fewer papers than men; that students writing with female advisors publish 7.7% more. Of particular note is that gender pairing matters: male students working with female advisors publish 10.0% more than male students working with male advisors; women students working with male advisors publish 8.5% less. There is no difference between the publishing patterns of male students working with male advisors and female students working with female advisors. The results persist and are magnified when we focus on the quality of the published articles, as measured by average Impact Factor, instead of number of articles. We find no evidence that the number of publications relates to the gender composition of the team. Although the gender effects are reasonably modest, past research on processes of positive feedback and cumulative advantage suggest that the difference will grow, not shrink, over the careers of these recent cohorts. © 2016 Pezzoni et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Petrone S.,Bocconi University | Rousseau J.,CREST ENSAE | Scricciolo C.,Bocconi University
Biometrika | Year: 2014

Bayesian inference is attractive due to its internal coherence and for often having good frequentist properties. However, eliciting an honest prior may be difficult, and common practice is to take an empirical Bayes approach using an estimate of the prior hyperparameters. Although not rigorous, the underlying idea is that, for a sufficiently large sample size, empirical Bayes methods should lead to similar inferential answers as a proper Bayesian inference. However, precise mathematical results on this asymptotic agreement seem to be missing. In this paper, we give results in terms of merging Bayesian and empirical Bayes posterior distributions. We study two notions of merging: Bayesian weak merging and frequentist merging in total variation. We also show that, under regularity conditions, the empirical Bayes approach asymptotically gives an oracle selection of the prior hyperparameters. Examples include empirical Bayes density estimation with Dirichlet process mixtures. © 2014 Biometrika Trust.

Chopin N.,CREST ENSAE | Robert C.P.,University of Paris Dauphine
Biometrika | Year: 2010

Nested sampling is a simulation method for approximating marginal likelihoods. We establish that nested sampling has an approximation error that vanishes at the standard Monte Carlo rate and that this error is asymptotically Gaussian. It is shown that the asymptotic variance of the nested sampling approximation typically grows linearly with the dimension of the parameter. We discuss the applicability and efficiency of nested sampling in realistic problems, and compare it with two current methods for computing marginal likelihood. Finally, we propose an extension that avoids resorting to Markov chain Monte Carlo simulation to obtain the simulated points. © 2010 Biometrika Trust.

Chopin N.,CREST ENSAE | Lelievre T.,ParisTech National School of Bridges and Roads | Stoltz G.,ParisTech National School of Bridges and Roads
Statistics and Computing | Year: 2012

Because of their multimodality, mixture posterior distributions are difficult to sample with standard Markov chain Monte Carlo (MCMC) methods. We propose a strategy to enhance the sampling of MCMC in this context, using a biasing procedure which originates from computational Statistical Physics. The principle is first to choose a "reaction coordinate", that is, a "direction" in which the target distribution is multimodal. In a second step, the marginal log-density of the reaction coordinate with respect to the posterior distribution is estimated; minus this quantity is called "free energy" in the computational Statistical Physics literature. To this end, we use adaptive biasing Markov chain algorithms which adapt their targeted invariant distribution on the fly, in order to overcome sampling barriers along the chosen reaction coordinate. Finally, we perform an importance sampling step in order to remove the bias and recover the true posterior. The efficiency factor of the importance sampling step can easily be estimated a priori once the bias is known, and appears to be rather large for the test cases we considered. A crucial point is the choice of the reaction coordinate. One standard choice (used for example in the classical Wang-Landau algorithm) is minus the log-posterior density. We discuss other choices. We show in particular that the hyper-parameter that determines the order of magnitude of the variance of each component is both a convenient and an efficient reaction coordinate. We also show how to adapt the method to compute the evidence (marginal likelihood) of a mixture model. We illustrate our approach by analyzing two real data sets. © 2011 Springer Science+Business Media, LLC.

Chopin N.,CREST ENSAE | Gerber M.,Harvard University
2015 23rd European Signal Processing Conference, EUSIPCO 2015 | Year: 2015

SMC (Sequential Monte Carlo) algorithms (also known as particle filters) are popular methods to approximate filtering (and related) distributions of state-space models. However, they converge at the slow 1 /VN rate, which may be an issue in real-time data-intensive scenarios. We give a brief outline of SQMC (Sequential Quasi-Monte Carlo), a variant of SMC based on low-discrepancy point sets proposed by [1], which converges at a faster rate, and we illustrate the greater performance of SQMC on autonomous positioning problems. © 2015 EURASIP.

Singh S.S.,University of Cambridge | Chopin N.,CREST ENSAE | Whiteley N.,University of Bristol
ACM Transactions on Modeling and Computer Simulation | Year: 2013

We consider the inverse reinforcement learning problem, that is, the problem of learning from, and then predicting or mimicking a controller based on state/action data. We propose a statistical model for such data, derived from the structure of a Markov decision process. Adopting a Bayesian approach to inference, we show how latent variables of the model can be estimated, and how predictions about actions can be made, in a unified framework. A new Markov chain Monte Carlo (MCMC) sampler is devised for simulation from the posterior distribution. This step includes a parameter expansion step, which is shown to be essential for good convergence properties of the MCMC sampler. As an illustration, the method is applied to learning a human controller. © 2013 ACM 1049-3301/2013/01-ART2.

Loading CREST ENSAE collaborators
Loading CREST ENSAE collaborators