BAYESIAN INFERENCE AND MAXIMUM ENTROPY METHODS IN SCIENCE AND ENGINEERING: Proceedings of the 30th International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering
1305(2011); http://dx.doi.org/10.1063/1.3573644View Description Hide Description
At this point in time, two major areas of physics, statistical mechanics and quantum mechanics, rest on the foundations of probability and entropy. The last century saw several significant fundamental advances in our understanding of the process of inference, which make it clear that these are inferential theories. That is, rather than being a description of the behavior of the universe, these theories describe how observers can make optimal predictions about the universe. In such a picture, information plays a critical role. What is more is that little clues, such as the fact that black holes have entropy, continue to suggest that information is fundamental to physics in general.
In the last decade, our fundamental understanding of probability theory has led to a Bayesian revolution. In addition, we have come to recognize that the foundations go far deeper and that Cox’s approach of generalizing a Boolean algebra to a probability calculus is the first specific example of the more fundamental idea of assigning valuations to partially‐ordered sets. By considering this as a natural way to introduce quantification to the more fundamental notion of ordering, one obtains an entirely new way of deriving physical laws. I will introduce this new way of thinking by demonstrating how one can quantify partially‐ordered sets and, in the process, derive physical laws. The implication is that physical law does not reflect the order in the universe, instead it is derived from the order imposed by our description of the universe. Information physics, which is based on understanding the ways in which we both quantify and process information about the world around us, is a fundamentally new approach to science.
1305(2011); http://dx.doi.org/10.1063/1.3573619View Description Hide Description
In this tutorial we review the essential arguments behing entropic inference. We focus on the epistemological notion of information and its relation to the Bayesian beliefs of rational agents. The problem of updating from a prior to a posterior probability distribution is tackled through an eliminative induction process that singles out the logarithmic relative entropy as the unique tool for inference. The resulting method of Maximum relative Entropy (ME), includes as special cases both MaxEnt and Bayes’ rule, and therefore unifies the two themes of these workshops—the Maximum Entropy and the Bayesian methods—into a single general inference scheme.
1305(2011); http://dx.doi.org/10.1063/1.3573636View Description Hide Description
We present a new approach to the foundations of quantum theory and information theory which is based on the algebraic approach to integration, information geometry, and maximum relative entropy methods. It enables us to deal with conceptual and mathematical problems of quantum theory without any appeal to Hilbert space framework and without frequentist or subjective interpretation of probability.
1305(2011); http://dx.doi.org/10.1063/1.3573647View Description Hide Description
Criticisms against multiple‐choice item assessments in the USA have prompted researchers and organizations to move towards constructed‐response (free‐text) items. Constructed‐response (CR) items pose many challenges to the education community—one of which is that they are expensive to score by humans. At the same time, there has been widespread movement towards computer‐based assessment and hence, assessment organizations are competing to develop automatic content scoring engines for such items types—which we view as a textual entailment task. This paper describes how MaxEnt Modeling is used to help solve the task. MaxEnt has been used in many natural language tasks but this is the first application of the MaxEnt approach to textual entailment and automatic content scoring.
1305(2011); http://dx.doi.org/10.1063/1.3573656View Description Hide Description
We present a way to generate heuristic mathematical models based on the Darwinian principles of variation and selection in a pool of individuals over many generations. Each individual has a genotype (the hereditary properties) and a phenotype (the expression of these properties in the environment). Variation is achieved by cross‐over and mutation operations on the genotype which consists in the present case of a single chromosome. The genotypes ‘live’ in the environment of the data. Nested Sampling is used to optimize the free parameters of the models given the data, thus giving rise to the phenotypes. Selection is based on the phenotypes.
The evidences which naturally follow from the Nested Sampling Algorithm are used in a second level of Nested Sampling to find increasingly better models.
The data in this paper originate from the Leiden Cytology and Pathology Laboratory (LCPL), which screens pap smears for cervical cancer. We have data for 1750 women who on average underwent 5 tests each. The data on individual women are treated as a small time series. We will try to estimate the next value of the prime cancer indicator from previous tests of the same woman.
Bayesian Action‐Perception loop modeling: Application to trajectory generation and recognition using internal motor simulation1305(2011); http://dx.doi.org/10.1063/1.3573657View Description Hide Description
This paper is about modeling perception‐action loops and, more precisely, the study of the influence of motor knowledge during perception tasks. We use the Bayesian Action‐Perception (BAP) model, which deals with the sensorimotor loop involved in reading and writing cursive isolated letters and includes an internal simulation of movement loop. By using this probabilistic model we simulate letter recognition, both with and without internal motor simulation. Comparison of their performance yields an experimental prediction, which we set forth.
1305(2011); http://dx.doi.org/10.1063/1.3573658View Description Hide Description
This paper describes an application of Bayesian programming to the control of an autonomous avatar in a multiplayer role‐playing game (the example is based on World of Warcraft). We model a particular task, which consists of choosing what to do and to select which target in a situation where allies and foes are present. We explain the model in Bayesian programming and show how we could learn the conditional probabilities from data gathered during human‐played sessions.
1305(2011); http://dx.doi.org/10.1063/1.3573659View Description Hide Description
Visual odometry refers to tracking the motion of a body using an onboard vision system. Practical visual odometry systems combine the complementary accuracy characteristics of vision and inertial measurement units. The Mars Exploration Rovers, Spirit and Opportunity, used this type of visual odometry. The visual odometry algorithms in Spirit and Opportunity were based on Bayesian methods, but a number of simplifying approximations were needed to deal with onboard computer limitations. Furthermore, the allowable motion of the rover had to be severely limited so that computations could keep up. Recent advances in computer technology make it feasible to implement a fully Bayesian approach to visual odometry. This approach combines dense stereo vision, dense optical flow, and inertial measurements. As with all true Bayesian methods, it also determines error bars for all estimates. This approach also offers the possibility of using Micro‐Electro Mechanical Systems (MEMS) inertial components, which are more economical, weigh less, and consume less power than conventional inertial components.
1305(2011); http://dx.doi.org/10.1063/1.3573660View Description Hide Description
In this article, we investigate the issue of the selection of eye movements in a free‐eye Multiple Object Tracking task. We propose a Bayesian model of retinotopic maps with a complex logarithmic mapping. This model is structured in two parts: a representation of the visual scene, and a decision model based on the representation. We compare different decision models based on different features of the representation and we show that taking into account uncertainty helps predict the eye movements of subjects recorded in a psychophysics experiment. Finally, based on experimental data, we postulate that the complex logarithmic mapping has a functional relevance, as the density of objects in this space in more uniform than expected. This may indicate that the representation space and control strategies are such that the object density is of maximum entropy.
1305(2011); http://dx.doi.org/10.1063/1.3573661View Description Hide Description
We argue for clear separation of the exchange problem from the exchange paradox to avoid confusion about the subject matter of these two distinct problems. The exchange problem in its current format belongs to the domain of optimal decision making—it doesn’t make any sense as a game of competition. But it takes just a tiny modification in the statement of the problem to breathe new life into it and make it a practicable and meaningful game of competition. In this paper, we offer an explanation for paradoxical priors and discuss adaptive strategies for both the house and the player in the restated exchange problem.
1305(2011); http://dx.doi.org/10.1063/1.3573605View Description Hide Description
In problems of model comparison between competing regression models, one must take care not to use improper priors. Improper priors introduce inverse infinities in the evidence factors, which do not cancel if one proceeds to compute the posterior probabilities of models which have different numbers of regression coefficients. We therefore derive a simple and parsimonious proper uniform prior for multiple regression models. We then look at the evidence values that result from using this prior.
1305(2011); http://dx.doi.org/10.1063/1.3573606View Description Hide Description
The qualification matches for the European Championship of 2008 and the World Championship of 2010 for national football teams are analysed using a number of different models. Friendly matches between national teams in the same period were added to provide connectivity between the qualification pools. That model for which the highest evidence is obtained is used to predict the outcome of the championship.
As the division into groups at the tournament is always heavily debated, it is also investigated whether and how this draw into groups influences the championship.
The World Championship will be taking place at the time this poster is presented so that some of the predictive powers of the models can be judged immediately.
1305(2011); http://dx.doi.org/10.1063/1.3573607View Description Hide Description
In this paper, we present a novel derivation of special relativity and the information physics of events. We postulate that events are fundamental, and that some events have the potential to be influenced by other events. However, this potential is not reciprocal, nor are all pairs of events related in such a way. This leads to the concept of a partially‐ordered set of events, which is often called a causal set. Quantification proceeds by distinguishing two chains of coordinated events, each of which represents an observer, and assigning a numerical valuation to each chain. By projecting events onto each chain, each event can be quantified by a pair of numbers, referred to as a pair. We show that each pair can be decomposed into a sum of symmetric and antisymmetric pairs, which correspond to time‐like and space‐like coordinates. We show that one can map a pair to a scalar and that this gives rise to the Minkowski metric. The result is an observer‐based theory of special relativity that quantifies events with pairs of numbers. Events are fundamental and space‐time is an artificial construct designed to make events look simple.
1305(2011); http://dx.doi.org/10.1063/1.3573608View Description Hide Description
Symmetries and transformations are explored in the framework of entropic quantum dynamics. This discussion leads to two conditions that are required for any transformation to qualify as a symmetry. The heart of this work lies in the application of these conditions to the extended Galilean transformation, which admits features of both special and general relativity. The effective gravitational potential representative of the strong equivalence principle arises naturally.
1305(2011); http://dx.doi.org/10.1063/1.3573609View Description Hide Description
Expectation maximization (EM) algorithm and the Bayesian techniques are two approaches for statistical inference of mixture models [3, 4]. By noting the advantages of the Bayesian methods, practitioners prefer them. However, implementing Markov chain Monte Carlo algorithms can be very complicated for stable distributions, due to the non‐analytic density or distribution function formulas.
In this paper, we introduce a new class of mixture of heavy‐tailed distributions, called mixture of skewed stable distributions. Skewed stable distributions belongs to the exponential family and they have analytic density representation. It is shown that skewed stable distributions dominate skew stable distribution functions and they can be used to model heavy‐tailed data.
The class of skewed stable distributions has an analytic representation for its density function and the Bayesian inference can be done similar to the exponential family of distributions.
Finally, mixture of skewed stable distributions are compared to the mixture of stable distributions through a simulations study.
1305(2011); http://dx.doi.org/10.1063/1.3573610View Description Hide Description
Stable distributions are a class of distributions which allow skewness and heavy tail. Non‐Gaussian stable random variables play the role of normal distribution in the central limit theorem, for normalized sums of random variables with infinite variance. The lack of analytic formula for density and distribution functions of stable random variables has been a major drawback to the use of stable distributions, also in the case of inference in Bayesian framework. Buckle introduced priors for the parameters of stable random variables to obtain an analytic form of posterior distribution. However, many researchers tried to solve the problem, through the Markov chain Monte Carlo methods, e.g.  and their references. In this paper a new class of heavy‐tailed distribution is introduced, called skewed stable. This class has two main advantages: It has many inferential advantages, since it is a member of exponential family, so the Bayesian inference can be drawn similar to the exponential family of distributions and modelling skew data with stable distributions is dominated by this family. Finally, Bayesian inference for skewed stable arc compared to the stable distributions through a few simulations study.
1305(2011); http://dx.doi.org/10.1063/1.3573611View Description Hide Description
We present SASC, Self‐Adaptive Semantic Crossover, a new class of crossover operators for genetic programming. SASC operators are designed to induce the emergence and then preserve good building‐blocks, using meta‐control techniques based on semantic compatibility measures. SASC performance is tested in a case study concerning the replication of investment funds.
1305(2011); http://dx.doi.org/10.1063/1.3573612View Description Hide Description
The scientific method relies on the iterated processes of inference and inquiry. The inference phase consists of selecting the most probable models based on the available data; whereas the inquiry phase consists of using what is known about the models to select the most relevant experiment. Optimizing inquiry involves searching the parameterized space of experiments to select the experiment that promises, on average, to be maximally informative. In the case where it is important to learn about each of the model parameters, the relevance of an experiment is quantified by Shannon entropy of the distribution of experimental outcomes predicted by a probable set of models. If the set of potential experiments is described by many parameters, we must search this high‐dimensional entropy space. Brute force search methods will be slow and computationally expensive. We present an entropy‐based search algorithm, called nested entropy sampling, to select the most informative experiment for efficient experimental design. This algorithm is inspired by Skilling’s nested sampling algorithm used in inference and borrows the concept of a rising threshold while a set of experiment samples are maintained. We demonstrate that this algorithm not only selects highly relevant experiments, but also is more efficient than brute force search. Such entropic search techniques promise to greatly benefit autonomous experimental design.
1305(2011); http://dx.doi.org/10.1063/1.3573613View Description Hide Description
Nested sampling is a powerful approach to Bayesian inference ultimately limited by the computationally demanding task of sampling from a heavily constrained probability distribution. An effective algorithm in its own right, Hamiltonian Monte Carlo is readily adapted to efficiently sample from any smooth, constrained distribution. Utilizing this constrained Hamiltonian Monte Carlo, I introduce a general implementation of the nested sampling algorithm.