447 research outputs found
Path Integral Approach to Random Neural Networks
Full text linkIn this work we study of the dynamics of large size random neural networks.
Different methods have been developed to analyse their behavior, most of them
rely on heuristic methods based on Gaussian assumptions regarding the
fluctuations in the limit of infinite sizes. These approaches, however, do not
justify the underlying assumptions systematically. Furthermore, they are
incapable of deriving in general the stability of the derived mean field
equations, and they are not amenable to analysis of finite size corrections.
Here we present a systematic method based on Path Integrals which overcomes
these limitations. We apply the method to a large non-linear rate based neural
network with random asymmetric connectivity matrix. We derive the Dynamic Mean
Field (DMF) equations for the system, and derive the Lyapunov exponent of the
system. Although the main results are well known, here for the first time, we
calculate the spectrum of fluctuations around the mean field equations from
which we derive the general stability conditions for the DMF states. The
methods presented here, can be applied to neural networks with more complex
dynamics and architectures. In addition, the theory can be used to compute
systematic finite size corrections to the mean field equations.Comment: 20 pages, 5 figure
Analytical Solution of the Off-Equilibrium Dynamics of a Long Range Spin-Glass Model
Full text linkWe study the non-equilibrium relaxation of the spherical spin-glass model
with p-spin interactions in the limit. We analytically
solve the asymptotics of the magnetization and the correlation and response
functions for long but finite times. Even in the thermodynamic limit the system
exhibits `weak' (as well as `true') ergodicity breaking and aging effects. We
determine a functional Parisi-like order parameter which plays a
similar role for the dynamics to that played by the usual function for the
statics.Comment: 8 pages, Roma preprin
Subextensive singularity in the 2D Ising spin glass
Full text linkThe statistics of low energy states of the 2D Ising spin glass with +1 and -1
bonds are studied for square lattices with , and =
0.5, where is the fraction of negative bonds, using periodic and/or
antiperiodic boundary conditions. The behavior of the density of states near
the ground state energy is analyzed as a function of , in order to obtain
the low temperature behavior of the model. For large finite there is a
range of in which the heat capacity is proportional to .
The range of in which this behavior occurs scales slowly to as
increases. Similar results are found for = 0.25. Our results indicate that
this model probably obeys the ordinary hyperscaling relation , even though . The existence of the subextensive behavior is
attributed to long-range correlations between zero-energy domain walls, and
evidence of such correlations is presented.Comment: 13 pages, 7 figures; final version, to appear in J. Stat. Phy
Response of an Excitatory-Inhibitory Neural Network to External Stimulation: An Application to Image Segmentation
Full text linkNeural network models comprising elements which have exclusively excitatory
or inhibitory synapses are capable of a wide range of dynamic behavior,
including chaos. In this paper, a simple excitatory-inhibitory neural pair,
which forms the building block of larger networks, is subjected to external
stimulation. The response shows transition between various types of dynamics,
depending upon the magnitude of the stimulus. Coupling such pairs over a local
neighborhood in a two-dimensional plane, the resultant network can achieve a
satisfactory segmentation of an image into ``object'' and ``background''.
Results for synthetic and and ``real-life'' images are given.Comment: 8 pages, latex, 5 figure
Mutual Information of Population Codes and Distance Measures in Probability Space
Full text linkWe studied the mutual information between a stimulus and a large system
consisting of stochastic, statistically independent elements that respond to a
stimulus. The Mutual Information (MI) of the system saturates exponentially
with system size. A theory of the rate of saturation of the MI is developed. We
show that this rate is controlled by a distance function between the response
probabilities induced by different stimuli. This function, which we term the
{\it Confusion Distance} between two probabilities, is related to the Renyi
-Information.Comment: 11 pages, 3 figures, accepted to PR
A Logic of Blockchain Updates
Full text linkBlockchains are distributed data structures that are used to achieve
consensus in systems for cryptocurrencies (like Bitcoin) or smart contracts
(like Ethereum). Although blockchains gained a lot of popularity recently,
there is no logic-based model for blockchains available. We introduce BCL, a
dynamic logic to reason about blockchain updates, and show that BCL is sound
and complete with respect to a simple blockchain model
On the conditions for the existence of Perfect Learning and power law in learning from stochastic examples by Ising perceptrons
Full text linkIn a previous letter, we studied learning from stochastic examples by
perceptrons with Ising weights in the framework of statistical mechanics. Under
the one-step replica symmetry breaking ansatz, the behaviours of learning
curves were classified according to some local property of the rules by which
examples were drawn. Further, the conditions for the existence of the Perfect
Learning together with other behaviors of the learning curves were given. In
this paper, we give the detailed derivation about these results and further
argument about the Perfect Learning together with extensive numerical
calculations.Comment: 28 pages, 43 figures. Submitted to J. Phys.
Statistical Mechanics of Support Vector Networks
Get PDFUsing methods of Statistical Physics, we investigate the generalization
performance of support vector machines (SVMs), which have been recently
introduced as a general alternative to neural networks. For nonlinear
classification rules, the generalization error saturates on a plateau, when the
number of examples is too small to properly estimate the coefficients of the
nonlinear part. When trained on simple rules, we find that SVMs overfit only
weakly. The performance of SVMs is strongly enhanced, when the distribution of
the inputs has a gap in feature space.Comment: REVTeX, 4 pages, 2 figures, accepted by Phys. Rev. Lett (typos
corrected
Large time off-equilibrium dynamics of a manifold in a random potential
Full text linkWe study the out of equilibrium dynamics of an elastic manifold in a random
potential using mean-field theory. We find two asymptotic time regimes: (i)
stationary dynamics, (ii) slow aging dynamics with violation of equilibrium
theorems. We obtain an analytical solution valid for all large times with
universal scalings of two-time quantities with space. A non-analytic scaling
function crosses over to ultrametricity when the correlations become
long-range. We propose procedures to test numerically or experimentally the
extent to which this scenario holds for a given system.Comment: 12 page
Time Dependent Local Field Distribution and Metastable States in the SK-Spin-Glass
Full text linkDifferent sets of metastable states can be reached in glassy systems below
some transition temperature depending on initial conditions and details of the
dynamics. This is investigated for the Sherrington-Kirkpatrick spin glass model
with long ranged interactions. In particular, the time dependent local field
distribution and energy are calculated for zero temperature. This is done for a
system quenched to zero temperature, slow cooling or simulated annealing, a
greedy algorithm and repeated tapping. Results are obtained from Monte-Carlo
simulations and a Master-Fokker-Planck approach. A comparison with replica
symmetry broken theory, evaluated in high orders, shows that the energies
obtained via dynamics are higher than the ground state energy of replica
theory. Tapping and simulated annealing yield on the other hand results which
are very close to the ground state energy. The local field distribution tends
to zero for small fields. This is in contrast to the Edwards flat measure
hypothesis. The distribution of energies obtained for different tapping
strengths does again not follow the canonical form proposed by Edwards.Comment: Minor changes and journal reference added. 10 pages 6 figure
- …
