*35C3 preroll music*
Herald Angel: What happens if you mix Shannon's information theory and
biological systems? A dish better served hot. Please welcome our computational
systems biology chef, who will guide you through investigating the information flow
in living systems. Please welcome with a very warm round of applause Jürgen Pahle.
*applause*
Jürgen Pahle: Thanks a lot and thanks for having me. It's great that so many of you
are interested in that topic, which is not about technical systems but actually
biological cells. So, I am leading a group in Heidelberg at the university
there and we are mostly interested in how information is processed, sensed, stored,
communicated between biological cells. And we are interested in that because it's not
obvious that they actually manage to do that in a reliable fashion. They don't
have transistors. They only can use their molecules mostly proteins, big molecules
that are little engines or little motors in the cell that allow them to fulfill
their biological functions. If information processing fails in cells, you get diseases
like epilepsy, cancer and of course others. Now, cellular signaling pathways
have been studied in some detail - mostly single pathways. More and more also
networks of pathways but surprisingly little conceptual work has been
done on them. So we know the molecules that are involved, we know how they
react, how they combine to build these pathways. But we don't know how, actually,
information is transferred or communicated across these pathways and we intend to
fill that gap in our group. And, of course, first we have to we have to model
these networks, we have to model these biochemical pathways. And this is how we
proceed. So you have a you have a cell - you can't see that here - but on the upper
left corner you have that scheme of a cell with all the different components. You
have volumes in the cell where chemical reactions happen. So chemical
reactions take biochemical species: ions, proteins, what have you, and they convert
them into other chemical species, and these reactions happen in the different
compartments. Now it's very important to assign speeds or velocities to these
reactions because these speeds determine how fast the reactions happen and how the
dynamic behavior then results. And once you have done that, you can translate all
of that into a mathematical model like the one shown here on the right. This is an
ordinary differential equation system, I don't want to go into detail. I only have
like two or three formulas that might be interesting for you. So this is just
any mathematical model you have of these systems and then you can start
analyzing them. You can ask questions like: "How does the system change over
time?" That's simulation. "Which parts influence the behavior most?" "What other
stable states? Do you have oscillations, do you have a steady state?" and so on.
Now, you don't have to do that by hand, because we are actually also developing
software - that's just another thing. I guess you know that all models are wrong.
We try to build useful ones. So I said you don't have to do this by hand because we
are also into method development and we are building scientific software. One of
the softwares we build is called COPASI: COmplex PAthway SImulator. It's free and
open source, you can all go to that website, download it, play around with it
if you want. Because we also use more demanding computations which we send to
compute clusters, we also developed a scripting interface for COPASI, which is
called CoRC, the COPASI R connector. And this allows you to use the COPASI backend
with all the different tools that are in COPASI from your R programming environment
and then you can build workflows and send them to compute cluster. We think it's
easy to use. If you play around with it and you get stuck, then just let me know.
So this is software you can use, you can play around with. And where do we get the
models? Well, there is a model database that is called Biomodels.net, also free to
use, you can go there and download models. At the moment they have almost 800
different manually curated models, and almost ten times of that that are built
automatically. You can just download them in the so-called SBML format, which is the
Systems Biology Markup Language, then import it into COPASI or other software and
play around with them.
OK, so coming back to biology, one of our favorite systems is
calcium signaling. Calcium signaling works roughly like this: You have these
little - I mean the oval thing is a cell - then you have these red cones, that are
hormones, and other substances that you have in your bloodstream or somewhere
outside the cell. They bind to these black things, which are receptors on the cell
membrane. And then a cascade of processes happens that in the end leads to
an in-stream of calcium ions, these blue balls, from the ER - which is not
emergency room, but endoplasmatic reticulum, which is one of the
compartments in the cell - into the the main compartment, the cytosol of the cell.
And also calcium streams into the cell from outside the cell. And this leads to a
sharp increase of the concentration of calcium, until it's pumped out again. There
are pumps that take calcium ions and remove them from the cytosol, and pump
them out of the cell and back into the ER. This is very important because calcium is
a very versatile second messenger. That's what they call it. It regulates
a number of very important cellular processes. If you move your muscles, your
muscle contraction is regulated by calcium, learning, secretion of
neurotransmitters, transmitters in your brain, fertilization. A lot of different
things are regulated by calcium and, if you simulate the dynamic processes, you get
behavior like that. Here you can see it oscillates, it shows these regular spikes.
So this is the calcium concentration over time. Now, if you actually measure this in
real cells, and this is data measured by collaboration partners of mine in England,
you see it's not that smooth. You get these differences in amplitude of the
peaks, you get secondary spikes, you get fluctuations around the basal level, and
this is because you have random fluctuations in your system. Intrinsic
random fluctuations that are just due to random fluctuations in the timings of
single reactive events. Single reactions, biochemical reactions that happen. And in
in order to capture this behavior, because this behavior is
important, that can hamper reliable information transfer, we have to resort to
special simulation algorithms, for example the so-called Gillespie algorithm. And if
you do that and apply it to the calcium system, you can see you can actually
capture these secondary peaks and all the different other fluctuations you have in
there. Now, this is just a Monte Carlo simulation. I say "just". It's really time
consuming and demanding, because you have to calculate each and every single
reactive event in the cell. And that takes a lot of time. That's why we do that on a
compute cluster. I told you already, that calcium is a very versatile second
messenger. So you have very many different triggers of a calcium response in the
cell, things that lead to a certain calcium dynamics. And on the other hand,
downstream, calcium regulates many different things. And so you have this
hourglass or bow tie structure, and that's why people have speculated about the
calcium code: How can it be, that the proteins - I should go back - that actually do
all these cellular functions - [Softly] sorry - these green cylinders that bind
calcium and are then activated or inhibited by it, how can it be that they
know, which stimulus or which hormone is outside of the cell? They don't see them,
because there is a cell membrane around the cell, around the cytosol. So people
have speculated: Is there information encoded in the specific calcium waveform?
Is there calcium code? And how can it be that the proteins actually decode that code?
It's fairly established, that calcium shows amplitude modulation. So the
higher the amplitude of calcium, the more active get some proteins. It also shows
frequency modulation, meaning the higher the frequency of the calcium oscillations,
the more active get some proteins. But, maybe, there are other information carrying
features in the waveform, like duration, waveform timing and so on. Now a doctoral
student in my group, Arne Schoch, has looked into frequency modulation and he
actually showed that there are proteins, in that case NFAT, which is the nuclear
factor of activated T-cells, which are important in your immune system. They only
react to calcium oscillations of a certain frequency. So they they get activated in a
very narrow frequency band, and that's why we call it band-pass activation.
Okay, so I guess you all know signaling speeds of technical systems, they are fairly fast by
now. One of our results, because we quantify actually information transfer, is
that calcium signalling operates at roughly point four bits per second. If you
compare that to technical systems, that seems very low, but maybe that's enough
for all the functions that a cell has to fulfill. So how did we arrive at this result?
Well, we used information theory, classical information theory, pioneered by
people like Claude Shannon in the 40s, also by Hartley, Tuckey and a few other people.
So, they looked at technical systems, and they have this prototypical
communication system, where there is an information source on the left side,
then this information is somehow encoded. It's transmitted over a noisy channel
where the message is scrambled. Then it's received by a receiver, decoded, and then
hopefully you get the same message at the destination, that was chosen at the
information source. And in our case we look at calcium as an information source
and we study how much information is actually transferred to downstream proteins.
How do you do that? Well, information theory 101. Information theory primer.
In statistical information theory of the Shannon type, you look at random
variables. You look at events that have a certain probability of happening. So let's
say you have an event that has a probability of happening, and then Shannon
said that the information content of this event should be the negative logarithm -
which is shown here, the curve on the right hand side - should be the
negative logarithm of the probability, meaning that if an event happens all the
time - and I will show you an example later - there is no information content.
The information content is zero. There is no surprise, if that event happens, because
it happens all the time, it's like there's a sunny day somewhere in the desert.
However, if you go to lower probabilities, then the surprisal becomes bigger and the
information content rises. Now, in a system you have several events that are possible.
And if you take the average uncertainty of all possible events you get something that
Shannon called entropy. This is still not information, because information is a
difference in entropy. So you have to calculate the entropy of a system, and
then you calculate the entropy that is remaining after an observation, say. And
this difference is the information gained by the observation. Now, coming to a
simple example, let's say we have a very simple weather system where you can only
have rainy and sunny days. And let's say they are equally likely. So you have a
probability of 50%, the average of the negative logarithm is 1. So, when you
observe the weather in the system, you gain one bit per day. You can also think of
bits as the information you need, or a cell needs, to answer or decide on one yes
or no question. Now, if it's always sunny and no rain, then you get zero information
content or uncertainty. The average is zero. So you don't get any information if
you observe the weather in the desert, say. 80/20: You get a certain bit number per
day, in that case .64 per day, and you can do that for Leipzig. In that case,
Leipzig has ninety nine rainy days per year, according to the Deutsche
Wetterdienst. This gives you an information of .84 bit per day. You can do
it in a general way. So let's say you have one event with a probability of p and
another event with a probability of 1 minus p and then you get this curve, which
shows you that the information content is actually maximal if you have maximal
uncertainty, if you have equally likely events. If you have more possible events -
in that case four different ones: sunny, cloudy, rainy, and thunderstorm - you get
two bit and this is because of the logarithm. So if you have double the
amount of events and they are equally likely you get one bit more. Hope I didn't lose
anyone? Now we are always looking at processes, dynamic things, things that
change over time, and if we look at processes we have to look at transition
probabilities. So we have to change probabilities to transition probabilities
and you can summarize them in a matrix. So let's say, if we have a sunny day today,
it's more likely that it's also sunny tomorrow and less likely that it's
raining, maybe only 25 percent. And, if it's rainy today, you can't tell, it's
equally likely. These processes are also called Markov process. Markov was a
Russian mathematician and you have them everywhere. These Markovian processes are
used in your cell phones, in your hard drives, they're used for error correction,
the page rank algorithm of Google is one big Markov process. So, you're using them
all the time, nothing technological would work nowadays without them. Because we
have knowledge about today's weather, the uncertainty about tomorrow's weather decreases.
So now we have an entropy rate, instead of an entropy. The difference is,
again, the information you gain by today's weather. You can do the maths in our
example. The entropy would be .92 bit per day and the entropy rate, given that you
know today's weather, is less. It's .87 bit per day. Now, to complicate things a
bit more, maybe, we also look at a second process in that case air pressure and you
can measure air pressure with these little devices, the barometers and maybe, if it's
sunny today and the air pressure is high, in 90 percent you get a sunny day
tomorrow. Normally in 10 percent of the cases you get a rainy day and so on you
can go through the table. In our case, I looked it up yesterday. We had a high air
pressure and it was raining. So in our little model system it would mean, that
it's sunny today. Now, I told you information is a decrease in uncertainty.
How much information do we get by the barometer, by knowing the air pressure?
This is the difference in uncertainty without barometer and with the barometer
and in our case we have to assume that the probability of high and low air pressure
is the same. And we get .39 bit per day, that we gain by looking at the air
pressure. Now, what does that have to do with biological systems? Well we have two
processes. We have a calcium process that shows some dynamics and we have the
process of an activated protein that does something in the cell. So we can look at
both of these and then calculate how much information is actually transferred from
calcium to the protein. How much uncertainty do we lose about the
protein dynamics, if we know the calcium dynamics? This is mathematically exactly
what we are doing and this is called transfer entropy. It's an information-
theoretic measure developed by Thomas Schreiber in 2000. There are some
practical complications, that we are working on, and this is what we are using
actually for the calculations. So in our case we have data from experiments or we
use models of calcium oscillations and then we couple a model of a protein to
these calcium dynamics. This gives us time courses, both of calcium and protein,
stochastic time courses, including the random fluctuations. And then we use the
information-theoretic machinery to study them. And some of our results I want to show
you. For example, if you increase the system size, if you increase the particle
numbers, if you make the cell bigger, then the information that you can transfer is
higher. Meaning, if the cell invests more energy and produces more proteins, it can
actually achieve a more reliable information transfer, which comes of course
with costs for the cell. Also, it seems, that if you use more complicated dynamics
- meaning not only spiking, but maybe bursting behavior where you have secondary
spikes - then you can transmit more information because the input signal
carries more information or can carry more information in its different features.
Another result is that proteins - a very interesting result I think - is that
proteins can actually be tuned to certain characteristics of the calcium input.
Meaning, with all the different calcium sensitive proteins in the cell they are
tuned to a specific signal. So they only get activated or these pathways only
allow information transmission, if a certain signal is observed in the cell by
these proteins. So, in a way the 3D structure of the protein defines how it
behaves dynamically, how quickly it binds and so on, how many binding sites it has,
and then this dynamic behavior determines to what input signals that protein is
actually sensitive. On the right hand side you can see some calculations we did. The
peaks actually show where this specific protein, which is a calmodulin-like protein
- you don't have to memorize that, it's a very important calcium sensitive protein -
where these differently parameterized models actually get activated and allow
information transfer. And this allows differential regulation because you have
all the different proteins. You have only one calcium concentration and only the
proteins that are sensitive to a specific input get activated or do their things in
the cell. Now if you look at more complicated proteins - so Calmodulin, the
one I just showed you, was only activated by calcium - more complicated proteins,
like protein kinase C, for example, they are both activated and inhibited. So they show
biphasic behavior, where in an intermediate range of calcium
concentration they get activated, with very high or very low concentrations they
are inactivated. You can actually see that these more complicated proteins allow a
higher information transfer and again producing these more complicated proteins
might be more costly for the cell, but it can be valuable, because they allow more
information to be transferred. And this you can see in this plot where we actually
scanned over the activation and the inhibition constant of these model
proteins and you can see that you have these sweet spots where you get a very
high information transfer. So color coded is transfer entropy. Now, coming to a
different system: Just quickly, we also looked at other systems of course. Calcium
signaling is just one of our favorite ones. We also looked at bacteria and this is
E. coli, a very famous model system for biologists. These are cells that can
actually move around because they have little propellers at their end. They want to
find sources of nutrients, for example, to get food. So they swim into a direction
and then they decide whether to keep swimming in that direction
or whether to tumble, reorient randomly, and swim in some other direction. The
problem for them is they are too small. They can't detect a concentration gradient
of nutrients, of food between their front and the back of the cell. So they have to
swim in one direction and then they have to remember some nutrient concentration of
some time back and then they have to compare: Is the nutrient
concentration actually increasing? Then I should continue swimming. If it's
decreasing, I should reorient and swim in some other direction. This allows them to,
on average, swim towards sources of food. In order to compare over time the nutrient
concentrations they have to memorize, they have to know how much nutrients where
there sometime ago. For that they have a little memory and the memory is actually
in the - you can see on the left hand side the receptor that actually senses these
nutrients. They can be modified, these receptors, we call that methylated. So they
get a methylation group attached. They have different states of methylation, five
different ones in that model we are looking at. This builds a memory. And we
looked into that, we quantified that with information theory. This is a measure,
this is called mutual information. It's not transfer entropy, it's another measure
of, in that case, statical information. You can see, this is the amount of
information that is actually stored about the nutrient concentration that is outside
of the cell. This is in nats, it's not in bits. It's just a different - you can
translate them - it's just a different unit for information. You can also see how the
different methylation states - so these are the colored curves - how they go
through or how they are active with different nutrient concentrations. This is
ongoing research. So, maybe, next time, hopefully, next time, I can show you much
more. Just to finish this, we also look at timescales, because the timescales have to
be right. The system adapts. So if you keep that cell in a certain nutrient
concentration, it adapts to that nutrient concentration and goes back to its normal
operating level. Now, if you increase the nutrient concentration again, it shows some
swimming behavior. So it adapts, but it also has to decide, it also has to compare
the different nutrients at different positions. That's how they have to manage
the different timescales of decision making and memory or adaptation and we are
looking into that as well. Coming to the conclusions, I hope I could convince you
that information theory can be applied to biology, that it's a very interesting
topic, it's a fascinating area and we are just at the beginning to do that. I also
showed you that it's such that in signaling pathways the components can be
tuned to their input, which allows differential regulation. So even though
you don't have wires you can still specifically activate different proteins
with one signal or multiplex, if you want. We are of course in the process of
studying what features of the input signal are actually information-carrying. So we
are looking into things like wave form and timing. And we want to look into how these
things change in the deceased case. So, if you have things like cancer where certain
signalling pathways are perturbed or fail, we want to exactly find out what does that
do to the information processing capabilities of the cell. We also found
out that estimating these information theoretical quantities can be a very
tricky business. Another project we are doing at the moment is actually only on
how to interpret these in a reliable manner, how to estimate these from sparse
and noisy data. So that's also ongoing work. I would like to thank some of my
collaborators, of course, my own group, but also some others, in particular the Copasi
team, that is spread all over the world. And with that I would like to thank you
for your attention and I would be happy to answer any question you might have.
Thank you.
*applause*
Herald Angel: ... a very warm applause for Jürgen. If you have questions, there
are two microphones, microphone number one, microphone number two and please
speak loudly into the microphone. And, I think the first one is microphone number two.
Your question please. Microphone 2: Has there been any work done
on computational modelling of the G-protein coupled receptors and the second messenger
cascades there. Jürgen: Can you repeat that, sorry.
Microphone 2: Has there any work been done on computational modelling of G protein-
coupled receptors Jürgen: G protein?
Microphone 2: Yeah. Jürgen: Oh yes, I mean we are doing that
because calcium is actually... I mean the calcium signal is actually triggered by a
cascade that includes the G protein. Most of these receptors are actually G coupled
or G protein coupled receptors. So that's what we are doing.
Angel: Thank you. Microphone number two again.
Microphone 2: First of all thanks for the talk. I want to ask you talked a
little bit about how different proteins get activated by different signals and
could you go a bit into detail about what kind of signal qualities the proteins can
detect? So are they triggered by specific frequencies or specific decays, like which
characteristics of the signals can be picked up by the different proteins?
Jürgen: Well, that's actually what we study. I mean we have another package that
is linked here, the last one, the oscillator generator. This is a package in
R that allows you to create artificial inputs, where you have complete control of
all the parameters like amplitude and duration of the peak, duration of the
secondary peak, frequencies of the primary peaks of the secondary peaks, refraction
period and so on. You have complete control and at the moment we are also
running scans and want to find out what proteins are actually sensitive to what
parameters in the input signal. What we know from calcium is that, for example,
calcium calmodulin kinase 2, also a very important protein in the nervous system,
that shows frequency modulation. It has also been shown experimentally where they
put that protein on a surface, they immobilized it on a surface, and then they
superfused it with calcium concentrations or with solutions of different calcium
concentration in a pulsed manner and they measured the activity of that protein and
they showed that, with increasing frequency, the activation gets bigger. At the same
time it also shows amplitude modulation, okay? It's also sensitive to the
amplitude, meaning the absolute height of the concentration of calcium.
Microphone 2: Thanks. Jürgen: Thank you.
Angel: And again number two please. Microphone 2: Hey. So you talked about a
lot of on and off kinetics and I wonder, if you think about neurons, which are not only
having on and off, but also many amplitudes that take a big role in development of
cells and synapses. How do you measure that, so how do you measure like baseline,
sporadic activity of calcium? Jürgen: Well, in our case there are
different ways of measuring calcium. That's not what we are doing...
Microphone 2: ... not really measuring, sorry, but more like how do you integrate it
in your system? Because it's not really an on/off reaction but it's more like a
sporadic miniature. Jürgen: Yeah, I mean in the case of
calcium you have these time courses, okay? And we look at the complete time
course. So we have the calcium concentration sampled at every second or
half second in the cell by different methods. So our collaboration partners
they use different dyes that show fluorescence, say, when they bind calcium.
Some others show bioluminescence. And then we use these time courses. In the neural
system it's a bit different. There you also get the analog mode, where neurons are
directly connected and they exchange substances, but most of the case you have
action potentials and I didn't go into neural systems at all because things there
are totally different. You get these action potentials that are uniform mostly,
so they they all have the same duration, they all have the same amplitude. And then
people in neuroscience or computational neuroscience mostly they boil the
information down to just the timings of these peaks and they use this information
and mathematically this is a point process and you can use different mathematical
tools to study that. We are not really looking into neurons. We are mostly
interested in non-excitable cells, like liver cells, pancreatic cells and so on,
cells that are not activated, they don't show massive depolarization, like
neurons. Thank you. Angel: Thank you. And obviously again
number two. Microphone 2: Hi. So, you mentioned CaM
kinases 2. I got that you don't work on neuroscience specifically, but I'm
pretty sure you have a quite extensive knowledge in the subject. What do you
think about this, I would say, hypotheses that were quite popular a few years ago, I
think in the US mainly, about the fact that the cytoskeleton of neurons can
actually encode and decode through kinases in the cytoskeleton memories like bits in
- you know - in a hard drive. What's your feeling?
Jürgen: Well, I'm not going to speculate on that specific hypothesis because I'm
not really into that, but I know that many people are also looking into spatial
effects which I didn't mention here. I mean the model I showed you is a spatially
homogeneous model. We don't look at concentration gradients within the cell,
our cells are homogeneous at the moment, but people do that. And of course then you
can look into things, for example, like a new topic is morphological computation,
meaning that spatially you can also perform computations. But, if you're
interested in that, I mean, we can talk offline...
Microphone 2: ... do you buy into this theory...
Jürgen: ... I can give you some pointers there..
Microphone 2: ... but do you have a good feeling about these theories or you think
they're clueless. Jürgen: Well, I think that the spatial
aspect is a very important thing. And that's also something we should
look at. I mean, to me random fluctuations are very important, intrinsic fluctuations
because you can't separate them from the dynamics of the system. They are always
there, at least some of the fluctuations. And also the spatial effects are very
important, because you not only have these different compartments,
where the reactions happen, but you also have concentration gradients across the
cell. Especially with calcium, people have looked into calcium puffs and calcium
waves because, when you have a channel, that allows calcium to enter, of course directly
at that channel you get a much higher calcium concentration and then in some
cases you get waves that are travelling across the the cell. And to me it sounds
plausible that this also has a major impact on the information processing.
Yeah. Thank you. Angel: Thank you. In this case, Jürgen,
thank you for the talk. And please give a very warm applause to him.
*applause*
Jürgen: Thank you.
*applause*
*postroll music*
subtitles created by c3subtitles.de in the year 2019. Join, and help us!