Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/555 Thanks!
The title of our next talk is quite
descriptive, without me giving too much
of an introduction, I think
basically we live in a world in which
algorithms make more and more decisions
about our daily lives.
Many people are working on improving
these algorithms.
Not as many are actually
thinking about the implications.
Say hi to your new boss.
How Algorithms Might Soon Control
Our Lives is the title of our next
talk. Please give a warm welcome
to Andrew Stevens.
So hello, everyone,
I have to say, I'm really quite excited
being here again and terrified
as well, but mostly excited.
And first and foremost, I want to
thank the organizers for inviting
me again and for letting me speak at is
really, really great conference.
So
as we said before, the title of my talk
is Say Hi to Your Boss and I'm going
to talk about algorithm's and about like
shifting decision powers from
humans to machines.
And in case you were asking yourself,
why is this important?
Well, let's just ask a friend
here. I usually like to do that with
Google autocomplete.
And normally this gives me kind of like
some controversial statements or
algorithms are stupid or algorithms will
never work.
But in this case, it seems like and
it's pretty unambiguous that algorithms
will play a very large role in this
world. And as I said, this is like
a big chance because algorithms can
improve our lives a lot.
But it's also a problem
because we're shifting a lot of the
decisions that are now made by people
into the hands of machines.
And in many cases, we don't understand
much how these machines work and
how exactly that make the decisions.
And so I would say my main qualification
for making this talk is that I shot
myself in the foot with data.
And this is a lot of times and I became
interested in how why algorithms
are doing like so many things
that we don't anticipate and why they
sometimes behave in ways that seem
strange and like contradictory
to what we actually want to achieve with
them.
And so that's what I want to talk about.
And we're going to do it like this.
So first, I will give
you some theory about what
an algorithm actually is and how
machine learning algorithms to make
decisions and how this whole
big data thing and
like the new data driven society age
plays into this whole affair.
And finally, we'll show you some of the
use cases for algorithms in our daily
lives today.
And after that, we will be equipped with
everything we need to know in order to
start with some experiments.
So I'm coming from physics.
And when I would try to understand
something, I usually do an experiment and
try to to break the thing I like, make it
explode or whatever.
And so we're trying to do the same
thing here with our algorithms.
And I have picked out two and
two case studies that are going to
present one about discrimination
through algorithms and another one
about the anonymization.
So and finally, I want to end with
some proposals and some ideas on how we
can actually make the most of algorithms
in this kind of setting and also
like to control
and better understand what algorithms are
doing.
OK.
So first and
as I said, I want to talk a bit about
algorithms, and I just want to give you a
very, very basic overview of
machine learning and decision making, the
algorithms. So please excuse me if there
are any experts in the audience and
probably making like a lot of
simplifications here.
OK, um, what is an algorithm
here? I give you an example.
Um, basically an algorithm is just a
recipe that can be followed by a computer
or a human being and
gives a human being on the computer
step by step instructions to achieve
a certain goal. So in this case, we want
to activate a trapdoor and we
want to do that only if I'm on the trap
door. So the algorithm has to decide if
it's me and then if it's me, it can open
the trap door. Otherwise it has to wait.
And now this is a pretty fancy algorithm
because it needs some information about
me and it needs
kind of an intelligent way to
decide if it's the right person that
is standing on the Treptow or not.
So how does the algorithm get that
information?
Well, it uses machine learning and
machine learning is a
way to automatically generate a
model that we can check
against some training data and
which we can then use to explain
the data and in addition to also
predict some unknown data.
So, as you might know from school and
just memorizing data and like
reproducing what you already know can get
you through tests, but normally it won't
make you passed with flying colors.
So ideally, we want to have something
that can, in addition of memorizing data,
also make predictions about data that we
have never seen before.
And this is what machine learning helps
us to do and a bit more
formalized the way we could,
like, look at it as
a model and some data.
So here on the right, I just show you
several possible models
that we can choose from.
Normally, we can write them as explaining
some variable Y as a function
M for model, which
takes some attributes or variables X
and some parameters P and
returns some value for the quantity
that we want to predict.
And now we can use data
to train our models and to like select
the models that are compatible
to our training and eliminate the ones
that are not compatible to it.
And you can see here on the right, we
have eliminated all these models that are
shown in red, whereas the models that are
green can actually be compatible with our
data.
And now we can use those models to make a
prediction about unknown data points as
well and which is shown here.
And usually you see there is some
error, also some discrepancy between
the model and the data that we try to
explain. And this error is usually called
Epsilon.
Now, Epsilon
can be decomposed further into like
several parts, so there's a systematic
error, which is mostly due to
like miss calibrations or like
measurements that we make each time when
we try to like like measure a given
variable. So we can think about this as,
for example, the speedometer on your car,
which gives you intentionally a reading
which is too low in order to make sure
that you not overstep the speed limit.
And in addition to the systematic errors,
we have also some noise in our data,
which is due to like the error, the
internal process that has generated the
data or the model that we
use and the measurement apparatus
that we use to capture the data.
And finally, we have some hidden variable
errors, which is not random
noise, but which are errors that are due
to variables that have an impact on the
outcome of the model, but which we do not
know and which we therefore cannot use
to model the data.
So
that's the basics of like model
generation. And now you probably all have
heard about big data and data driven
society. And the effect
that this has on model generation is
threefold.
For once you can see here, for example,
we have more or less the data volume
in 2000 compared to the data volume in
2015.
You can see that today we have a lot more
data on our hands to make predictions and
train models.
And we also have data of
a much greater variety than before.
So to understand this effect,
we can have a look at this graph here,
which shows some random data
that we measure with a pretty large
noise, as you can see.
And this data also contains some
information. And I don't know
who you can tell me if
either the green points or the red points
are showing have a higher value.
So I guess not.
But now what we can do is we can just
take the data and leverage it, and
by doing that, we can reduce
the amount of noise in our data.
And when we have enough samples that we
can look at, we can make the noise so
small that we can really detect some
signal in the data.
In this case, the signal is just 0.01
high.
And so having more data in our hands
allows us to train models
which can take into account
smaller effect.
Also, as I said, big data
does not only give us more of the same
data, but it gives us different kinds of
data so we can think, for example, about
all the smart devices that you have in
your home, like your smart fridge, your
door, the maybe automated
smoke detector that all collect
data about you and your interactions.
And like, we can use the data to
incorporate it into our models and to
make better predictions.
And so this moves
some of the noise that where
that was into hidden variables, into
the model where we can use it to make
predictions.
Now, interpreting
models can be hard
or it can be very simple, depending on
the model. So there's some machine
learning algorithms like like
decision tree classifiers, which are
pretty easy to interpret
because we can just follow
this graph here and see exactly
how the algorithms makes his decision
or its decision about a given
data point.
Other models, like, for example,
this neural network on the right side are
really hard to interpret.
So we can't get an intuitive
feeling for how this model actually makes
its decisions.
And in fact, you maybe have
seen those pictures here.
They show basically a neural network
working in reverse.
So
they give us an idea of how,
like, this neural network understands
a picture in this case.
And you can see that, for example, we
have several structures here that emerge
at different places in the image and that
are generated by the neural network while
it's recognizing the features
of the image.
And this method has been developed
actually, because it's really, really
difficult to understand what a neural
network is doing otherwise.
So the only way we have to do that
is to like try to see
what kind of input data the
network produces when we give it a
certain output data.
So.
Now, what can you do with algorithms
here to try to classify
the uses of algorithms in our daily life
until like three different risk groups,
so you could say that you have a low
risk group which basically just affects
our lives on the super
superficially and on the algorithms
that make the decisions there.
If they go wrong or if they misbehave,
it would be only mildly annoying.
Then we have the medium risk area where
failure or misbehavior of an algorithm
would be a bit more severe to our life,
but wouldn't be fatal.
Which is only in the high risk area here,
where algorithms really can take
decisions that can affect human
lives and that can really be
life changing in that sense.
Now, a few examples for the first group.
Would be, for example, personalization
services, so whenever you go to a website
like Facebook or Amazon or Netflix
and the website shows you some content
and it tries to show you content, which
is very interesting to you, and it uses
an algorithm to do that and tries
to predict from the articles that you
have viewed before which articles,
for example, you will find interesting.
So this so-called recommendation engine,
and it's in wide use in all kinds of
services today.
Also, we have, of course, individualized
ad targeting.
You might notice if you go to some
website and then you like your
product and afterwards you surf around on
the website, on the Web, and
then ads for this kind of product like
seem to like haunt you everywhere you go.
And this is also due to like machine
learning algorithms that like try to
predict which kind of ads you will find
interesting and that like show these ads
to you on all kinds of different
websites.
And of course, there are algorithms that
can do customer ratings.
So, for example, if you want to like
order product online, they could
estimate how likely it is that you would
pay the invoice for that article.
And if it's not very high, then
it would the system would only send you
the article if you paid the money in
advance.
And of course, there are things like
customer demand prediction.
So the holy grail of this would
be to actually know what do you want
to buy before you know it and then send
it to you, to your door.
And I think after reading
a pattern, I think this is also what
Amazon is trying to do in some cases.
So these things just like affect
our lives superficially.
And if something happens, it
affects us not very in a very deep
way. And there are other uses of
algorithms in our life, for example, and
a big topic that is coming up now with
big data and more data that we can
collect about individuals is personalized
health. So making decisions about
possible treatments and lifestyle based
on data that we collect about you, for
example, your heart rate, your
pulse, your how much you move around,
how many stairs you climb each day.
So this is a large potential for
improving, for example,
areas such as medicine, but also other
ones. And we use
the same or similar classification
algorithms as and the
applications that I showed you before.
So another thing is person
classification.
So here we want to predict,
for example, how likely it is that a
person will commit a crime or
will be a terrorist.
And these are kind of algorithms that are
already in use today by, for example,
governments to like
issue restricted travel
permits and to like
mark some people that have a high
risk profile due to the algorithm for
screening. I think there are many topics
here also that deal with this problem,
that problem especially.
And of course, they are autonomous cars,
planes and machines, which
are currently being developed already in
service and which will take over
like driving from
people in a few years or a few decades
maybe. And finally, this
automated trading, which is mostly
invisible to us, but which has also a
huge impact because 95 or
even 99 percent of all trades today
are actually performed by algorithms in
the machines anymore.
So.
Finally, there's the high risk area
where we have such things like military
intelligence and or intervention.
We also have already some governments
that already use algorithms to
like predict targets
for, for example, drone strikes.
And we also can have, of course,
governments that use machine learning and
algorithms for political oppression.
So, for example, to train
firewalled systems using
juristic algorithms to detect
traffic that should be filtered out.
And there's also critical infrastructure
services like the electricity grid
or other things that are
critical to us and which are also
sometimes ungoverned or like controlled
by algorithms already.
So as you can see already today,
we have many areas in our life with
algorithms and not humans make
the decisions. And if we would like
plotless again on this graph, you would
see that that most things with
algorithms decide today actually in the
green on the yellow area.
And we have some things that are be
touching the critical part of our lives.
And now what big data
and advanced machine learning would do in
the coming years is probably to make
to both widen the applicability
of algorithms so we can use them for
domains where we couldn't use them
before, like speech recognition,
customer service and many other things.
And we will also, like penetrate deeper
into our lives and making decisions which
really can affect us on a more personal,
more intimate and more critical
level.
Good. So this is all I
wanted to show you in theory, and now
I want to use the remaining time to
show you two experiments, which I did.
So there are lots of things that can go
wrong when you use algorithms.
But I picked two topics
here that I find especially important.
And the first topic that we are looking
at is discrimination through algorithms.
So here the question is,
can an algorithm that is trained
by a human or by an
earlier manual decision process
actually also discriminate against
certain groups of people?
You know, like discrimination still is a
very big problem in our society.
And we have like fought for many,
many years to, like, push it back.
And the question is, of course,
now, as we shift so
much of the decision, power from humans
to machines, can we actually
eliminate the discrimination that we
still have in the system or are we going
to carry it over to like this automated
decision making?
The definition of discrimination,
again here is
a treatment or consideration
of a certain person
that is made based on his or her
group, class or category category and
is not based on an on his or her
individual merit.
So that means
that we like professor or
we put at a disadvantage certain
kinds of people, according to their their
group or some protected attribute, which
can be, for example, the ethnicity,
the gender or the sexual orientation
of the person.
And now we
need a way, of course, to measure this
discrimination and
the measurement of the truth here, I
mean, steroids, of course,
is has been developed in the US
and it's called disparate impact.
And it's quite nice because it uses
a very clear and simple mathematical
model to to explain
discrimination.
So basically, this model says
that we have a process, see,
which acts on people that have
a given aptitude X or don't have it.
So, for example, men
and women and we measure the
outcome of this process and we are
interested in the probability of
the decision being, yes, for
a member of the Group X versus the
probability of business for the
member of the other group.
And so we can just have
a look at the conditional probability P
of making it through this process.
Being a member of Group X
equals zero, divided by the probability
of making it through the process of
being and being a member of the other
group. And when we can when we divide
these two quantities, we get the
parameter tower, which describes
the amount of discrimination that we have
in the system. And for
normal purposes, we can choose a given
title, for example, 80 percent.
And if we see that the smaller we can
say, oh, this process contains
discrimination and this is nice because
it measures discrimination, not only if
it's done intentionally, but
also when it's happening inadvertently.
So without wanting it.
So it doesn't really matter after the
process or the people in this process
want to discriminate if they do it.
Nevertheless, maybe unconsciously, this
measure can give us an idea about it.
And of course, in practice we can
deal with probabilities that we have to
to measure the.
So the number of people
in each category and then we can,
like, make an estimate of this parameter,
Tolba, just dividing this number
here by these two numbers and dividing
this, again, by this number divided by
the other two numbers. So it's pretty
easy, very straightforward.
And now
I want to show how we can
use this to test a given process
that we take from we
take decision power from people
and give it to an algorithm.
And the example that I choose here is
a H.R.
process or a hiring process.
And we want to
hear select candidates
based on the data, for example,
the CV and other data about themselves
that they submit to a potential
employer employee.
And the benefits of this are,
of course, saving time in the screening
process and also improve the choice
of candidates and this is a I choose
this example because it's actually
something that is widely done already.
So chances are that if you have
applied to a job recently, you have
probably been subject subjected to
this kind of process and also several
startups in the US, but also in Europe to
try to implement this kind of data driven
hiring processes.
So it's something that's really already
happening.
OK, again, the setup is pretty simple.
We have some information about the
candidate that we submitted to a human
reviewer and that human Rabiya makes a
decision to invite a candidate or not.
And it also gives that information to
an algorithm as a training data.
And the algorithm tries then to replicate
the decision of the human whether to hire
the candidate or not.
So
the set up, as I say, we used to live
any work samples and other publicly
available information about the candidate
that we can get as an input, we then
use a human to like make the decision
about a given candidate, either yes or
no. And we train the algorithm
on this data.
So and the approach that we have here is
a so-called big data approach.
So we basically try to get as much
data about every other candidate
as we can and put it all into the
algorithm and let the algorithm figure
out what it does with it.
So the decision model for this
rather simple and show it here
in order to decide if we hire a given
candidate, we can define a function
as which is the score
that has several parts.
One part is to measure the score of the
merit of the candidate, which is based on
on his or her abilities.
And another part is a discrimination
Malazan bonus.
So we can see that as
increasing or decreasing the total score
of the candidate based on his or her
membership in a given group.
And then, of course, we also have some
element of luck, which we have said to 20
percent, for example, here.
And then we just add
all these three components together.
And if the 50 are larger
than so, we add these two things
together. And if they are larger than the
given bar, we invite the candidate.
If they are not, we don't invite him or
her. And you can see here
the bar has a different
height, depending on the group of the
candidate, if there is discrimination in
the system.
OK, now
we can train a predictor for that
model to which we give the information
about the candidate and also a lot of
other information which it causes here.
So everything else that we can find, for
example, in public records or in other
information that we can get our hands on.
And then we trained
as a predictor for the to
predict the outcome of the hiring
process. And we can see
what the results are. So and since
it's pretty hard to get our data on,
to get our hands on real world data,
and what I did instead here is to
simulate 10000
samples of an agent based model
where we just choose a function,
see you and some disparate impact
and generate training data with that.
And then we can use a standard machine
learning algorithm, in this case, a
support vector machine to test that
data and measure
the discrimination that the algorithm
produces.
OK, this is shown in this graph
here.
It's a bit complicated, so let's go
through it one by one.
So what we see on the X axis is the
amount of information that our
algorithm has about the attributes X
of the candidate.
So this is the protected attribute
to which we don't want to give
information away.
And so if we are at
zero and means that the algorithm doesn't
have any information at all about the
candidate, if we are at one and
it means the algorithm has all the
information about the protected attribute
of the candidate, and if we are at zero
point five, it means that we have the
correct information and about 50 percent
of the cases.
OK, then we have our
perimeter tower, which is a disparate
impact, and here we have set this value
to zero point five, and this means that
the chance of making it through the
process, being a member of Group X
is twice as high as for members, for
people that are not member of this group.
Now, here, above, we see the
prediction, fidelity of our algorithm,
which is between 86
and about 90 percent, and
which also increases as we increase the
number at the rate gamma to which the
information leaks into the system.
And finally, we have here
the towel.
So the discrimination, the amount of
discrimination done by the
algorithm measured, again, as a function
of the information leakage.
So what that means is that the more
information we provide about this
projected attribute we provide to the
algorithm, the better it is able to
discriminate against people in that
group. So if we are
here and the algorithm doesn't have any
information at all about the protected
group, it can't discriminate against
those people. So the ratio
of success between the individual groups
is one. And this is actually
great because it means if we can build
an algorithm that doesn't have any idea
about these protected attributes, we can
eliminate all the discrimination that's
in the system.
On the other hand, if by some
accident the algorithm gets
full information and these attributes,
it can discriminate just as
as well as a human against people in
either group. So that means if we give
too much information to our algorithm, we
will have the same problem in the hiring
process as before.
So we will also have to have
discrimination against people, not by
human systems, but by machine
and say.
And now you say probably,
OK, this is stupid.
Why would we give information
about this protected group to the
algorithm? And of course,
the answer is we don't normally.
But the problem with big data and
with like having a lot of different data
types and different data sources on our
hand is that even if we don't give that
information to the algorithm explicitly
and there is some amount of information
about the attribute that leaks through
with all the other information that we
provide.
And this is basically the essence of
the dilemma of having too much data on
our hands, because it's always very
hard to to keep
information about sensitive things
leaking into our data set.
And, of course, is like a purely
theoretical,
pure theoretical formulation now.
But I actually
try to validate this by using publicly
available data. So what I did was
to get GitHub user data,
which is on which we
can obtain through an API and which gives
us information about all the people on
GitHub.
So, um.
First, we need, of course,
information about the protected attribute
of the group and this case, we choose the
gender.
So either man or woman, and
to do that, we have to manually
classify and
the people that we put into this study.
So what I did was just to
look at the profile pictures and get up
about 5000 and like classify the people
into like like man and woman.
So this gave us the training data for
this kind of simulation.
And what I did then was to retrieve
additional information about each user.
So, for example, the number of projects
on the website, the number of stargazers,
the number of followers, etc., etc.
So everything I could get my hands on and
then I would use that data to make
a prediction about the gender of the user
just based on the information that I put
into the system.
So and I want to say again, this is only
a proof of concept, so I used a very
small data set.
I didn't do any optimization.
I just wanted to see how easy it is
actually to get this kind of
contamination into our
algorithm.
So first,
when I used only very basic things
like, for example, the number of
stargazers or followers of
each user, I couldn't get any
prediction about the gender of the
person. And I mean, this is already great
because if
your colleague says, oh, you know, woman,
they are not good programmers, you can
just now show in this data and then you
basically can disproven.
And that's because it's not possible to
predict gender from the publicly
available data.
And but for me, it was, of course, a bit
disappointing because I wanted to prove
that we can discriminate against these
people. So I need to get more data.
And luckily, GitHub helps us out of
that by providing a
events API, which contains
the full event stream of any action
or almost any action that a given user
has done on the side. So every time that
you open a pull request or you make a
comment or you do something else on
GitHub and there's an event created for
that, and you can download all the public
events on the site through this website
here and like process them and use them
for, for example, data analysis.
And this is what I did. So for all the
users that I had my sample, I downloaded
this event data and like tried to
get some more information that I could
use to discriminate the gender of the
people.
And, for example, the data that that
I looked at.
Here we see the event
frequency so averaged over all the
events as a function of the hour.
And you can see that now there seem
to be some significant differences
between men and women in our data set.
So that's something that the algorithm
could use to make a prediction about the
gender and likewise, in
the type of events that
we have in our data set.
There are also differences in
the frequency of individual event
types. So that's also something that the
algorithm can use to make a decision
about gender.
Now, for the last thing, I went a bit
crazy and I did something that
you normally do in spam detection
that is taking like to commit messages of
individual contributors
and just like inputting them into like a
support vector classifier.
And that basically looks
at the frequencies of individual words
and each commit and tries to find a
difference in the text
between man and woman.
And this already gave me some
good, like good fidelity of
predicting the gender and combining it
with the other information that I had.
I could in fact, achieve 50
percent better chance
of like predicting the gender than by
just guessing. So this is
not very impressive and
we can probably do much better.
But again, this was only like a proof of
concept to see how easy it actually is
to get get this kind of information
leaking into the system.
And so this basically means that if we
can make a predictor for the gender of
the person, GitHub and the algorithm
that we used to like make the decision
about the hiring process can also
generate this kind of information.
If we give him if you give it this data
and use it against the
people.
So the take away from this is that the
algorithm will readily
learn discrimination from us if we
provide them with the right training
data and also information
leakage so that getting information
about protected attributes in our data
sets that we don't want to have, there is
actually a pretty easy and can happen
if we are not careful.
How can we fix this?
Well, it's actually harder than you might
think because often we don't
even have the information about the
protected attributes in our data sets
because we don't want to to take the data
from the user. I mean, imagine if
you would apply for a job and your
employer or potential employer would ask
you for information about
your sexual preferences, your gender,
your ethnicity and everything.
And plenty of other things probably
wouldn't go down so well.
But this is actually the kind of
information that we would need in order
to see if there is some disparate impact
in our data. Because if you don't have
that attribute information, we cannot
like, um, calculate
any fidelity or any like like measure
of the discrimination that is in the
process. And this is what is
so dangerous about this, because our
algorithm can discriminate against people
without us even noticing.
OK, this
is already the first case
study that I wanted to show, and we
have seen that,
that getting information into our
dataset that we shouldn't have is
pretty bad.
And like the
worst kind of information leakage that
you can imagine is if you can identify
someone from the data that you have
obtained for them earlier.
And I mean, again, if
we ask Google about its opinion
on privacy, it's the
picture is rather bleak.
And it seems
that many people have already getting
used to the idea that we are in the
privacy area right now.
And so with the second experiment here, I
want to show how easy it is actually
to anonymize
and given user data even without wanting
it.
And what is actually the
anonymization where the anonymization
means that we have some
information recorded about an individual
or person and we use that information
to predict the identity of that
individual in another data set.
So is kind of like your data is following
you around, even if you like, for
example, change the devices which you're
working on, you change your
your user accounts.
The system is still able to identify
you just by using the data that you have,
like put into the system earlier or that
it was like measured of you earlier.
And the anonymization becomes an
increasing risk as the data sets
that we can use about individual users
get bigger and bigger, actually.
So I hope it's working.
OK, now let's
have a look at the math here, though,
is the anonymization is a pretty big
subject. The math is rather fun, I assure
you so.
And you maybe have played
this game with some of your friends where
you just think about some famous person
and the other and your friend
has to guess who that is by just asking
you a series of yes or no question.
And this actually works pretty
efficiently so that after like maybe 10
or 20 question questions, you can
know exactly which person your
friend was thinking of.
And this works so well,
because if we have like
several several
buckets that we can, um,
that are either true or false for a given
user, we can
create a unique fingerprint for the
user in our system.
And if you look at the probability
of like having a collision, so like
having two users that have exactly
the same true false values,
this is getting increasingly unlikely
the more buckets or the more different
types of information we can put into
our system.
And so, like the exact number or the
exact probability for finding a
correlation between users is depending
on the actual distribution of
the information in the buckets.
So if you have a uniform distribution, we
can calculate that number.
And as you can see, it decreases
exponentially, which is why this game
that I talked about earlier is working so
well since, for example, if you
assume that you have like one million
famous people that you can think of, then
it would probably be sufficient to have
like thirty two bits of information to
uniquely identify them all.
And we can imagine that with
big data, we have much we have many more
buckets that we can actually use so we
can identify not only a few million
people, but easily a few billion
different people using that technique.
And most real world data sets
are, of course, not uniformly distributed
so that we have more the case that
that many users are in the same bucket.
So, for example, many there are many
people that like the same kind of music.
And so they are all like have the same
information or the same attribute.
And using that attribute
to the anonymized to the users wouldn't
give us much to wouldn't do as much good
because it wouldn't help us to like
narrow down the number of users in our
system. But there are also many
attributes that are pretty unique to each
one of us. For example, the place that we
are living or the combination of that
place with the work, where we going.
So having a few of those quite
unique data points for each user is
usually already enough to the anonymize
us with a very high fidelity.
And again, I wanted to see if this is
actually working in practice, so
what I did was to get the data set
in this case from Microsoft Research
Asia, which contains GPS
data about of about
200 people that track their whole
activity for sometimes
several years, sometimes several months,
and like use
the data to create a movement profile, so
to say.
So I also have an animated version of
that that you can see here,
the different trajectories
of individual users.
I don't know if anyone recognizes the
city.
Now it's Beijing, actually,
and if you're wondering what
this square is, so
I looked at Google Maps and it seems to
be the university, so I guess
it's like another field of study.
Whenever you need, like some guinea pigs
to take data for you, you go and ask
students. So.
OK, so this is a pretty rich data set
we have like for in some cases, hundreds
of thousands of data points per
individual and I wanted to see how
easy it would be with this data to
actually anonymize users.
So what I did was to first look
at individual trajectories.
So here we have like the GPS traces
of the individual's color coded and
then to apply a very simple
grid. So like in this
case, a four by four grid and
just measure, um, the frequency
with which a given individual
has some data and this on
this on the square.
So doing this for
the two hundred people gives me something
like this. So this is four to four by
four grid. And you can see the colors
represent the number of times a given
person has been in a given square.
So why it would mean that the person has
been here very often black would mean the
person has never been in this given
square.
And you can already see that
reflect the 60 examples that I show you
here, the many of them, they
seem to be quite unique, for example,
this one and this one.
So it should be possible to like
kind of make a fingerprint for a given
user using that data.
And of course, if we need more resolution
to like, for example, disambiguate uses
like this here where we have like the
same data, more or less, and we can't,
like, decide which user we have,
we can just increase the resolution, for
example, to eight by eight or to 16
by 16. So here.
And now
coming back to our buckets, if you would,
measure the distribution of the
attributes that we have here, we
can get an idea how good our choice
is, actually. And you can see
the choice that we have make made is
actually pretty bad, because in the first
bucket of the bucket, with the most
datapoints, we have about like 10 to
the six or 1000000 points.
But the interesting part of this curve,
which is, by the way, logarithmic
is here. So in this, like,
long tail of the distribution where
we have only sometimes
one or sometimes a couple of individual
persons in that given bucket.
So if we can get some information in
these buckets, it's easy to use that
to the anonymise our users.
OK, and how do we do that?
Again, we use a very simple measure and
we just take the fingerprint
of one user or one trace
and multiply it with the fingerprint of
another trace, but picks up the pixel,
which gives us then the value on the
right. And then we take these individual
values here and send them up.
And this gives us kind of a score of how
similar to use as our two trajectories
are.
So doing this,
we can take 75 percent
of our data as a training set.
So we just like teach our algorithm to
like, recognize individual users and
then we can use the remaining 25 percent
to test how good our algorithm is at
recognizing the users now
and then we look at the average
probability of identification and
also of the rank that the user has
in this and this prediction.
And this is shown here.
So what a show is the, um,
the probability of,
like, finding the right user
within, for example, the first two,
the first for the first six users
that have the highest scores for a given
chase. And you can see that
for even 16,
16 squares. So the four by four grid that
I showed you in the beginning, the ID
rate is already 20 percent here.
So we can identify uniquely
one fifth of our user by just using
16 data points.
And the more data points we use,
actually, the more the better we can
identify the users in our data set.
And with 1024
individual data points, which would be
quite easy to get in a real
world setting, we can uniquely
identify almost 30 percent of the users.
And again, I want to state that this is
just a proof of concept.
And so there has been no optimization
done and no like fine tuning of
parameters or anything.
And we can also use that technique to not
only identify single users, but also to
find similarities between users.
So this could be interesting, for
example, to see who is
related to whom and who are you
visiting, who are your friends may be.
And this is what I did here.
So I used the same metric as before.
And I just told the system to give me the
users that are most similar to each
other. And you can see here in
green the trajectories of one user
and then read to the trajectories
of the other users.
And the areas that are yellow are
actually where the two of them coincide.
You can see there are some hits of the
system which don't seem too good, but
there are lots there.
There's some hits also where you can see
like a really big agreement
between the two datasets.
And I mean, I don't know
who was taking this data because it's
anonymized, but I would guess in this
case that it's either like a taxi driver
or maybe a bus driver, because
you can see that we cover almost the
whole Beijing area with these two traces.
And so this technique makes it really,
really easy to like, identify
users and also like find out
who they are related and who are which
other users are similar to them.
And we can, of course, like improve
the identification rate of the
system by, for example, taking
into account not only the spatial
information, but also like the temporal
information, for example, the day night
cycle, which you see here in the
background. So here the green groups
have been taking at night and the Red
Cross has been taking at the during day.
So like you have the data, for example,
going to work in the morning and coming
back in the evening can
then be used to increase the prediction,
fidelity for identifying a given
user.
And of course, we could also, like change
the choice of our buckets and
like change the way we do the
fingerprinting in order to increase the
fidelity of the algorithm.
So there's plenty of room for
optimization. And as I said, this is only
like a proof of principle.
But there are other like similar
works in the literature which show that
with even a very simple method, you can
achieve quite good edification rates
in such a data set.
Now, to summarize this, this
means that the more data we have about
a given entity, a person, the more
difficult it is actually to keep
algorithms from directly learning and
using the identity of that object
for a prediction instead of an attribute.
That means, as I said before, that the
data which a given user are given
person generates follows
him or her around the whole life.
So even if you like, would change all
of your smartphones, all of your devices,
some parts of your behavior would
probably stay the same.
And this could be used to
identify you later in the process, again,
with a pretty high fidelity.
So that's one of the biggest risk of big
data for me, because it's like
very easy if we not avoid
it to, like, destroy
the privacy of our users.
OK, yeah, thanks.
So what can we do about this?
I don't have all the answers, of course,
but I have a few ideas and I mean,
there are lots of people
working on like
a political and societal and
technological solutions for this.
So here I just want to give a brief
overview of things that can
be important in order to avoid these
two scenarios that I have shown here.
So to
the group of people that we probably have
to to educate the most urgently
about this is, of course, data
scientists. So the people that actually
work with the data and create these
algorithms, because today
and there is in Germany, for example,
you need like a three year apprenticeship
in order to cheesecake, but there's
nothing comparable in order to like an
algorithm Maciste and like to develop
these kind of algorithms that have a
large influence on our daily lives.
So they probably should be
a better curriculum in universities and
even in schools, maybe to educate
people not only about the possibilities
of data analysis and like about
scraping even like the last few percent
of fidelity from a given algorithm,
but also about like the risks and the
danger of using these kinds of
technologies, especially when other
people are involved.
And another thing that we should be
careful with is indulging
data without actually needing it.
So to data
like one of the most popular approaches
and big data is just to take everything
that you can get.
So all the data that we can get our hands
on to give it to the algorithm and to let
it decide how it uses
it. And this is good not only
because it increases the fidelity of
our predictions, but as I explained
earlier, it can be also very dangerous
because maybe the algorithm can learn
things which it isn't supposed to learn.
And also we should really be
more careful with the data that we give
in to these systems.
And of course, the other things that we
can do, we can try to do to remove
discrimination and disparate impact, and
there's also like a lot of academic
work giving techniques
and methods that we can use for or for
doing this. But here, the problem again,
is that most people that actually work in
the fields where these algorithms are put
into practice either don't know
about these things, are not interested in
those. So I think here we have a big
potential for like improving
the education of data scientists and data
analysts as citizens.
We can also do something, of course.
So the first thing is to not blindly
trust the decisions made by algorithms.
So if most people have kind
of a bias to think that a decision made
by a computer, by algorithm is maybe more
fair than a decision made by a human.
And I think this is something we have to
get rid of because algorithms as a show
can be just as discriminating against
people as humans can.
So, um, and if we can't,
like, question their
decisions, we can at least test them
and see if there's actually
discrimination in the system.
And now this sounds pretty easy, but it's
actually very hard because the algorithms
are mostly like in the hands of big
organizations or corporations and are,
of course, like a closely guarded trade
secrets in most times.
And this means that we have to
use techniques such as reverse
engineering in order to to like find
out how the internals of
the algorithm might work.
And I have to say, I'm a bit pessimistic
about this because, um, whereas
where the companies or the organizations
could use like like huge buckets and huge
amounts of data to train these
algorithms, the amount of data that we
can use for reverse engineering then
is minuscule, is very small in
comparison.
So it's really not very likely that we
would be able to make a good decision
based on these kinds of techniques.
And of course, we can also one thing
that we can do is to fight back with
data. So by collecting
data about decisions that are made
of about us from the algorithms and
by centralizing that we can like
create a lot of opportunities for other
researchers and other people to analyze
these data sets and to like find
discrimination and other things in them.
And so I would encourage you to, um,
if you like, are reluctant to like
like give away your data.
I can, of course, understand it, but, um,
in some cases, it's really the only way
to make sure that someone can actually
work with the data and detect
also like like find injustices
that are caused by it.
So we have to really think about
differently of giving away our data and
like like also creating data and machine
learning against machine learning.
So.
As a society, we can, of course,
create better regulations for algorithm,
and this is actually something that has
been done.
I mean, the beginning of the year,
our minister of justice was demanding of
Facebook to to open
up their algorithm. And this
was much ridiculed at the time.
But I think it actually has some merit,
because if we can't
understand how corporations
or companies are using algorithms, we
can't know if they're discriminating
against certain people or if they're
treating us fairly.
So having as an auditing
system in place, that allows at least
a group of people to have a look at this
algorithm and to see how they're working
would be a first step in the direction of
making these things more transparent.
And of course, making access to the data
more easy and a safe way is also
important to be able to to detect
any problems that we have with it.
And finally, of course, I
mean, this is maybe already too late, but
we should do our best to impede, like,
the creation of so-called data
monopolies, because if one organization
or one sector has all the data in its
hands, we have already lost.
Because even if you have the same
algorithms, the same technologies
are at our hands.
Most of the value and data analysis isn't
the amount of the data that we can can
have. So if there's an adversary or
like an organization that has like orders
of magnitude more data to work with than
we, it's really unlikely that we will
be able to, like, compete
with that adversary on the same scale.
So.
As a final word,
I would say that algorithms
are probably a lot like children,
so they're very smart and they're really
eager to learn things.
And we, as the
data analyst, as the programmers, we have
to teach them to behave in the right way
and we should try to raise them to be
responsible adults.
OK, so thanks.
His father didn't like being poor.
We do have a few minutes left
for Q&A, I would like to ask you
to queue up at the microphones at the CIA
if you're watching at home.
We also
have a human computer interface
to relay questions to us.
I'd say we begin with that.
Do you have a question for us?
Yes.
Rootie is asking, what discrimination
number would you guess for discrimination
from politicians over people's choice
in one or several countries?
Um, politicians
about people's choice.
You mean can you
be a bit more precise on that?
I think it's difficult to.
We'll get back to that question.
We have one question in song
number two, please.
Thank you for your talk.
Does it make any sense or is there any
hope that I am as an individual
can fake my my
data patterns or
can I disturb?
The pattern recognition
in a sensible way, in a sensitive.
Yeah, um, yes, I think you surely
can. That's the question is only if this
would be effective to, for example,
protect you against the anonymization,
because as I said, like, taking
90 percent of your data can be useless
if 10 percent of your data points
are in packets or like in
attributes that are unique or almost
unique to your person.
So if you want this matter to
be effective, I think you would have to
be really convincing.
And, um, I mean, I haven't
had a look at the very big data set, so
I really can't give a quantitative
answer. But I'm rather pessimistic about
this approach.
I have to say, OK, we
do have a few more questions.
I would ask the people in the room, if
you have to change rooms right now,
please do so in a quiet manner
so we can do the Q&A
without yelling.
We do have another question in the IOC
and after that, it's number four IOC,
please.
Atomic engineer is asking
if a human is generally able
to create an algorithm which is not
discriminating.
And he's doing an analogy to
random numbers where a human cannot
really create truly random
numbers because she or he or she would
always have a preference.
Hmm.
Yeah, that's a very interesting question.
Um, I mean, it really
comes down to the algorithm
having the information about a protected
class or not having it so it doesn't
have the information.
It can't be discriminating
by definition, because it can
only randomly guess if a person belongs
to a given group or not.
So in that sense, algorithm can be
perfectly unbiased, but only if they
don't have any information that that
gives away the protected status
of an object or person that they're
making a decision about.
So it's definitely possible that.
OK, the next question by number four,
please.
Thank you for your talk.
You say that algorithms discriminate in
the same way that humans can,
but I wonder if the real challenges that
algorithms discriminate in a slightly
different way than humans do.
And for example, you gave the example
that we can person we can
identify gender or other markers
from the data set.
Yeah, but what if these attributes
that identify that correlate with gender,
class, race, etc.
also correlate with other positive
attributes, such as
the study that you're more efficient
work when you live closer to your
the side of your employer.
But if you have a very segregated
society, that means that those who are
richer are also then classified
as more efficient workers and
when in the scoring of potential
employees.
And so the question is,
if such a thing occurs,
it's not just that discrimination can
can be an unintended outcome,
but also if the company wants to
discriminate, you cannot prove it because
you say we just hired the most qualified
candidate, but in fact, you just hired
certain kinds of people.
Yes. Yes. I mean, that's exactly the the
argument about discrimination,
because if
you don't have the information about
how many people of a given
class of a given protected status
applied, for example, for a given job,
you can't figure out if there is any
discrimination in the process.
And so that means that you
have somehow to get that information into
the system in order to make an audit and
actually see if there's some unfair
bias in there.
And I mean, the other question I don't
know if I understood it correctly is if
you
if you can in fear information
about the gender from other things or
and it's I mean, this is certainly the
case because as I said in the talk,
like many things, like, for example, the
neighborhood that you live in, as you
said, we give away information about the
protected attributes as well.
All right. We have a few more questions I
would I would ask you to keep in short,
please, the microphone number five
in the back
hurts.
And often her statement is that
the more data you actually collect, the
less you can actually do with it, because
it's just too much.
Is there any scenario where this
statement makes any sense?
Yeah, definitely is.
I mean, given an algorithm,
more data to train with is not always a
good thing.
It's pretty easy to do overtrain
algorithms, not so
to give it to make a model that is like
perfectly fitting the data that you give
it, but that has very little predictive
power for new data that you see.
But in general, increasing
the number of days of data points
is always like improving the quality
of the model if the data that you have
is from the same model as
well. So it could also happen that the
data that you have is not homogeneous, so
that one part of the data
is fitting well with one model, but the
other part of the data is fitting well
with another one. So in that case, it
might be difficult training a large
amount of data on a single model, but
it depends on the individual case, I
would say. So it's really not easy to
answer in that sense.
Thank you. We have time for two more
short questions.
I would ask one question from
the again.
Yes, Anayansi Lucas' asking,
isn't the black box nature of the machine
learning algorithms one of the biggest
problems can be
solved by better visualization or
understanding and what it really is
doing?
Yeah, for me, having algorithms that
are not open to scrutiny and that we
can't understand is one of the biggest
problems, of course. And, uh, um,
visualizing data can help, of course.
But as I said briefly in the talk,
since the space of possible parameters
in the space space of possible data
points is so enormous,
even for very small and
learning problems, that it's really
difficult to produce a given
visualization that would you that would
give you a high confidence and
a good information about, for example,
discrimination in the data set.
So it can certainly help.
But, uh, I think it's not a
perfect answer either.
OK, we have time for one more question.
Microphone number one, please.
Thank you.
In the beginning, you displayed the
green, yellow and red,
somehow agreed to give me more damaging.
The example you made about the green was
about some kind of algorithm that gave
to you information.
Don't you think that at the time of
exposure influence,
how much is damaging?
Because if I get to influence the to here
is worse than just two days.
Mm hmm. Can you say that again for the
time of exposure, with time of
exposure to an algorithm to influence
your behavior has to be considered as
a factor to understand
if it's true.
Yeah, that's a very important point.
I mean, I also had like an experiment and
where I look at the interaction of the
algorithms with a person that he is like,
for example, showing articles, too.
And, um, this is like a topic of itself,
I would say. So there's definitely very
rich interaction. The results are not
captured by most models.
So like the algorithm influencing the
behavior of the person and then that
again influencing the actions of the
person and influencing the machine
learning of the further data.
So there's definitely some feedback in
the system. Absolutely.
Yeah.
OK, that's all the time we have.
Thanks again for the great talk.