Dear viewer, these subtitles were generated by a machine via the service YouTube and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/2512 Thanks!
[Music]
okay so um our next speakers will be uh
cardor uh lenus and
Dominique uh and they are going to be
talking about disclosure hack and back
how to mess things up in various ways
please give a big Applause to our
[Applause]
speakers
yeah thanks a lot for coming um we uh
basically thought we want to talk a
little bit about the adventures we've
had in the past year uh with a specific
focus on the past half year in incident
response in handling incidents causing
incidents uh by uh disclosure and we
basically talk about uh our disclosure
Adventures first so many of you may know
that the CCC regularly will discloses
vulnerabilities to companies
uh we also have an email address where
we help people doing so that email
address will play a role in a in a bit
here um so kle my good friend uh
reported a lot of incidents uh in in bul
basically was like he had like this um
like a spreadsheet to keep track of all
the all the disclosures and we had to
you know look into it together and make
sure that you know which state is it in
and do you have everything and
eventually we only made it a small blog
post of 6.4 million data records in over
50 leaks that we disclosed to the to
those companies now you would think
that's a process you could scale right
you build a spreadsheet like have we
sent the email have you received the
response right is it fixed but once this
is at 50 and you're dealing with 50
different organizations it gets a bit
more complicated still disclosure is a
simple Pro in theory right you find a
vulnerability you tell the people that
are responsible for the vulnerability
they fix
it if you look at this as a in terms of
like a you know call Flow you know you
do the research report the reportage to
the company just in case they forget to
uh alert the CT or the data Protection
Officer you just do it for them as well
right just so they you know um they
usually uh send you an act then they uh
perform their duties and notify the DPA
they fix the issue they send you like a
little thank you note uh and they notify
the customers you can of course imagine
which parts they sometimes like to
forget about so maybe we then do a
little publication to take care of the
customer notification for them on top of
that so now with 50 of these in the past
years or in the past year um kle has a
lot of Little Adventures to talk about
and please give him a big round of
[Applause]
applause so yeah let's start with one of
the cases that went quite well I would
say so there was this Spanish media Camp
company that owns for example The Daily
Newspaper uh lpis and they had a problem
that many companies had before they
forgot about this dogit folder which has
a config file and here this config file
included credentials for the gitlab um
that was a very simple CA with a large
effect because uh in this account I had
access to 478 projects uh which was I
don't know probably all gitlab projects
of this company including some
infrastructure as a code
stuff um so yeah we had a vulnerability
and then we tried to report it and that
was quite hard in the beginning I
reported a elastic search that leaked uh
couple of months before to the same
company they never replied um this time
this issue was more severe so I I tried
different channels uh so first of course
I wrote this email to to the company but
I also notified the third but still
there was no response so I checked uh
for uh people working at this company in
the it Security Department on LinkedIn
LinkedIn is great most of my LinkedIn
friends had it security issues
um but this time um we used Facebook to
reach out to someone who was in it
security and finally we managed to get
this uh leak fixed they um thank me I
yeah so until then this process was
perfect more or less and um then they
asked for a call
I sometimes say yeah okay why not uh
because those people work right friendly
um I told them when I would be available
and then they were quiet so in the end
there was a little drop in the
performance but it was
okay something different the United
Nations HCR um they are not responsible
and they do not care when it comes to
data of other people so there is this
Services advisor for different countries
for example for Iraq and it HS
information for refugees that need help
in the in those countries um this
service advisor leaked accounts of about
900 employees of I don't know dozens of
Aid organizations that are active in
different parts of this country and and
this time it was very interesting
because the United Nations have assert
so there's a place where I can go to to
report issues um first they said they
are not the the responsible entity for
this because the unhcr is different um
they are not covered by the CT later I
received a thank you for submitting this
issue but the vulnerability was not
fixed um so I need need needed to reach
out to them again and again until um
even the last Gmail account uh well it
was in fact deleted I I don't know they
they deleted some data and then um the
vulnerability couldn't be abused anymore
also they did not forget to mention the
United Nations Hall of Fame program
because as some other entities as well
they have a Hall of Fame where people uh
are mentioned after successfully
reporting a it security issue but uh
here they only mentioned that the unhcr
does not fall onto under the Hall of
Fame program um maybe next
time switch to it security made in
Germany
um
let's
hello yeah um let's switch to it
security made in Germany the TN they
they even certify you when you're secure
um as other companies before they had
this issue with n and dogit folders or
files happens but they chose to ignore
the report so I needed to reach out to
them I I think via phone um they ignored
the email um we had some discussions um
then they told me this is not our IP
space we do not use this software
anymore uh it's not important to us we
cannot do nothing um however um it's IP
space of the T Nord they still decided
to not fix this issue and I it this
morning and um yeah if you like go and
check for data leaks in the T N domain
space you will find database credentials
and source code you will find a Google
service account that can still well
access some services and um the database
is not directly accessible but um they
made it easy there's at Miner and PHP
admin to allow access to databases and
there you will find uh hash passwords of
tau.com accounts um I told them about
this issue but they said they have
changed passwords it's not a problem
yeah um there was another interesting
disclosure December last year um the US
military was using those devices that
you can see on the slide here they used
it to check people in Afghanistan and
Iraq against watch lists um they they
wanted to catch terrorists I guess and
we found those devices on eBay and in
one of those devices is there was a
biometric database of about um two
2.5000 people um we had up to 10
fingerprints per PE per person Iris
scans and a photo from the face then
there were details like name hate
sometimes weight eye color whatever
um now half a year later a couple of
weeks ago they finally reached out to us
um hello this is Agent Smith we would
like to have our devices back
um also they like to meet they would
even come to my
hometown and they they have questions
um hello agent Smith I don't know if
you're listening today um I received
your email about 10 days before Camp I
was quite busy because of Camp um we
will mention to reply soon
um in the meantime if you would like to
have such a device um they are still on
eBay um however um it could be that some
people will visit you if you do so or
the next time you travel to the US it
might be challenging to enter the
country
small fun fact I also learned by the US
National Archives and Records
Administration that I'm finally owner of
the chaos Computer
[Applause]
Club um for every single data leak this
organization has a record and I ask for
the record because they have Freedom of
Information loss over there as well they
did not reply yet but I'm really looking
forward to read those documents that are
somewhere um yeah that's it about the US
Department of Defense um we will come to
one last very
interesting uh shoe shop on the internet
we are introducing the crypto King uh
Philip Pline um
crypto King is a name this person shows
himself for himself so he he is the
crypto King and in this shoe shop you
can buy shiny shoes they are a bit
expensive but I mean they are shiny you
can pay with uh Dogecoin or 24 well in
total diff uh 24 different crypto
currencies they they have even crypto
King and crypto Queen watches um the
future is
now so yeah um from a technical
perspective this issue was quite boring
again so we found a leak somewhere and
there was this FTP server so first
mistake storing customiz data on the fdp
server um then they lost credentials to
the server to us and to everyone else on
the
internet then they were ignoring all
vulnerability reports and this time we
were creative we we reached out to the
third in Switzerland because this
company is based in Lugano a very nice
city uh re reached out to two dpas
because uh they have uh stores in the
offline world as well uh including B
Berlin K first and Dam so I reached out
to the Berlin DPA as well as to the
Swiss DPA um I called the hotline
hotline um I reached out to some people
via LinkedIn including the I don't know
not the CEO but CIO possibly um they
ignored us um however we decided not to
travel to Lugano and we also did not
send a
fox
um I was checking from time to time if
this vulnerability was eventually fixed
uh after I don't know two years maybe or
one and a half year the server vanished
from the internet so this FTP server is
just not available anymore at least not
under this IP address um so if you
bought something offline or online at
this shop it might be that someone else
has this information as well and
I um to sum up this this first part of
the presentation some lessons learned so
finding bugs is still easy there are
lots of very simple data leaks it's
boring to look into those data leaks
from a technical perspective but it
becomes quite interesting if you check
for the different
um behaviors companies have when you
approach them about a data leak so
reporting bugs is the actually
interesting thing here and sometimes you
find very motivated people that are very
happy that you approach them and that's
really nice to to work with those people
uh um to make them understand what
happened why that happened and how to
prevent this mistake in the
future
um for companies um that could have such
issues please assume good intentions
when people approach you um they want to
help sometimes by accident they do a
MySQL dump and not just a select count
of rows or something like that um please
be prepared so make it easy for us to to
approach you and have a plan what
happens afterwards when you when we
approached you and um please keep us in
the loop and also give some kind of
acknowledgement because it happens quite
often that a company reads your email
fixes the issue but never says said
thank
you
um two or three hours ago there was this
talk by lilth on the same stage here um
doing such stuff is very interesting
from a direct action activist experience
as well um because uh companies fail to
inform affective people quite often
public disclosure is needed however when
doing it security research at least in
Germany there's always this hacker
paragraph behind you that might hurt you
um so um yeah they suck please uh remov
them um sometimes the responsible
disclosure V is not not the best way if
you want to maximize impact but it's not
always the most effective way to to do
to change things and sometimes yeah [ __ ]
posting is fun if company do mistakes
there are lots of golden
opportunities so that was our uh short
dive into a couple of funny uh stories
from vulnerability reporting and uh
disclosure
now the second part of this talk is what
we call um aeka
infrastructure
exploration exploration so we uh we got
to know a little um ransomware gang uh
initial exess broker and uh we were uh
actually surprised uh how they mess up
and of course to those guys you don't
disclose you just hand in a talk at Camp
so welcome to our talk um in Soviet
Russia incident response Hecks you and
uh
so
what it all started with an email uh
that went like this DCCC no spam no joke
my employer was hacked I was on The Blue
Team the hacker was mediocre left a few
traces we were able to monitor him a bit
no data was stolen from us the attacker
attempted cab roasting to take over my
beloved domain
controller I found an SSH key and IP and
looked it up thanks to Smart cards and
Labs worse could be prevented so they
were uh they not they not only um
survived the incident without uh much uh
compromise they also had this little SSH
key that they were so friendly to share
with us now with this SSH key in my hand
I thought well let's call Matias and
Dominic and see what we can do with this
little SSH key and I think Dominic will
tell you all about
it yeah so this was a pretty interesting
opportunity um not finding
vulnerabilities in the infrastructure of
like regular organizations but rather in
some special organizations so we asked
around and we found um our friend
heckerman here who wanted to look into
this
further so heckerman did a bit of um Lou
all right can you turn the volume
up heckerman looked at uh what happened
and we already knew from the first email
okay the attackers compromised the
victim and uh what they did there uh
they created a scheduled task for uh
persistence they dropped two binaries t
XA and SSH XA and they had an SSH
private key um now with this schedule
task which was quite obious cated the
tour invocation the SSH invocation was
hidden amongst a lot of different
commands to make it harder to spot but
in the end we had a reverse shell
connecting back to the
infrastructure on the AA infrastructure
you could find data not only from one
victim but from a a lot of victims there
were plenty of SSH keys and then you
find the usual stuff from ad compromises
so from Windows uh environment
compromises you find Caro roasting
tickets car roast information you find
complete active directory Dums you find
eler stems and you find the tooling that
the group uses to do their
exploitation now our friend he back man
of course first made a backup of all
that information just so that it doesn't
go get lost right better safe than sorry
so make a backup first and then say see
what you can do with it now
interestingly the first reverse shell
worked with a low privilege user um
which connects to the server a Hackman
did a bit of poking around and realized
the same SSH key could also be used to
connect to the server is
root so it's not only the in it's a
victims that [ __ ] up their
configurations also the ransom Crews
right they all also make mistakes server
hardening is hard and sometimes mistakes
happen so with root access of course
there was a lot more exploration that
could be done um but first Hackman asked
us hey don't you want to help me in the
notification process because that's is a
lot of work as you have heard from GLE
here so of course we stepped in and and
helped out and notified in uh 30 victims
uh that they've been breached
now if you ever wondered what does a C2
server of some initial access broker
ransomware crew game look like it looks
like this this is the root folder so
obviously uh the crew was also working
as
root and they dropped everything there
right they dropped all the data Dums in
there they had all of their scripts in
there they cloned G repositories they
had some uh automation tools bash
scripts everything in the ru folder uh
so this looks like the home folder of a
very unorganized penetration tester
right
there the crew wasn't really
sophisticated right I mean they they
used a lot of Open Source tools you can
see a list of tools that was found on
the server on the right side many things
if you're working in in red teaming or
penetration test testing you might also
have
used uh they also use the same tactics
that that a lot of the red teams use I
mean makes sense red teams want to
emulate real attackers after all so
apparently that seems to be working in
this case uh they try to stay away from
the victim boxes as much as possible
tried to do a lot of work over um over
sock proxies or or over VPN connections
if they managed to gain VPN
access uh and then did exploration of
the victim networks in The Bash history
of course we also have a backup of the
bash history you could see um also not
only how they exploit the
vulnerabilities but also how they find
the victims of course they use shun and
they have a list of of um commands they
use to find new victims to
attack um some more site information so
apparently sometimes keyboard layouts
are also hard and uh if you uh forget to
switch from killic keyboard layout uh to
us layout then you might type in the
wrong command and that can be discovered
in The Bash history as well also um not
only victims leak API Keys Al in The
Bash history you sometimes also find API
keys so we now have a nice Showdown API
key from this initial access broker we
don't use of course we don't use this we
just have it as a
backup so one thought we we had um while
looking over this is okay they probably
not only have one C2 server that
probably have multiple maybe they just
clone the machine and reuse the same
configuration everywhere maybe we can
just scan the internet and find the same
SSH host key to find more servers but um
they did after all use individual keys
per server so that didn't work out in
the
end so back to the disclosure this is
this is a disclosure talk uh we had to
notify 30
victims now what you want if you're uh
if you have a security incident it just
happens ideally you want to detect it
yourself you have some infrastructure in
place maybe a seam maybe a sock who
knows you want to evict the attacker and
ideally you want to do it before you
need to like disclose to people that you
lost all their Data before you need to
involve law enforcement or whatever what
you don't want to happen is for CCC to
call you and tell you hey we realize you
have a breach but that's exactly what
happened um so we uh tried to figure out
who to contact which wasn't always easy
I mean we had lot of AD dumps but it's
not necessarily always straightforward
to figure out which ad dump belongs to
which organization and then the next
stum is next step is even more hard uh
figuring out who to contact in that
organization um fortunately we have
someone on our side who has a lot of
experience with disclosing stuff to
people and we could reuse the same
tactics so uh I'm not going to repeat
the flowchart but it's basically the
same thing uh we mostly try to contact
security staff or um or uh IT staff in
general but sometimes we also reached
out to the company
CEOs uh we always tried to go via email
first um we provided information about
the breach and a list of indicators of
compromise so that the um companies
could take action overall um we
contacted 28 organization and in um in
addition we also um gave the data on did
we give the data no we also just uh
informed the BSI and Sir B so the German
authorities and the national SS from uh
the countries where we could figure them
out some of the organizations actually
responded really quickly I think the the
fastest one uh uh wrote you an email two
hours after we sent out the initial
email they were super grateful like uh
they they weren't aware of the the
breach and uh were uh thank the
CCC uh told them told us how great they
find this and and promise to donate
money I don't know if that happened in
the
end and we tried to get in a bit of an
information exchange for them so to to
ask them hey if you find other stuff in
your network particularly if you find
new IP addresses or new SSH Keys we
would be very interested interested in
that now if you compare the reaction um
from this incident response disclosure
to the vulnerability disclosure it was
pretty similar we had a couple of um
companies that really had the incident
under control they were already aware
they already evicted the attackers
themselves we had a couple of
organizations which were grateful which
had more questions um and and wanted uh
Our advice and then we had a lot of
organizations who didn't respond at all
at first so apparently uh companies are
not only not interested in
vulnerabilities they're are also not
interested in them being breached
apparently
um this felt a little bit um
demotivating at first but of course we
are uh also persistent it's not only the
attackers that are persistent so we
tried to reach out uh via different
memes and you already know Matias fa
cantle's favorite uh way to reach out is
via LinkedIn so kle now has a lot more
LinkedIn friends and some organizations
have more uh information about data they
might have
lost of the organizations that reacted
well one stood stood out um they already
had uh seen the attacker and had already
evicted them um ker provided them with a
list of accounts where we knew that that
the attackers at lead has had the um
Cabos hashes of these accounts um they
checked it and said okay we already have
this discovered we we changed almost all
of them
already and um fortunately they also
figured out uh where the the scans came
from and they figured out new IP
addresses that they uh attackers were
using and gave us an additional SSH key
which was very really nice of them and
we uh thank you very
much so uh this is now victim 2 where we
got an additional um IP address and an
additional SSH key so that's Ser B Ser B
was config in a different manner Hackman
unfortunately could not loog into that
server um the lockin shell was sent set
to bin for so we couldn't uh heck Pacman
couldn't get a shell on this server but
what he discovered is he could do port
forwarding and this way of course he
could first try to basically port scan
the server and see what local ports are
open on that
server um and he discovered this way
he could have tunnels to the victims
networks where there were active
connections going
on there was also very handy server
running a service running on that server
uh on Port 80 of Local Host we had a web
server which just gave us uh gave the
output of uh lsof so we could see uh
which ports were
open now if you look carefully okay you
see some SSH connections which are the
reverse shell that are still act active
you can also see a pattern of the other
open ports they have
11109 11102 11105 and then you have a
different set of ports all ending in
0902 and
055 that's quite curious and uh of
course if heck backman discovers such a
thing he wants to probe this further and
he wants to do a bit more enumeration
turns out these three ports are always
forwarded to the victim networks it's
one RDP Port uh it's one SSH Port that's
forwarded and the other one uh did we
ever uh figure that out was it SMB or it
was SB right so uh three interesting
ports and um if you poke at these ports
you can figure out who they belong to
and that of course helps us in the
disclosure Pro process even if we cannot
download files from the server or get a
shell on that
server um so we could notify uh more
victims I think overall in the end we
notified about 50 organizations um that
they had an issue um not all of them
responded to
us uh some of them asked us what are we
supposed to do I mean in general what
you're supposed to do is run incident
response and even though incident
response always comes unexpected it's
something that you can prepare for um if
I understood correctly after this talk
there will be an incident response talk
here so maybe you should stick around
and figure out how this stuff works in
detail now we do have enough time for
for the bonus material material which is
not related to this uh um this let's say
C2 [ __ ] here but but uh some other
cases that we worked in the past
nowadays the ransomware crews um have
two ways to monetize uh their their
access the first way is the is the
classic way they encrypt your network
and then they demand Ransom so that you
can get a decryption key and get back
running and the other way is they start
to uh exfiltrate your data and they
threaten you to publish it if you don't
pay up
H and even though the internet in
Germany is quite shitty uh they managed
to ex um to exfiltrate more and more
data nowaday so um we had some contacts
with organization the first one lost 40
G gigabyte of data second one lost uh 30
terabyte of data and then we had a third
one that lost even more so good news uh
internet uh speed in Germany is getting
better um attackers are now able to
exate terabytes of data in a short
amount of time
but is that really that much of a
problem of course it's a problem if you
lose terabytes of your data but uh if it
gets published does anyone really is
anyone really able to download it all
kle uh spent a week trying to uh
download one of the the dumps uh of
course this was published uh over tour
um and they had their own web server
which which hosted the stuff it's it's
one of this classic uh ransomware
disclosure blocks but somehow they
messed their I messed up their
infrastructure there as well their HTTP
server didn't uh support range requests
and they gave uh the wrong HTTP codes
and of course also the infrastructure
was was unstable so uh downloads tended
to to end at some point in time um we
tried to use a tool area two onion
downloader which in theory is great to
do this stuff but nevertheless with this
set set up we did not manage to download
all of the data and and stopped after we
only had 20% after one week so this is
uh actually a tactic of the the
attackers right they want your money uh
and this is why they they steal your
data uh they are not really interested
in your data they are interested
interested in in pressuring you into
paying them uh and the same of course
goes into pressuring you to pay them to
take it back offline
now what lessons can we learn uh from
from this experience of course um real
world uh Defenders have a lot of
problems in uh in them having to defend
a big infrastructure which has grown
over a long amount of time uh mistakes
happen and they try and they tend to
compound so over time your your security
posture typically dictor
Ates um but as K tole also pointed out
you often have loansome heroes in it who
keep it all running and who give really
uh all the their heart to to maintaining
it and keeping it secure as best as they
can still it often feels like the cards
are stacked against the Defenders but as
you can see the attackers have similar
problems when they maintain their own
infrastructure right they also have a a
growing in a growing C2 infrastructure
maintaining that is difficult
configuration mistake can happens um
credential management is hard right
sometimes you lose an S key and then
people have uh have access to your setup
and there's plenty of rooms for f up
it's not only C2 infrastructure it's
also the way you publish the data and it
might also be the way uh you do
encryption if you manage to encrypt the
data but that's a story for a different
talk we might tell that at at Congress
maybe now if you want to be a real world
Hackman you see there's there's a lot of
angles that you can attack when you're
looking at uh the C2 infrastructure uh
Ransom Rec Crews obviously also have
have Staffing problems not all of their
staff is competent and some of them
don't really know what they're doing um
they had tend to be one tricks ponies
right you saw the list of tools none of
this is hard to defend against none of
this is custom uh it is known how to
prevent all of these mistakes so uh
don't despair uh it is possible to to
defend against these attackers in many
cases um helping ransomware victims uh
of course also is a lot of work
especially when you want to get in touch
with them but it can be really rewarding
if you manage to help them and if you
manage to help them uh actively def
evict the
attackers with that being said if you
want to disclose uh either
vulnerabilities or um give uh SSH keys
to the CCC uh there's a email address
here that you can use for that and we
encourage you all to do so thanks a lot
yeah any questions or SSH keys that you
would have for
us no questions great talk or or or the
talk I don't I I never know we'll see
that I don't I don't so you're you're
raising your hand
no I thought the microphone
was I don't see the microphone so do I
get it right to protect against data
leaks a good strategy is to just leak
more data so you can download
it so the question was if the the way to
defense against data leak is just to
leak more data so that it t gets lost in
the sea of data I mean have you seen
these ransomware blocks there's so much
data out there it's hard to download but
there is a
ton so if I understand it correctly you
have to walk towards the microphone if
you want to ask a question there are no
questions so we can oh no there is one I
don't there is a so uh do you think for
uh a company it might be a good idea to
DS the league platform instead of paying
the
ransomware I I think the tour Network
would not appreciate dos over tour but K
can say something about
that I run to exit notes I would not
like
[Music]
that
[Music]