0 00:00:00,000 --> 00:00:30,000 This subtitle is not finished yet. If you are able to, please support us and watch the talk in amara for the last changes: https://c3subtitles.de/talk/746 Thanks! 1 00:00:00,000 --> 00:00:13,850 *preroll music* 2 00:00:13,850 --> 00:00:18,720 Vasilios: Hello, everyone, thanks for coming today. I'm going to introduce the ultrasound 3 00:00:18,720 --> 00:00:24,500 ecosystem, which is an exotic and kind of little known ecosystem. So I would like to 4 00:00:24,500 --> 00:00:29,820 start with a short story about the product, which is also our motivation for 5 00:00:29,820 --> 00:00:39,870 this work. So some time ago, there was a product that worked in the ultrasound 6 00:00:39,870 --> 00:00:44,450 spectrum that cannot be perceived by humans. And the product was actually an 7 00:00:44,450 --> 00:00:50,700 interesting idea. It was very promising and everything, but it also had a fatal 8 00:00:50,700 --> 00:00:56,730 flaw. So now that I've done this introduction, I would like to tell you 9 00:00:56,730 --> 00:01:01,350 more about the story of the product and how it came to be and what was it? What 10 00:01:01,350 --> 00:01:08,150 was its lifecycle. So 2012, a company called SilverPush was a startup in 11 00:01:08,150 --> 00:01:14,810 India. It was founded there and they had this ultrasound device tracking product. 12 00:01:14,810 --> 00:01:18,770 I'll go more into the technical details later. So for a couple of years, they were 13 00:01:18,770 --> 00:01:26,789 working on that product. And it wasn't until 2014 that they basically got some 14 00:01:26,789 --> 00:01:32,769 serious funding from a venture center or other angel investors for millions. So in 15 00:01:32,769 --> 00:01:38,840 2014, they also got a few months after they got funded. They also got some press 16 00:01:38,840 --> 00:01:44,940 coverage about the product and they got some pretty good reviews on their 17 00:01:44,940 --> 00:01:49,260 newspapers and articles about what the product could do. And at the same time, 18 00:01:49,260 --> 00:01:52,950 they were doing what most of the companies are doing, like publishing patents about 19 00:01:52,950 --> 00:02:00,029 their technology and everything. So things later started to go like year after year 20 00:02:00,029 --> 00:02:06,499 and half maybe started to go not that well for them. The security community noticed 21 00:02:06,499 --> 00:02:10,759 and there was some press coverage about the product that was not so positive 22 00:02:10,759 --> 00:02:19,250 anymore. So this is one of the very first emails that appear on the Web regarding 23 00:02:19,250 --> 00:02:24,709 the product. So it's from a W3C working group. So a researcher there is 24 00:02:24,709 --> 00:02:31,220 basically. Notifying the other members of the group that, OK, there is this product, 25 00:02:31,220 --> 00:02:36,349 maybe there are transparency issues, and certainly the users are not aware of 26 00:02:36,349 --> 00:02:42,489 what exactly is going on there. So let's keep an eye on it. And so this was a very 27 00:02:42,489 --> 00:02:48,370 one of the very first things published about the product from the privacy and 28 00:02:48,370 --> 00:02:54,470 security perspective. So what happened then was the press took notice and they 29 00:02:54,470 --> 00:03:01,379 got all those headlines urging users to be very careful. And, oh, this is a this is 30 00:03:01,379 --> 00:03:08,769 evil, take care. People are eavesdropping on you and stuff. So, of course, this led 31 00:03:08,769 --> 00:03:14,950 also on the FTC to take action. They organized a workshop on cross device tracking 32 00:03:14,950 --> 00:03:22,060 in general, I think, and they made specific mentions for ultrasound cross device tracking 33 00:03:22,060 --> 00:03:28,170 don't worry if you're not familiar with this terms, I'm going to define everything later. So 34 00:03:28,170 --> 00:03:33,390 what basically they were saying is transparency issues. How do how do we 35 00:03:33,390 --> 00:03:39,860 protect ourselves? How is that thing working? So, then the users, of course, 36 00:03:39,860 --> 00:03:43,689 started to react. And there were like many people who were unhappy, they were 37 00:03:43,689 --> 00:03:48,430 complaining, what is this? I don't want that thing. So people were actually 38 00:03:48,430 --> 00:03:53,479 suggesting solutions and the solutions that were making sense. And of course, you 39 00:03:53,479 --> 00:04:01,260 have always the users that are completely immune to what you have there. So what 40 00:04:01,260 --> 00:04:10,549 happened then is like five months after the FTC took much more serious action 41 00:04:10,549 --> 00:04:16,000 regarding this specific product. So it sent a letter to all the developers. And 42 00:04:16,000 --> 00:04:19,780 the letter was essentially saying, you know, you're using this framework in 43 00:04:19,780 --> 00:04:27,160 Europe. We've seen it in Google Play store. It's not enough that you are asking 44 00:04:27,160 --> 00:04:31,380 for the microphone permission. You should let the users know that you are tracking 45 00:04:31,380 --> 00:04:36,330 them if you are doing so. Otherwise, you are violating rule X, Y, Z, and you're not 46 00:04:36,330 --> 00:04:42,590 allowed to do that. So this was pretty serious, I would say. And what happened 47 00:04:42,590 --> 00:04:46,610 next is basically the company withdrew from the US market and said, you know, we 48 00:04:46,610 --> 00:04:51,040 have nothing to do with the U.S. market and this product is not active there. You 49 00:04:51,040 --> 00:04:57,320 shouldn't be concerned. So end of story like the product is not out there in the 50 00:04:57,320 --> 00:05:03,660 US at least anymore. Are we safe? So it seemed to us that it was assumed that this 51 00:05:03,660 --> 00:05:10,030 was an isolated security incident. And to be fair, very little became known about 52 00:05:10,030 --> 00:05:15,090 the technology. At this point. The press moved on to other hot topics at the time, 53 00:05:15,090 --> 00:05:20,950 people went quiet, like if people are not using it, it's fine. So everyone 54 00:05:20,950 --> 00:05:25,840 seemed happy. But we're curious people. So we had lots of questions that were not 55 00:05:25,840 --> 00:05:35,040 answered. So our main questions was like why they were using ultrasounds. We'll see 56 00:05:35,040 --> 00:05:41,620 that what they are doing, you can do with our technologies, how such frameworks 57 00:05:41,620 --> 00:05:47,120 work. We had no idea there was no coverage or nothing about it. The technical, 58 00:05:47,120 --> 00:05:53,340 technically speaking, out there, are there other such products there? Because we were 59 00:05:53,340 --> 00:05:59,410 aware of one. Everyone on all the articles was referring to that one product, but we 60 00:05:59,410 --> 00:06:03,210 were not sure if there are others doing the same thing. And of course, we were 61 00:06:03,210 --> 00:06:07,870 looking for a report about the whole ecosystem and how it works. And there was 62 00:06:07,870 --> 00:06:13,290 nothing. So what do you do then if if there are no technical resources? 63 00:06:13,290 --> 00:06:19,740 Basically, we decided to do our own research and come up with this report that 64 00:06:19,740 --> 00:06:24,980 we were lacking. So we're done with motivation so far. We were pretty pumped 65 00:06:24,980 --> 00:06:29,891 up about looking into it. OK, what's there? The rest of the presentation will 66 00:06:29,891 --> 00:06:34,010 go as follows. Like first I'm going to introduce ultrasound tracking and other 67 00:06:34,010 --> 00:06:40,620 terminology, then I'm going to go on with the attack details. And indeed, we have an 68 00:06:40,620 --> 00:06:47,610 attack again against the Tor browser. Then we're doing a formal security analysis of 69 00:06:47,610 --> 00:06:53,350 the ecosystem and try to pinpoint the things that went wrong. And then we'll try 70 00:06:53,350 --> 00:07:00,470 to introduce our countermeasures and advocate for proper practices. So to begin 71 00:07:00,470 --> 00:07:06,940 with, I'm Vasilis. I've done this work with other curious people. These are 72 00:07:06,940 --> 00:07:13,440 showing how Yanick Fratantonio, Christopher Kruegel and Giovanni Vigna from UCSB and also 73 00:07:13,440 --> 00:07:19,280 Federico Maggi from Polytechnical Damilola. Let's now start with the 74 00:07:19,280 --> 00:07:26,340 ecosystem, so apparently ultrasounds are used in a lot of places and they can be 75 00:07:26,340 --> 00:07:30,920 utilized for different purposes, some of them are cross device tracking that are 76 00:07:30,920 --> 00:07:36,770 referred already to audience analytics, synchronized content, proximity, marketing 77 00:07:36,770 --> 00:07:41,460 and device pairing. You can do some other things, but you will see them later. So to 78 00:07:41,460 --> 00:07:48,920 begin with what cross device tracking is, cross device tracking is basically the holy 79 00:07:48,920 --> 00:07:53,630 grail for marketers right now because you're using your multiple devices 80 00:07:53,630 --> 00:07:57,990 smartphone, laptop, computer, maybe your TV and to them, your appear as different 81 00:07:57,990 --> 00:08:02,330 people. And they all want to be able to link to link those devices to know that 82 00:08:02,330 --> 00:08:06,920 you're the same person so that they can build their profiles more accurately. So, 83 00:08:06,920 --> 00:08:13,300 for instance, if you're watching an ad on the TV, they want to be able to know that 84 00:08:13,300 --> 00:08:19,720 it's you so that they can push relevant ads from your smartphone or follow up ads. 85 00:08:19,720 --> 00:08:27,860 Um. So this is employed by major advertising networks, and there are two 86 00:08:27,860 --> 00:08:32,780 ways to do it, either deterministically or probabilistically, that deterministic 87 00:08:32,780 --> 00:08:38,780 approach is much more reliable. You get 100 percent accuracy and works as follows. 88 00:08:38,780 --> 00:08:43,200 If you are Facebook, the users are heavily incentivized to log in from all their 89 00:08:43,200 --> 00:08:49,220 devices. So what happens is that. You can immediately know that, OK, this user has 90 00:08:49,220 --> 00:08:55,191 these three devices and I can put relevant content to all of them. However, if you 91 00:08:55,191 --> 00:08:58,910 are not Facebook or Google you, it's much more unlikely that the users would want to 92 00:08:58,910 --> 00:09:03,580 log into your platform from their different devices. So you have to look for 93 00:09:03,580 --> 00:09:09,970 alternatives. And one tool to come up with those alternatives are ultrasound beacons. 94 00:09:09,970 --> 00:09:18,480 So, um, ultrasound tracking products are using ultrasound because they may sound 95 00:09:18,480 --> 00:09:22,590 exotic, but basically there they are. What they are doing is they are encoding a 96 00:09:22,590 --> 00:09:29,460 sequence of symbols, um, in a very high frequency that it's inaudible by humans. 97 00:09:29,460 --> 00:09:35,520 That's the first key feature. The second one is they can be emitted by most commercial 98 00:09:35,520 --> 00:09:39,400 speakers and they can be captured by most commercial microphones, for instance, 99 00:09:39,400 --> 00:09:48,620 found on your smartphone. So the technical details are the following. I know there 100 00:09:48,620 --> 00:09:54,380 are a lot of experts in these kinds of things here, so I'm averaging out what how 101 00:09:54,380 --> 00:09:57,590 the companies are doing it right now. I'm not saying that this is the best way to do 102 00:09:57,590 --> 00:10:01,520 it, but this is more or less what they're doing. Of course, they have patents, so 103 00:10:01,520 --> 00:10:06,230 each one of them is doing a slightly different thing so they don't overlap. 104 00:10:06,230 --> 00:10:13,330 They're using the near ultrasound spectrum between the eight eight kilohertz and 20 105 00:10:13,330 --> 00:10:18,940 kilohertz, which is inaudible by usually by adults. They divide it in smaller 106 00:10:18,940 --> 00:10:27,380 chunks. So if you divide it in chunks that have size of 75 Hertz, you get 26, about 26 107 00:10:27,380 --> 00:10:33,920 chunks, and then you can assign letter of the alphabet on each one of them. And then 108 00:10:33,920 --> 00:10:38,109 what they are doing is usually within four to five seconds. They emit sequences of 109 00:10:38,109 --> 00:10:45,410 characters. Usually they contain for four to six characters in there, and they use 110 00:10:45,410 --> 00:10:51,670 it to incorporate a unique ID corresponding to their source, they attach 111 00:10:51,670 --> 00:10:56,710 the beacon to. So there is no ultrasound beacon standard, as I said previously, but 112 00:10:56,710 --> 00:11:00,450 there are lots of patents, so each one of them is doing a slightly different thing. 113 00:11:00,450 --> 00:11:06,350 But this is a basic principle. We did some experiments and we found out that within 114 00:11:06,350 --> 00:11:13,880 seven meters, you get pretty good accuracy in low error rate. So of course, this depends 115 00:11:13,880 --> 00:11:20,250 exactly how you encode things. But with applications found on Google Play, this 116 00:11:20,250 --> 00:11:24,990 worked up to seven meters. Um, we couldn't find computer speakers that were not able 117 00:11:24,990 --> 00:11:33,310 to emit near ultrasound frequencies and work with this technology and.. we this is 118 00:11:33,310 --> 00:11:36,590 pretty known for this kind of frequencies, they cannot penetrate through physical 119 00:11:36,590 --> 00:11:41,000 objects, but this is not a problem for their purposes. And we did some 120 00:11:41,000 --> 00:11:46,720 experiments with our research assistant and we can say that they are audible by 121 00:11:46,720 --> 00:11:54,420 animals. So if you combine cross device tracking and ultrasound because you get 122 00:11:54,420 --> 00:12:02,350 ultrasound cross device tracking. So now what you can do with this and this is this is a 123 00:12:02,350 --> 00:12:07,320 pretty good idea, actually, because it offers high accuracy, you don't ask the 124 00:12:07,320 --> 00:12:16,720 users to log in, which is very high, very demanding thing to ask for. You can embed 125 00:12:16,720 --> 00:12:22,860 those beacons in websites or TV ads, and this technology, however, requires some 126 00:12:22,860 --> 00:12:26,210 sort of sophisticated backend infrastructure. We're going to see more 127 00:12:26,210 --> 00:12:30,260 about it later. And you also need the network of publishers who are willing to 128 00:12:30,260 --> 00:12:36,740 incorporate incorporate beacons in their content, whatever this content is. And 129 00:12:36,740 --> 00:12:41,250 then, of course, you need an ultrasound cross device tracking framework that is going 130 00:12:41,250 --> 00:12:47,080 to run on the user's mobile device, a smartphone. So these frameworks are 131 00:12:47,080 --> 00:12:52,680 essentially and as the advertising SDK is the key that the developers can use to display 132 00:12:52,680 --> 00:12:57,060 ads on their free apps. So it's not that the developers are going to incorporate 133 00:12:57,060 --> 00:13:04,491 the ultrasound framework is going to incorporate an advertising SDK with 134 00:13:04,491 --> 00:13:09,610 varying degrees of understanding of what it does. So here is how ultrasound cross device 135 00:13:09,610 --> 00:13:15,740 tracking works. On step one, basically, we have the advertising client. He just wants 136 00:13:15,740 --> 00:13:20,200 to advertise, advertises his products. He goes to the ultrasound cross device 137 00:13:20,200 --> 00:13:25,250 tracking provider who has the infrastructure set up, set up a campaign, 138 00:13:25,250 --> 00:13:31,610 and they provide their associates a unique ultrasound because with this campaign and 139 00:13:31,610 --> 00:13:37,660 then pushes this become to content publishers to incorporate them 140 00:13:37,660 --> 00:13:43,860 incorporated into their content, depending on what the advertiser advertising client 141 00:13:43,860 --> 00:13:49,270 is trying to achieve. So this is step three or step for a user is basically 142 00:13:49,270 --> 00:13:56,950 accessing all of those content providers either. This is a TV ad or a website on 143 00:13:56,950 --> 00:14:03,030 the Internet and one this once this content is loaded or displayed by your TV. 144 00:14:03,030 --> 00:14:08,010 At the same time, the device, the devices speakers are emitting the ultrasounds. And 145 00:14:08,010 --> 00:14:13,580 if you have the ultrasound cross device tracking framework on your phone, which is usually 146 00:14:13,580 --> 00:14:18,910 listening on the background, then it picks up the ultrasound and on step six, it 147 00:14:18,910 --> 00:14:25,060 submits it back to the service provider, which now knows that, OK, this guy has 148 00:14:25,060 --> 00:14:31,700 watched this DVR or whatever it is, and I'm going to add this to his profile and 149 00:14:31,700 --> 00:14:38,220 push our target dates back to his device. So, of course, by doing this, they're just 150 00:14:38,220 --> 00:14:45,900 trying to improve, improve their conversion rate and get more customers. 151 00:14:45,900 --> 00:14:52,970 Another use of ultrasounds currently in practice is proximity marketing, so venues 152 00:14:52,970 --> 00:14:59,380 basically set up multiple, multiple ultrasound meters. This is kind of fancy 153 00:14:59,380 --> 00:15:05,350 name for speakers and this is kind of the nice thing about the ultrasound. You just 154 00:15:05,350 --> 00:15:11,470 need speakers. So they put this in multiple locations in their venue, either 155 00:15:11,470 --> 00:15:18,320 a supermarket or a stadium, for instance, and then there is a customer up. If you're 156 00:15:18,320 --> 00:15:23,310 a supermarket, there is a supermarket up. If you're an NBA team, which will see 157 00:15:23,310 --> 00:15:29,730 later, you have this fun application that the fans of your team can download 158 00:15:29,730 --> 00:15:35,080 and install on their smartphones. And then once this app, this happens, listing on 159 00:15:35,080 --> 00:15:40,550 the background and it picks up the ultrasound and submits them back to the 160 00:15:40,550 --> 00:15:47,660 company. So the main purpose of using is this is basically to study in user 161 00:15:47,660 --> 00:15:55,220 behavior, in user behavior, provide real time notifications like, OK, you are in 162 00:15:55,220 --> 00:15:59,610 this aisle on the supermarket, but if you just walk two meters down, you're going to 163 00:15:59,610 --> 00:16:06,170 see this product in discount. Or the third point, which kind of incentivizes the 164 00:16:06,170 --> 00:16:11,240 users more, is basically you're offering reward points for users visiting your 165 00:16:11,240 --> 00:16:17,600 store. And actually there is a product doing exactly that on the market. So some 166 00:16:17,600 --> 00:16:23,830 other uses are device pairing. And this basically relies on the fact that 167 00:16:23,830 --> 00:16:29,029 ultrasounds do not penetrate through objects. So if you have a small TV, say, 168 00:16:29,029 --> 00:16:36,810 with or Chromecast, for instance, they can emit random PIN through ultrasound. Your 169 00:16:36,810 --> 00:16:40,700 device picks it up and submits it back to the device through the Internet. And now 170 00:16:40,700 --> 00:16:44,410 you've proved that you are on the same physical location with the with Chromecast 171 00:16:44,410 --> 00:16:51,320 or whatever your TV is. Also, Google recently acquired sleek login. They are 172 00:16:51,320 --> 00:16:55,770 also using ultrasounds for authentication. It's not entirely clear what their product 173 00:16:55,770 --> 00:17:00,120 is about, though. And also you have audience measurement and analytics. So 174 00:17:00,120 --> 00:17:07,240 what they are doing is basically if you're if you incorporate multiple beacons in the 175 00:17:07,240 --> 00:17:12,169 night, then you can basically track the reactions and the behavior of the users of 176 00:17:12,169 --> 00:17:17,559 it, of the audience in the sense that first, you know, how many people have 177 00:17:17,559 --> 00:17:21,470 watched your ad a second, you know what happened. So if they show it's Sanderlin 178 00:17:21,470 --> 00:17:26,709 between and this, so they submit only the first beacon of the two, if you have two, 179 00:17:26,709 --> 00:17:34,389 then you also track their behavior. OK, so we've seen all these technologies and then 180 00:17:34,389 --> 00:17:40,779 we started wondering how secure is that thing? Like, OK, what security measures 181 00:17:40,779 --> 00:17:46,470 are there applied by companies and everything? So I'm going to immediately 182 00:17:46,470 --> 00:17:51,369 start with the exploitation of the technology. So to do that, we just need 183 00:17:51,369 --> 00:17:59,470 the computer with speakers and the Tor browser and the smartphone with an ultrasound app 184 00:17:59,470 --> 00:18:03,000 and a state level advisory. I'm going to say more about the state level advisory 185 00:18:03,000 --> 00:18:11,679 later, but just keep in mind that it's on the Tor threat model, so. I have a 186 00:18:11,679 --> 00:18:15,059 video of the attack. I'm going to stop it, I'm going to pose it in different places 187 00:18:15,059 --> 00:18:24,669 to explain some more stuff. Yeah, OK, so I'm going to set up the scene before that. 188 00:18:24,669 --> 00:18:28,019 So let's make the assumption that we have a whistle blower that wants to leak some 189 00:18:28,019 --> 00:18:34,189 documents to a journalist, but he doesn't know that the journalist is working with 190 00:18:34,189 --> 00:18:38,699 the government and his main intent is basically to deanonymize him. So the 191 00:18:38,699 --> 00:18:42,559 journalist does the following, asks the whistleblower to upload the documents to a 192 00:18:42,559 --> 00:18:48,360 Tor hidden service or a website that he owns. And the whistleblower basically thinking 193 00:18:48,360 --> 00:18:55,340 that he's safe to do that through Tor loads the page. So now I'm having I have the 194 00:18:55,340 --> 00:19:07,279 demo, which is exactly that implements exactly that scenario. So the whistle 195 00:19:07,279 --> 00:19:12,970 blower opens the Tor browser, so the setup is the following, we have the phone next to 196 00:19:12,970 --> 00:19:16,780 the computer. This can be up to seven meters away, but for practical purposes, 197 00:19:16,780 --> 00:19:21,169 it has to be next to the computer. So we have the Tor browser. What are we going to do 198 00:19:21,169 --> 00:19:28,749 first? For the purpose of the demo, we use them smart for listening framework that's 199 00:19:28,749 --> 00:19:36,529 visible to the.. to the user. This is basically the demo(?). Those apps, ultrasound 200 00:19:36,529 --> 00:19:40,929 cross device tracking apps run in the background, so now we're setting set it on listening 201 00:19:40,929 --> 00:19:46,280 mode so that it starts listening. Of course, in normal framework, the user 202 00:19:46,280 --> 00:19:52,570 doesn't have to do that part. But we want to show that. We want to show that what's 203 00:19:52,570 --> 00:20:02,570 happening. So now the whistleblower is going to load the innocuous were paid, 204 00:20:02,570 --> 00:20:12,980 suggested by the journalist and see what happens to. OK, now we've loaded the page 205 00:20:12,980 --> 00:20:20,319 and the phone is listening in reality in the background, so let's see what happens. 206 00:20:30,589 --> 00:20:36,320 OK, this is looks pretty bad. We have lots of information about the user visiting our 207 00:20:36,320 --> 00:20:43,700 hidden service. I assume you already have some clues about how this happened, what the 208 00:20:43,700 --> 00:20:54,850 information that we have is the following. First of all. We have his IP address, 209 00:20:54,850 --> 00:21:02,070 phone number. Don't call this phone number, because this isn't right. The ID 210 00:21:02,070 --> 00:21:10,859 is he may end his Google account email. So this is enough to say and his location, of 211 00:21:10,859 --> 00:21:15,389 course, and this is enough to say that we essentially deanonymized him, even if we 212 00:21:15,389 --> 00:21:22,340 had the IP address, that would have been enough. So before I explain exactly how 213 00:21:22,340 --> 00:21:26,169 the attacked work, I'm going to introduce some tools that the attackers have at 214 00:21:26,169 --> 00:21:32,320 their disposal. The first one is a Bitcoin injection. So what you can essentially do 215 00:21:32,320 --> 00:21:37,229 is basically craft your own ultrasound beacons and push them to devices, 216 00:21:37,229 --> 00:21:40,809 listening for beacons, and then their devices are going to treat them like valid 217 00:21:40,809 --> 00:21:45,160 beacons and submit them back to the company's backend. And then the same 218 00:21:45,160 --> 00:21:49,989 things. Basically, you can also replace ultrasound beacons, meaning that you can 219 00:21:49,989 --> 00:21:55,320 capture them from virus location. And this is actually happening on the wild at a 220 00:21:55,320 --> 00:22:04,309 large scale for a specific application. And then once you capture those beacons, 221 00:22:04,309 --> 00:22:11,429 you can replace them back to the company's back end through the user's devices to 222 00:22:11,429 --> 00:22:16,990 give you a clue. There is a company that incentivizes users to visit stores by 223 00:22:16,990 --> 00:22:22,720 providing them offers and end points when they are visiting stores and people are 224 00:22:22,720 --> 00:22:27,660 capturing the beacons and are replaying them back to their devices from home. So they 225 00:22:27,660 --> 00:22:30,580 are selling the beacons through the Internet so that they don't have to go to 226 00:22:30,580 --> 00:22:39,559 the actual stores. OK, the problem here is basically that the framework is handling 227 00:22:39,559 --> 00:22:43,000 every beacon. It doesn't have a way to distinguish between the valid and 228 00:22:43,000 --> 00:22:48,050 maliciously crafted beacons. And my favorite tool for the attackers is basically a beacon 229 00:22:48,050 --> 00:22:55,350 trap, which is a code snippet that once you loaded, you basically reproduce 230 00:22:55,350 --> 00:23:01,679 one or more inaudible beacons that the attacker chose to. So this can happen in 231 00:23:01,679 --> 00:23:06,759 lots of ways on the demo. I use the first one. So you build a website and you have 232 00:23:06,759 --> 00:23:12,189 some JavaScript there just playing the ultrasounds from the back. What else you can 233 00:23:12,189 --> 00:23:17,769 do is basically start crosseyed scripting vulnerability. Just exploit it on any 234 00:23:17,769 --> 00:23:22,929 random website and then you can inject beacons to the visitors of this website 235 00:23:22,929 --> 00:23:30,249 or a man-in-the-middle attacks just adding or javascript snippet on that 236 00:23:30,249 --> 00:23:37,779 user's traffic or they send an audio message to the to the victim. So how did 237 00:23:37,779 --> 00:23:41,830 Tor deanonymization attack work? It's the following. So first the adversary needs to 238 00:23:41,830 --> 00:23:50,049 set up, set up a campaign, and then once he captures the the beacon associated with 239 00:23:50,049 --> 00:23:55,540 that campaign, he builds a beacon trap and essentially on step three lures, the user 240 00:23:55,540 --> 00:24:00,789 to visit it. This is what the journalist basically did for the whistleblower on our 241 00:24:00,789 --> 00:24:05,840 scenario. Then the user loads the resource. He has no idea this is possible. 242 00:24:05,840 --> 00:24:12,440 And she slapped him amidst the ultrasound, beacon. If you if your smartphone has such a 243 00:24:12,440 --> 00:24:17,460 framework, it's going to pick it up and submit it back to the provider and I don't 244 00:24:17,460 --> 00:24:22,179 know about you, but when I'm using Tor, I'm not connecting my phone through to the 245 00:24:22,179 --> 00:24:25,509 Internet, through the Tor network. My phone is connected through my normal Wi- 246 00:24:25,509 --> 00:24:34,039 Fi. So now the ultrasound service provider knows that the you know, this smartphone 247 00:24:34,039 --> 00:24:37,690 device omitted that specific beacon. And then I step seven, basically the 248 00:24:37,690 --> 00:24:42,809 adversary, which is state level adversary. Can simply subpoena the provider for the 249 00:24:42,809 --> 00:24:48,509 AP or other identifiers, which from what we've seen, they collect plenty of them. 250 00:24:48,509 --> 00:24:54,519 OK, so the first two elements, we have them already like the Tor browser 251 00:24:54,519 --> 00:25:02,919 computer, which biggest fine smartphone with ultrasound tracking enabled 252 00:25:02,919 --> 00:25:08,330 framework. Fine. What about the state level adversity? So we didn't have a state 253 00:25:08,330 --> 00:25:13,090 level adversity handy. So what we did is basically we redirected the 254 00:25:13,090 --> 00:25:18,540 traffic from step six to the advertized backend. And I want to stress a point 255 00:25:18,540 --> 00:25:28,600 here. This is not. A long, long shot assumption. So what we've seen in October 256 00:25:28,600 --> 00:25:33,179 is the following. I don't know how many of you realize, but AT&T was running a spy 257 00:25:33,179 --> 00:25:41,029 program, a thing called Hammesfahr, and it was providing paid access to governments 258 00:25:41,029 --> 00:25:45,269 only with an administrative subpoena, which is not doesn't even need to be 259 00:25:45,269 --> 00:25:50,789 obtained by it's ads. So it's pretty easy for them to get access to this kind of 260 00:25:50,789 --> 00:25:55,520 data. Especially we're talking about an IP address. It's not it's very easy for them 261 00:25:55,520 --> 00:26:01,570 to get it. So we also came up with some more attacks. First one is profile, 262 00:26:01,570 --> 00:26:07,710 corruption. Advertisers really like to build profiles about you, your interests 263 00:26:07,710 --> 00:26:15,210 and your behavior. So what you are basically doing is you can inject beacons 264 00:26:15,210 --> 00:26:21,209 to other people or even to your own phone and then you can malform their profile. 265 00:26:21,209 --> 00:26:28,309 Exactly. The impact of this attack depends on how the backend of the advertising 266 00:26:28,309 --> 00:26:33,039 company and the infrastructure works, but the attack is definitely possible. And 267 00:26:33,039 --> 00:26:40,169 then there is information leakage attack were works under a similar assumption. You 268 00:26:40,169 --> 00:26:44,510 can replay Beacon's eavesdrop and replay because your own phone to make your 269 00:26:44,510 --> 00:26:49,519 profile similar to that of the victims. And then based on how recommendation 270 00:26:49,519 --> 00:26:56,299 systems work, you're very likely to get similar arts and similar content with that 271 00:26:56,299 --> 00:27:01,019 of the victims. So of course, this also depends about exactly how the 272 00:27:01,019 --> 00:27:07,369 recommendation system is implemented, but it's definitely possible. OK, so we've 273 00:27:07,369 --> 00:27:11,539 seen certain things that makes us think that, OK, the ecosystem is not very 274 00:27:11,539 --> 00:27:19,409 secure. Um, we try to find out exactly why this happened. So we did a security 275 00:27:19,409 --> 00:27:24,580 evaluation or we came up with four points. The first one is that we came up with we 276 00:27:24,580 --> 00:27:31,749 realized that the threat model is inaccurate, that ultrasound, because none 277 00:27:31,749 --> 00:27:39,130 of the implementations we've seen had any security features. Um, they also violated 278 00:27:39,130 --> 00:27:44,149 the fundamental security principle and they lacked transparency when it comes 279 00:27:44,149 --> 00:27:49,269 when it came to user interface. So let's go through them one by one. So inaccurate 280 00:27:49,269 --> 00:27:52,999 and model. Basically what they do is basically they rely on the fact that 281 00:27:52,999 --> 00:27:58,360 ultrasounds cannot penetrate the walls and they travel up to seven meters reliably. 282 00:27:58,360 --> 00:28:05,559 However, as I said, as a matter of fact, they also assume that you cannot capture 283 00:28:05,559 --> 00:28:10,870 and replay because because of that, that's the reason, um, what what's happening in 284 00:28:10,870 --> 00:28:15,029 practice, that you can get really close using beacon traps. So their assumption 285 00:28:15,029 --> 00:28:21,800 is not that accurate. Um, also, the security capabilities of beacons are 286 00:28:21,800 --> 00:28:30,129 heavily constrained by the low bandwidth the channel is has the limited time that 287 00:28:30,129 --> 00:28:33,580 you have to reach the users. So if someone is in a supermarket, he's not going to 288 00:28:33,580 --> 00:28:37,170 stop somewhere for a very long time. So you have limited time and a noisy 289 00:28:37,170 --> 00:28:42,440 environment. So you want a very low error rate. And adding crypto to the beacons 290 00:28:42,440 --> 00:28:49,139 it may not be a good idea, but it also results. This also results in replay in 291 00:28:49,139 --> 00:28:54,259 injection attacks being possible. Um, we also hear the violation of the privilege 292 00:28:54,259 --> 00:28:59,849 of, uh, sorry, the principle of least privilege. So what happens is basically all these 293 00:28:59,849 --> 00:29:05,110 apps need full access to the microphone. And based on the way it works, it's 294 00:29:05,110 --> 00:29:10,489 completely unnecessary for them to gain access to the audible frequencies. 295 00:29:10,489 --> 00:29:14,669 However, even if they want to, there's no way to gain access only to the ultrasound 296 00:29:14,669 --> 00:29:20,530 spectrum, both in Android and iOS. You have to gain either access to the whole 297 00:29:20,530 --> 00:29:26,629 spectrum or no access at all. So this, of course, results in the first malicious 298 00:29:26,629 --> 00:29:32,229 developers can at any time start using their access to the microphone. And of 299 00:29:32,229 --> 00:29:38,520 course, all the benign ultrasound enabled apps are perceived by as malicious by the 300 00:29:38,520 --> 00:29:45,399 users. And this actually will say more about it later. So lack of transparency is 301 00:29:45,399 --> 00:29:51,259 inclose. This is a bad combination with what exactly we've seen previously, 302 00:29:51,259 --> 00:29:55,919 because it that we've observed large discrepancies between apps when it comes 303 00:29:55,919 --> 00:30:00,879 to informing the users and also lots of discrepancies when it comes to providing 304 00:30:00,879 --> 00:30:06,110 opt out options. And there is a conflict of interest there, because if you're a 305 00:30:06,110 --> 00:30:12,600 framework developer, developer, you want to advise for proper practices for your 306 00:30:12,600 --> 00:30:17,960 customers, but you are not you're not going to enforce them or kind of blackmail 307 00:30:17,960 --> 00:30:22,499 them. Either you do it properly or you're not using my framework. So there is a 308 00:30:22,499 --> 00:30:27,190 conflict of interest there. So what happened because of a lack of 309 00:30:27,190 --> 00:30:33,289 transparency is the following. Signals 360 is one of those frameworks. An NBA team 310 00:30:33,289 --> 00:30:39,500 started using this in May. And then a few months after there is a sue and someone 311 00:30:39,500 --> 00:30:43,839 claims, you know, that thing is listening in the background. And what's interesting 312 00:30:43,839 --> 00:30:49,220 is on the claim, what they are saying is, OK, I gave permission through the Android 313 00:30:49,220 --> 00:30:54,119 permission system for them to access the microphone, but it was not explained to me 314 00:30:54,119 --> 00:30:58,840 exactly what they were doing. And this is in close ties with what the FTC was saying 315 00:30:58,840 --> 00:31:08,740 in the letter a few months ago. Also, again, the same story, um, football team 316 00:31:08,740 --> 00:31:14,340 starts using such a framework a few months after people are complaining that they are 317 00:31:14,340 --> 00:31:21,679 being eavesdropped on. Um, I think what happened here is that. When the team was 318 00:31:21,679 --> 00:31:25,749 playing a match, the application started listening for ultrasounds, but not all 319 00:31:25,749 --> 00:31:29,560 your fans are going to be in the stadium, so you end up listening for ultrasounds in 320 00:31:29,560 --> 00:31:37,029 a church and other places. So, yeah, people were also pissed. Um, OK, just to 321 00:31:37,029 --> 00:31:41,989 put it into perspective how prevalent these technologies are, the ecosystem is 322 00:31:41,989 --> 00:31:48,000 growing. Even though that one company withdrew. There are other companies in the 323 00:31:48,000 --> 00:31:54,899 ecosystem are coming up with new products as well. So the number of users is 324 00:31:54,899 --> 00:32:00,110 relatively low, but it's also very hard to estimate right now. We could find around 325 00:32:00,110 --> 00:32:05,269 10 companies offering ultrasound related products and the majority of them is 326 00:32:05,269 --> 00:32:09,780 gathered around proximity marketing. There was only one company doing ultrasound 327 00:32:09,780 --> 00:32:16,590 cross device tracking. At least we found one. Um, and this is mainly due to 328 00:32:16,590 --> 00:32:21,290 infrastructure complexity. It's not easy to do all those things. And secondly, I 329 00:32:21,290 --> 00:32:26,140 also believe that the whole backslash from the security community is incentivized 330 00:32:26,140 --> 00:32:32,599 other companies from joining because they don't want a tarnished reputation. OK, so 331 00:32:32,599 --> 00:32:36,919 we have this situation right now. Companies are using ultrasound. What are 332 00:32:36,919 --> 00:32:48,340 we going to do? So this was our initial idea. This is what we thought first. But 333 00:32:48,340 --> 00:32:54,950 we want to fix things. So we tried to come up with certain steps that we need to take 334 00:32:54,950 --> 00:33:02,019 to actually fix that thing and make it usable, but not dangerous. Um, so we 335 00:33:02,019 --> 00:33:07,240 listed what's wrong with it. We did it already. We we developed some quick fixes 336 00:33:07,240 --> 00:33:12,330 that I'm going to present later and medium term solutions as well. And then we 337 00:33:12,330 --> 00:33:16,830 started advocating for a long term changes that are going to make the ecosystem 338 00:33:16,830 --> 00:33:23,650 reliable. And we need the involvement from the community there. Definitely. So. We 339 00:33:23,650 --> 00:33:29,519 developed some short and medium term solutions, um, the first one is a browser 340 00:33:29,519 --> 00:33:37,890 extension, our browser extension basically does the following is based on HTML5, the 341 00:33:37,890 --> 00:33:45,899 Web audio API. Um, it filters all audio sources and places a filter between the 342 00:33:45,899 --> 00:33:51,280 audio source and the destination on the Web page and filters out ultrasounds. To 343 00:33:51,280 --> 00:33:55,489 do that, we use a heisel filter that attenuates all frequencies above 18kHz 344 00:33:55,489 --> 00:34:04,539 and it works pretty reliably. And we leave all audible frequencies, intact. 345 00:34:04,539 --> 00:34:10,060 But it's not going to work with obsolete legacy technologies such as 346 00:34:10,060 --> 00:34:16,789 flash. OK, we also have an adroit permission, I think this somewhat more 347 00:34:16,789 --> 00:34:22,980 medium term solution, what we did is we developed a unique developed parts for the 348 00:34:22,980 --> 00:34:28,810 Android permission system. This allows for fine grained control over the audio channel, 349 00:34:28,810 --> 00:34:35,099 basically separates the permission needed for listening to audible sound and the 350 00:34:35,099 --> 00:34:39,750 permission needed for listening to the ultrasound spectrum. So at least we force the 351 00:34:39,750 --> 00:34:44,559 applications to specifically declare that they are going to listen to four 352 00:34:44,559 --> 00:34:49,399 ultrasounds. And of course, users can, on the latest Android versions, can also 353 00:34:49,399 --> 00:34:54,369 disable this permission and it can act as an opt out option if the app is not 354 00:34:54,369 --> 00:35:02,899 providing it. We also initiated discussion on the Turbo Tracker, but, um, we have, 355 00:35:02,899 --> 00:35:09,380 um, we are advocating for some long term solutions, so we really need some 356 00:35:09,380 --> 00:35:15,650 standardization here. Um, let's agree on ultrasound to confirm that and decide what 357 00:35:15,650 --> 00:35:20,440 security features can be there. I mean, we need to figure out what's technically 358 00:35:20,440 --> 00:35:25,410 possible there because it's not clear. And then once we have a standard, we can start 359 00:35:25,410 --> 00:35:32,109 building some APIs. And the APIs are very nice idea because, um, they will work as 360 00:35:32,109 --> 00:35:37,250 the Bluetooth APIs work, meaning that they will provide some methods to discover, 361 00:35:37,250 --> 00:35:42,240 process, generate and emit the sound beacons through a new API related 362 00:35:42,240 --> 00:35:48,809 permission. And this means that we will stop having overprivileged apps. We won't 363 00:35:48,809 --> 00:35:54,310 need access to the microphone anymore, which is a huge problem right now. And of 364 00:35:54,310 --> 00:35:58,700 course, the applications will not be considered spying anymore. And there is 365 00:35:58,700 --> 00:36:03,630 also another problem that we found out while we were playing with those shops. 366 00:36:03,630 --> 00:36:08,240 Um, if you have a framework listening through the microphone, other apps cannot 367 00:36:08,240 --> 00:36:12,289 access it. So we are trying to open the camera app to record the video on the app. 368 00:36:12,289 --> 00:36:17,320 Camera app was crashing because the framework was locking the access to the 369 00:36:17,320 --> 00:36:22,349 microphone. Now we may have some developers from frameworks saying, you 370 00:36:22,349 --> 00:36:26,020 know, I'm not going to use your API. I'm going to keep asking for access to the 371 00:36:26,020 --> 00:36:34,090 microphone. But we can force them to use this API if we somehow, um, by default 372 00:36:34,090 --> 00:36:38,750 filter out the ultrasound frequencies from the microphone and 373 00:36:38,750 --> 00:36:44,640 provide the way to the user to enable them on a pure application basis from his 374 00:36:44,640 --> 00:36:56,200 phone. OK, so. Here's what we did, um, we analyzed them, multiple ultrasound 375 00:36:56,200 --> 00:37:00,329 tracking technologies, we saw what what's out there in the real world and reverse 376 00:37:00,329 --> 00:37:08,500 engineered such frameworks. We identified, um, quite a few security shortcomings. We 377 00:37:08,500 --> 00:37:16,150 introduced our attacks and proposed some, um, usable countermeasures. Um, and 378 00:37:16,150 --> 00:37:21,580 hopefully we initiated the discussion about standardizing ultrasound because, 379 00:37:21,580 --> 00:37:27,539 um, but there are still things left to do. So for the application developers, please, 380 00:37:27,539 --> 00:37:32,880 um, explicitly notify the users about what your app is doing. Many of them would 381 00:37:32,880 --> 00:37:41,150 appreciate to know that. Um, also, we need to improve transparency in the data 382 00:37:41,150 --> 00:37:47,150 collection process because they collecting lots of data and very few information were 383 00:37:47,150 --> 00:37:52,010 available about what kind of data they framework's collect. Um, we also think 384 00:37:52,010 --> 00:37:57,010 it's a good idea to have an opt in option if it's not too much to ask, at least an 385 00:37:57,010 --> 00:38:07,910 opt out and standard security practices, um, as always. So framework providers 386 00:38:07,910 --> 00:38:13,730 basically need to make sure that the developers inform the users and also make 387 00:38:13,730 --> 00:38:21,030 sure that the users consent regularly to listening for because like it's not enough 388 00:38:21,030 --> 00:38:25,809 if you consent once and then a month after the app is still listening for ultrasound beacons 389 00:38:25,809 --> 00:38:33,170 you have to periodically ask the user if it's still okay to do that. Um. Ideally, every time 390 00:38:33,170 --> 00:38:39,619 you are going to listen and then, of course, we need to work on standardizing 391 00:38:39,619 --> 00:38:43,930 ultrasound because this is going to be a long process and then building the 392 00:38:43,930 --> 00:38:48,430 specialized, specialized API. Hopefully this is going to be easier once we have a 393 00:38:48,430 --> 00:38:56,960 standard and see what kind of authentication mechanisms can we have in 394 00:38:56,960 --> 00:39:03,989 this kind of constrained transmission channel. So.. 395 00:39:03,989 --> 00:39:17,149 *applause* 396 00:39:17,149 --> 00:39:21,229 Herald: Thank you Vasilios. If you have any questions, please do line up at the four 397 00:39:21,229 --> 00:39:26,679 microphones here in the walkways and the first question will be the front 398 00:39:26,679 --> 00:39:30,959 microphone here. Mic: Hello and thank you for your 399 00:39:30,959 --> 00:39:35,240 presentation. And I have a couple of questions to ask that are technical and 400 00:39:35,240 --> 00:39:41,070 they are very related. First of all, do you think that blocking out in our system 401 00:39:41,070 --> 00:39:47,799 level the high frequencies for either microphone or the speakers as well, a 402 00:39:47,799 --> 00:39:53,070 something that is technically feasible and will not put a very high latency in the 403 00:39:53,070 --> 00:39:56,750 processing? Vasilios: So we did that through the 404 00:39:56,750 --> 00:39:59,350 permission. You are talking about the smartphone right? 405 00:39:59,350 --> 00:40:03,850 Mic: Yeah, basically, because you have to have a real time sound and microphone 406 00:40:03,850 --> 00:40:06,769 feedback. Vasilios: So we did that with the 407 00:40:06,769 --> 00:40:14,179 permission. And I think it's not it's not to resource demanding, if that's 408 00:40:14,179 --> 00:40:17,219 your question. So it's definitely possible to do that. 409 00:40:17,219 --> 00:40:21,820 Mic: And the second part is, so there is a new market maybe for some 410 00:40:21,820 --> 00:40:28,170 companies producing and microphones and speakers that explicitly block out 411 00:40:28,170 --> 00:40:33,860 ultrasounds, right? Vasilios: Possibly. Possibly. Um, I'm not 412 00:40:33,860 --> 00:40:38,690 sure if you can do this from the application level. We developed parts for 413 00:40:38,690 --> 00:40:43,869 the Android system. I think our first approach back then was basically try to 414 00:40:43,869 --> 00:40:48,130 build an app to do that from the application, from the user land. And 415 00:40:48,130 --> 00:40:53,100 basically, I'm not sure if you can I doubt actually an Android if you can filter out 416 00:40:53,100 --> 00:40:58,569 ultrasounds. But from a browser, we have our extension. It works on Chrome. You can 417 00:40:58,569 --> 00:41:04,250 easily use our code to do the same thing on the Firefox. 418 00:41:04,250 --> 00:41:06,600 Mic: Thanks. Herald: The next question is from the 419 00:41:06,600 --> 00:41:10,460 front right microphone. Mic: Thank you for your talk. I have a 420 00:41:10,460 --> 00:41:15,220 question about the attack requirements against the whistleblower using Tor. 421 00:41:15,220 --> 00:41:23,730 I'm curious, the attacker has access to the app on the smartphone and also access 422 00:41:23,730 --> 00:41:32,790 to the smartphone microphone. Wouldn't the attacker then be able to just listen in on 423 00:41:32,790 --> 00:41:37,340 the conversation of the whistleblower and thereby identify him? 424 00:41:37,340 --> 00:41:40,670 Vasilios: Yeah, absolutely. Absolutely. It's a major problem. The problem is that 425 00:41:40,670 --> 00:41:47,760 they have access to the microphone. So this is very this is very real and it's 426 00:41:47,760 --> 00:41:52,870 not going to be resolved even if we had access only to the ultrasound spectrum. 427 00:41:52,870 --> 00:41:57,359 What we're saying is basically, if we only had access to the ultrasound spectrum, 428 00:41:57,359 --> 00:42:04,820 you're still uhm you are still vulnerable to these attacks unless you incorporate 429 00:42:04,820 --> 00:42:10,420 some crypto mechanisms that prevent these things from happening. Is this your 430 00:42:10,420 --> 00:42:15,900 question or? Mic: Um, well, I can still pull off the 431 00:42:15,900 --> 00:42:19,350 same attack if I don't use ultrasound right? 432 00:42:19,350 --> 00:42:21,540 Vasilios: Through the audible spectrum? Mic: Yes, 433 00:42:21,540 --> 00:42:28,990 Vasilios: You can absolutely do. There is one company doing tracking in the audible 434 00:42:28,990 --> 00:42:35,560 spectrum. This is much harder to mitigate. We're looking into it about ways, but 435 00:42:35,560 --> 00:42:39,109 there are so many ways to incorporate beacons in the audible spectrum. The thing 436 00:42:39,109 --> 00:42:47,240 is that there is not much of an ecosystem in this area right now that so you don't 437 00:42:47,240 --> 00:42:52,640 have lots of frameworks are there as many as you have for ultrasounds. 438 00:42:52,640 --> 00:42:56,219 Mic: Thank you. Herald: Our next question will be from 439 00:42:56,219 --> 00:43:01,349 the Internet via our signal angel Signal Angel: $Username is asking, have 440 00:43:01,349 --> 00:43:08,170 you heard about exploiting parricide ultrasound emiters like IC component's? 441 00:43:08,170 --> 00:43:10,230 Vasilios: Can you please repeat the question? 442 00:43:10,230 --> 00:43:14,600 Signal Angel: Yes, sure. The question is, can you use other components on the main 443 00:43:14,600 --> 00:43:23,740 board or maybe the hard disk to emit ultrasounds and then broadcast the beacon 444 00:43:23,740 --> 00:43:28,960 via this? Vailios: Uh. So that's a very that's a 445 00:43:28,960 --> 00:43:35,450 very good question. The answer is I don't know, possibly, and it's very scary. Um, 446 00:43:35,450 --> 00:43:42,489 hopefully not, but I doubt it. I think there should be a way to do it. Um, maybe 447 00:43:42,489 --> 00:43:47,200 the problem is that you cannot do this completely in a completely inaudible way. 448 00:43:47,200 --> 00:43:51,760 Like you may be able to meet ultrasounds, but you will also emit some sort of sound 449 00:43:51,760 --> 00:43:58,010 in the audible spectrum so that the user will know that something is going on. 450 00:43:58,010 --> 00:44:02,520 Herald: The next question from the left microphone. 451 00:44:02,520 --> 00:44:06,559 Mic: Thank you for your talk and especially thanks for the research. So, 452 00:44:06,559 --> 00:44:12,919 uh, do you know of any framework's or, uh, STKs that cash the beacon's they find? 453 00:44:12,919 --> 00:44:17,760 Because for my use case, I my phone was mostly offline. I just make it online when 454 00:44:17,760 --> 00:44:21,950 I have to check something. So I'm not that concerned. But 455 00:44:21,950 --> 00:44:24,660 you do you know, if they like cash the beacons and and submit them later 456 00:44:24,660 --> 00:44:32,250 something like this. Of course they do. I'm not surprised, unfortunately. Yeah. 457 00:44:32,250 --> 00:44:39,450 Thanks. Next question from the rear. Right. Oh, what is the data rate? You can 458 00:44:39,450 --> 00:44:44,119 send in the ultrasound. Very good question. And it's totally relevant to the 459 00:44:44,119 --> 00:44:51,250 cryptographic mechanisms we want to incorporate from our experiments. Um, in 460 00:44:51,250 --> 00:44:58,480 four seconds you can basically send like five to six alphabet characters if you're 461 00:44:58,480 --> 00:45:04,500 willing to kind of reduce the range a lot less in less than seven meters, you may be 462 00:45:04,500 --> 00:45:11,970 able to send more. But the standard is not very robust in this sense. But these 463 00:45:11,970 --> 00:45:16,260 experiments were done with this kind of naive encoding that most of the companies 464 00:45:16,260 --> 00:45:22,930 are using. So if you do the encoding in a very smart way, possibly you can increase 465 00:45:22,930 --> 00:45:29,329 that. And a small second part, what's the energy consumption on the phone if that is 466 00:45:29,329 --> 00:45:35,110 running all the time? Wouldn't I detect that? So it's not, uh, it's not good. We 467 00:45:35,110 --> 00:45:38,890 saw that it was draining the battery and actually in the comments, I don't know if 468 00:45:38,890 --> 00:45:44,500 I had that comment here. Some people were complaining that, um, I tried and it was 469 00:45:44,500 --> 00:45:53,029 draining my battery. And, um, there is an impact. Absolutely. Amazon and Google Nest 470 00:45:53,029 --> 00:45:57,710 and all the other parts, aren't you more worried about that? You know, the always 471 00:45:57,710 --> 00:46:02,400 listening thing from Google and Amazon and everyone is coming up with some something 472 00:46:02,400 --> 00:46:10,130 like that that's always on. And so that it's kind of strange because a user's 473 00:46:10,130 --> 00:46:18,369 consent. But at the same time, they don't completely understand. So there is a gray 474 00:46:18,369 --> 00:46:22,670 line there, like you can say that the users, OK, you consented to that up, 475 00:46:22,670 --> 00:46:28,549 starting with your with your phone and listening on the background. But at the 476 00:46:28,549 --> 00:46:34,869 same time, the users don't have the best understanding. Always. Thank you. Next 477 00:46:34,869 --> 00:46:39,430 question from the front left microphone first. Thank you for the talk. I would be 478 00:46:39,430 --> 00:46:43,809 interested in how you selected your real world applications and how many you found 479 00:46:43,809 --> 00:46:51,119 that already use such a framework. So what was the first part of the question, how 480 00:46:51,119 --> 00:46:56,790 you selected your real world applications from the marketplace staff if you had any. 481 00:46:56,790 --> 00:47:04,109 So we're trying to do a systematic scan of the whole market, but it's not easy. So we 482 00:47:04,109 --> 00:47:09,440 not able to do that. There are resources on the Internet. Luckily, the companies 483 00:47:09,440 --> 00:47:15,710 need to advertise their product. So they basically publish press releases saying, 484 00:47:15,710 --> 00:47:22,000 you know, this NBA team started using our product. We did some sort of scanning 485 00:47:22,000 --> 00:47:27,890 through alternative datasets, but definitely we don't have an exhaustive 486 00:47:27,890 --> 00:47:33,049 list of applications. What I can say, though, is that there are applications 487 00:47:33,049 --> 00:47:40,250 with. Using such frameworks with nearly up to, if I remember correctly, up to one 488 00:47:40,250 --> 00:47:49,160 million installations. One notable example, OK, I'm not entirely sure what I 489 00:47:49,160 --> 00:47:55,380 wanted, but up to a million we definitely saw. OK, thanks. Do we have more questions 490 00:47:55,380 --> 00:48:02,500 from the Internet? Yes, E.F. is asking, is he aware of or are you aware sorry? Are 491 00:48:02,500 --> 00:48:05,569 you aware of any framework available by Google or Apple? In other words, how do we 492 00:48:05,569 --> 00:48:11,960 know that it's not, for instance, seriously snitching on us? How do we know 493 00:48:11,960 --> 00:48:19,910 that it's not true? It's not serious. Some maybe Aleksa snitching on us. We don't. I 494 00:48:19,910 --> 00:48:24,160 think that's a that's a very large discussion. Right. So is the same problem 495 00:48:24,160 --> 00:48:34,059 that these companies are having? Because if I go back here, basically the users are 496 00:48:34,059 --> 00:48:43,690 accusing them of eavesdropping. Especially here from reverse engineering those 497 00:48:43,690 --> 00:48:49,869 frameworks, we couldn't find any such activity, but again, it's very hard to 498 00:48:49,869 --> 00:48:54,259 convince the users that you are listening to the ultrasound spectrum. You if you're 499 00:48:54,259 --> 00:48:59,769 accessing the whole audible frequencies through the microphone, you're going to or 500 00:48:59,769 --> 00:49:04,119 you will always find yourself in this position. So I guess it's the same problem 501 00:49:04,119 --> 00:49:09,339 that Alexa has from Amazon. But in this case, you can actually solve it by 502 00:49:09,339 --> 00:49:15,410 constraining the spectrum that you gain access to. Next question from the front 503 00:49:15,410 --> 00:49:21,069 left microphone, please. Has anybody done an audible demonstration off these beacons 504 00:49:21,069 --> 00:49:26,230 bypassed by transposing them down an octave or two, I think might be useful for 505 00:49:26,230 --> 00:49:34,089 for or your talk to something like that. So you mean a demo, but using audible 506 00:49:34,089 --> 00:49:40,630 frequencies? Essentially, there is this one company, but they are being pretty to 507 00:49:40,630 --> 00:49:44,869 all of these companies are being pretty secretive with their technology. So they 508 00:49:44,869 --> 00:49:51,430 publish what's needed for marketing purposes like accuracy sometimes remains 509 00:49:51,430 --> 00:49:57,390 very limited technical details. But apart from these, you have to get your hands on 510 00:49:57,390 --> 00:50:04,829 the framework somehow and analyze it yourself. So in this kind of overview we 511 00:50:04,829 --> 00:50:08,130 need for the ecosystem, we had to do everything by ourselves. There was no 512 00:50:08,130 --> 00:50:15,789 resources out there were very limited, um, or recording it and playing it down and 513 00:50:15,789 --> 00:50:23,290 transposing it yourself, if you know where as a beacon of. Possibly I'm not I'm not 514 00:50:23,290 --> 00:50:31,779 entirely sure you could. Yeah. Another question from our signal, angel mestas, 515 00:50:31,779 --> 00:50:37,789 again asking, um, would it be possible, even if you have a low pass filter to use, 516 00:50:37,789 --> 00:50:44,810 uh, for instance, the cost effect and high cost effect to transmit the beacon via 517 00:50:44,810 --> 00:50:53,900 ultrasound, but in a regime which is as free for the app? So it's basically the 518 00:50:53,900 --> 00:50:59,799 question, can I somehow, via Aliasing USA address on signal to make a normal signal 519 00:50:59,799 --> 00:51:08,319 out of it? Possibly, I don't know. I think you are much more creative than I am, so 520 00:51:08,319 --> 00:51:16,819 maybe I should add more bullet points on this controversialist here. Apparently, 521 00:51:16,819 --> 00:51:23,150 there are many more ways to do this, possibly like hardware missions. This one 522 00:51:23,150 --> 00:51:29,619 sounds like a good idea, too. So next question from the real right microphone. I 523 00:51:29,619 --> 00:51:33,559 apologize if you explain the story they didn't understand, but is is sort of 524 00:51:33,559 --> 00:51:38,819 drowning out the signals, like jamming. They just broadcasting white noise in that 525 00:51:38,819 --> 00:51:43,810 spectrum, an effective countermeasure. And as a follow up, if it is, would it 526 00:51:43,810 --> 00:51:56,750 terrorize my dog? So absolutely, it's effective. I mean, this it works up to 527 00:51:56,750 --> 00:52:01,770 seven meters, but we're not saying it's not fragile, so you can do that, but it's 528 00:52:01,770 --> 00:52:05,829 noise pollution. And my dog, I don't think it was happy. I did it for a very limited 529 00:52:05,829 --> 00:52:10,280 time. I could see her ears moving, but I don't think she would appreciate it if I 530 00:52:10,280 --> 00:52:16,720 had the device at home doing this all the time. Do we have any more questions from 531 00:52:16,720 --> 00:52:22,460 the Internet? Yes, EULEX is asking to what extent could we use these for our own 532 00:52:22,460 --> 00:52:26,559 needs? For example, people in repressive situations, for example, activists could 533 00:52:26,559 --> 00:52:30,630 use it to transmit secret encrypted messages. Are there any efforts in this 534 00:52:30,630 --> 00:52:40,829 area? Yes, there are. People are developing ultrasound modems. I think 535 00:52:40,829 --> 00:52:51,030 there is even a tag on it. And yes, of course there is. So I would say, yes, I'm 536 00:52:51,030 --> 00:52:57,029 not entirely sure about the capabilities of this channel in terms of bandwidth, but 537 00:52:57,029 --> 00:53:01,890 this is why we we are not advocating to kill the technology just to make it secure 538 00:53:01,890 --> 00:53:06,900 and know its limitations. So you can do good stuff with it. And this is what we 539 00:53:06,900 --> 00:53:13,720 want. Next question from the Rio, right? Yeah, I'm wondering if you could transfer 540 00:53:13,720 --> 00:53:19,859 that technique from the ultrasound range also to the Audible Range, for example, by 541 00:53:19,859 --> 00:53:26,550 using watermarks, audio, watermarks, and then, well, your permission thingy with 542 00:53:26,550 --> 00:53:31,740 the ultrasound permissions would be ineffective and you could also track the 543 00:53:31,740 --> 00:53:37,810 user. How about this? Is it possible audio watermarks in the audible spectrum? Yeah, 544 00:53:37,810 --> 00:53:42,900 it's absolutely possible. Um, our countermeasures are not effective against 545 00:53:42,900 --> 00:53:50,490 this. Um, it's just that there is from our research, just one company doing this. Uh, 546 00:53:50,490 --> 00:53:57,119 so this one, um, I think technically it's a bit more challenging to do that. 547 00:53:57,119 --> 00:54:02,809 Instead, they're just admitting they are doing it in a very basic way. So 548 00:54:02,809 --> 00:54:08,480 hopefully, um, if there is a clear way to do it through ultrasounds, they are not 549 00:54:08,480 --> 00:54:15,400 going to reside reside in the audible spectrum. But our countermeasures are not 550 00:54:15,400 --> 00:54:22,640 effective against the audible. Um. Watermarks. Yeah, thanks, next question 551 00:54:22,640 --> 00:54:28,960 from the front left microphone. I've heard that I don't think it's very credible, but 552 00:54:28,960 --> 00:54:34,079 I've heard that there is some sound on this sub sound spectrum. There were some 553 00:54:34,079 --> 00:54:40,700 experiments showing that they can influence our mood, the mood of humans. Is 554 00:54:40,700 --> 00:54:47,900 there any relevant information about how ultrasounds could affect us? So without 555 00:54:47,900 --> 00:54:54,580 being an expert in this particular area? I've read similar articles when I was 556 00:54:54,580 --> 00:54:59,190 looking into it. I can tell you it's very annoying, especially if you're listening 557 00:54:59,190 --> 00:55:05,680 to it through headphones. You cannot really hear the sound, but you can if 558 00:55:05,680 --> 00:55:11,599 you're using headphones, you can feel the pressure. So if I don't know what kind of 559 00:55:11,599 --> 00:55:19,809 medical condition you may develop, but you won't be very sane after. Do we have any 560 00:55:19,809 --> 00:55:27,289 more questions? Yes. One further question, um, would it be possible to, um, use a 561 00:55:27,289 --> 00:55:33,999 charming solution to get rid of the signals? Yes, but you you're going to 562 00:55:33,999 --> 00:55:38,450 follow the you know, it's going to result in noise pollution, but if you are being 563 00:55:38,450 --> 00:55:46,690 paranoid about it, yes, it's and it's, I think, a straightforward thing to do. Any 564 00:55:46,690 --> 00:55:53,330 more questions? One more on the front left microphone. Know, you said that physical 565 00:55:53,330 --> 00:55:59,049 objects will block the ultrasound. How solid do the physical objects need to be? 566 00:55:59,049 --> 00:56:04,680 So, for example, does my pocket block the ultrasound and thus prevent my phone to 567 00:56:04,680 --> 00:56:11,579 call the environment and vice versa? OK, well, that's a good question. I don't 568 00:56:11,579 --> 00:56:16,529 think that clothes can actually do that unless it's very thick. Thin girls 569 00:56:16,529 --> 00:56:27,190 definitely block it. Um. Thick glass, I would say it reduce the transmission rate, 570 00:56:27,190 --> 00:56:35,559 the signal to noise ratio by a lot, but it could go through it, so. You need 571 00:56:35,559 --> 00:56:42,690 something quite concrete, metal. I don't think it goes through it. So are there any 572 00:56:42,690 --> 00:56:48,160 more? Doesn't look like it, maybe, maybe one more sorry. Oh, good signal, good bye. 573 00:56:48,160 --> 00:57:02,350 Kitty is asking, could you name or compile a list of tracking programs and apps? So. 574 00:57:02,350 --> 00:57:07,410 That's a good question. We're trying to make an exhaustive list and try to resolve 575 00:57:07,410 --> 00:57:16,529 this in a systematic way. I've already listed two Macenta frameworks. One is the 576 00:57:16,529 --> 00:57:20,160 Silverbush one three actually. One is the Silver Paswan. There is another one used 577 00:57:20,160 --> 00:57:32,940 by single 360. So developed the signal 360, and then there is a listener one. 578 00:57:32,940 --> 00:57:39,609 These are very popular. Um, and then its developer is incorporating them into their 579 00:57:39,609 --> 00:57:48,749 applications in different ways, offering varying levels of transparency for the 580 00:57:48,749 --> 00:57:54,339 users. So it's better if you start knowing what the frameworks are and then trying to 581 00:57:54,339 --> 00:57:59,039 find the applications using them, because you know what? You're looking in the code 582 00:57:59,039 --> 00:58:06,280 and you can develop some queries and enabling you to access an ability to to 583 00:58:06,280 --> 00:58:13,509 track which applications are using them. What what we observed for Silverbush is 584 00:58:13,509 --> 00:58:18,820 basically after the company announced that they are moving out of the US and because 585 00:58:18,820 --> 00:58:24,390 of the whole backslash, maybe even before that, um, companies started to drop the 586 00:58:24,390 --> 00:58:30,109 framework. So all their versions had the framework, but they are not using it 587 00:58:30,109 --> 00:58:52,549 anymore. I think that's it. Thank you very much, Vasilios Lovelady's. 588 00:58:52,549 --> 00:59:02,932 Subtitles created by c3subtitles.de in the year 2021. Join, and help us!