0 00:00:00,000 --> 00:00:30,000 Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/421 Thanks! 1 00:00:09,130 --> 00:00:11,469 So my name is 2 00:00:11,470 --> 00:00:13,689 in the Herald and 3 00:00:13,690 --> 00:00:15,479 I will give you a little introduction to 4 00:00:15,480 --> 00:00:17,619 Kay Kunstler, who is 5 00:00:17,620 --> 00:00:20,139 from who's currently working in Japan 6 00:00:20,140 --> 00:00:22,209 at, uh, Kiu, 7 00:00:22,210 --> 00:00:24,309 and he is 8 00:00:24,310 --> 00:00:26,889 going to give a talk about, 9 00:00:26,890 --> 00:00:29,739 um, about eyewear computing. 10 00:00:29,740 --> 00:00:31,419 And this is a really interesting topic 11 00:00:31,420 --> 00:00:33,519 because our human brain is not 12 00:00:33,520 --> 00:00:35,919 just limited for specific 13 00:00:35,920 --> 00:00:38,079 visual processing, but it's actually 14 00:00:38,080 --> 00:00:40,209 kind of, um, you could 15 00:00:40,210 --> 00:00:42,279 say it is doing things 16 00:00:42,280 --> 00:00:44,379 on its own. It is processing things on 17 00:00:44,380 --> 00:00:46,539 its own. And its aim in 18 00:00:46,540 --> 00:00:49,209 the end is to enhance 19 00:00:49,210 --> 00:00:51,309 to maybe enhance the computing 20 00:00:51,310 --> 00:00:53,559 we do by 21 00:00:53,560 --> 00:00:56,019 specific devices, such as, 22 00:00:56,020 --> 00:00:59,109 for example, eyewear. 23 00:00:59,110 --> 00:01:01,209 And he's going to talk 24 00:01:01,210 --> 00:01:03,759 about the beginnings of that. 25 00:01:03,760 --> 00:01:05,859 And so give a 26 00:01:05,860 --> 00:01:08,319 big round of applause for high concer. 27 00:01:14,290 --> 00:01:14,979 Thank you. 28 00:01:14,980 --> 00:01:17,639 Thank you very much, Ystem, 29 00:01:17,640 --> 00:01:18,729 the mic is on. 30 00:01:18,730 --> 00:01:20,830 Um, can I get my slides? 31 00:01:24,410 --> 00:01:25,909 So for me, it's always. 32 00:01:29,730 --> 00:01:31,829 You know, I give a lot of talks, but for 33 00:01:31,830 --> 00:01:34,769 me, being here at the 34 00:01:34,770 --> 00:01:36,569 Chaos Communication Congress is always 35 00:01:36,570 --> 00:01:39,059 special. So I'm also a little bit nervous 36 00:01:39,060 --> 00:01:41,489 and I hope that I won't be wasting 37 00:01:41,490 --> 00:01:43,739 your time for the next 25 minutes 38 00:01:43,740 --> 00:01:45,809 or so on. So the topic I want to talk 39 00:01:45,810 --> 00:01:48,239 to you about today is where computing 40 00:01:48,240 --> 00:01:50,369 augmenting the human mind, as mentioned, 41 00:01:50,370 --> 00:01:52,739 and working at Kayoko Immediate Design 42 00:01:52,740 --> 00:01:55,079 University in Yokohama, Japan. 43 00:01:55,080 --> 00:01:56,999 And before I start to give you an 44 00:01:57,000 --> 00:01:59,009 overview, I'll show you a quick demo, 45 00:01:59,010 --> 00:02:01,049 because before and I had some problems 46 00:02:01,050 --> 00:02:02,459 with Bluetooth in this room. 47 00:02:02,460 --> 00:02:04,809 So I think now the demo works. 48 00:02:04,810 --> 00:02:06,869 So what you see 49 00:02:06,870 --> 00:02:08,939 here, I'm currently wearing 50 00:02:08,940 --> 00:02:11,489 prototype from Ginns, from Japanese 51 00:02:11,490 --> 00:02:13,619 glasses maker, and 52 00:02:13,620 --> 00:02:15,629 it uses electrodes to measure my eye 53 00:02:15,630 --> 00:02:18,389 movements. And you see my life blinks. 54 00:02:18,390 --> 00:02:20,729 It also has an accelerometer inside. 55 00:02:20,730 --> 00:02:22,859 So you can get also the motion 56 00:02:22,860 --> 00:02:23,860 of the head. 57 00:02:24,930 --> 00:02:27,209 So I'll go into details how this 58 00:02:27,210 --> 00:02:29,459 thing works and what you can do with 59 00:02:29,460 --> 00:02:30,460 it also. 60 00:02:31,230 --> 00:02:33,449 But because the demo worked right 61 00:02:33,450 --> 00:02:35,309 now, I thought it's good to start with 62 00:02:35,310 --> 00:02:36,310 it. 63 00:02:37,140 --> 00:02:38,140 And 64 00:02:39,600 --> 00:02:41,699 so let me give you an overview over the 65 00:02:41,700 --> 00:02:43,799 top. OK, so first I give you a 66 00:02:43,800 --> 00:02:45,749 little bit of background about myself, 67 00:02:45,750 --> 00:02:47,819 then I'll go into details why I think 68 00:02:47,820 --> 00:02:50,099 I where computing is is interesting 69 00:02:50,100 --> 00:02:51,719 or something to explore. 70 00:02:51,720 --> 00:02:53,939 And then I'll talk about how we can 71 00:02:53,940 --> 00:02:56,129 make it accessible so we can use 72 00:02:56,130 --> 00:02:58,619 it every day. And normal people, not 73 00:02:58,620 --> 00:03:01,229 only as geeks can use it, especially 74 00:03:01,230 --> 00:03:02,999 enabling technologies like 75 00:03:03,000 --> 00:03:04,709 electroencephalography. 76 00:03:04,710 --> 00:03:06,929 And then I'll give a little bit 77 00:03:06,930 --> 00:03:08,999 of an outlook on how we 78 00:03:09,000 --> 00:03:11,279 can be able to measure 79 00:03:11,280 --> 00:03:13,349 other cognitive states, something 80 00:03:13,350 --> 00:03:15,569 like cognitive workload or so 81 00:03:15,570 --> 00:03:16,949 on in real life. 82 00:03:16,950 --> 00:03:18,569 But that's more or less in outlook. 83 00:03:18,570 --> 00:03:20,699 And we just started with this work. 84 00:03:20,700 --> 00:03:22,649 So as a background, I worked in wearable 85 00:03:22,650 --> 00:03:23,669 computing. 86 00:03:23,670 --> 00:03:26,099 And this means, you know, I used 87 00:03:26,100 --> 00:03:28,619 mostly motion sensors and other sensors 88 00:03:28,620 --> 00:03:30,329 that put them on users and tried to 89 00:03:30,330 --> 00:03:31,949 figure out what they're doing. 90 00:03:31,950 --> 00:03:34,079 My former supervisor, Paul, you 91 00:03:34,080 --> 00:03:36,330 see him here and in this picture 92 00:03:37,530 --> 00:03:39,419 called the call to the Christmas tree set 93 00:03:39,420 --> 00:03:41,429 up because we put a lot of sensors on the 94 00:03:41,430 --> 00:03:43,199 people and try to figure out what they're 95 00:03:43,200 --> 00:03:45,329 doing and try to support 96 00:03:45,330 --> 00:03:47,549 them during everyday tasks or doing 97 00:03:47,550 --> 00:03:49,829 maintenance work, but also sports. 98 00:03:49,830 --> 00:03:52,079 And a lot of my work was 99 00:03:52,080 --> 00:03:54,179 trying to figure out how we can get from 100 00:03:54,180 --> 00:03:56,309 these dedicated sensors towards 101 00:03:56,310 --> 00:03:58,719 something you can wear with 102 00:03:58,720 --> 00:04:00,629 your smartphone or sensors that are 103 00:04:00,630 --> 00:04:03,329 embedded in everyday government. 104 00:04:03,330 --> 00:04:05,399 And then since the last 105 00:04:05,400 --> 00:04:06,659 two years, that got more and more 106 00:04:06,660 --> 00:04:09,089 interested in what's going on in 107 00:04:09,090 --> 00:04:09,929 our mind. 108 00:04:09,930 --> 00:04:12,209 So can I also detect 109 00:04:12,210 --> 00:04:14,700 cognitive state or cognitive activities? 110 00:04:15,960 --> 00:04:17,819 In this case? 111 00:04:17,820 --> 00:04:20,189 The eyes got more and more interesting. 112 00:04:20,190 --> 00:04:21,989 You know, I started with Ichi and I 113 00:04:21,990 --> 00:04:23,999 found, you know, I'm a little bit lazy, 114 00:04:24,000 --> 00:04:26,249 so I found it too difficult 115 00:04:26,250 --> 00:04:28,859 to make sense out of EEG signals. 116 00:04:28,860 --> 00:04:31,229 And the eyes were another way 117 00:04:31,230 --> 00:04:34,319 to observe the mind because, 118 00:04:34,320 --> 00:04:35,429 you know, there are or 119 00:04:36,480 --> 00:04:37,829 we use them every day. 120 00:04:37,830 --> 00:04:39,899 We use them even while we are 121 00:04:39,900 --> 00:04:41,849 sleeping, even when we have our eyes 122 00:04:41,850 --> 00:04:44,099 closed, our eyes closed, they're 123 00:04:44,100 --> 00:04:45,419 still moving. 124 00:04:45,420 --> 00:04:47,879 And there's a lot of work in psychology 125 00:04:47,880 --> 00:04:49,799 and cognitive science and other related 126 00:04:49,800 --> 00:04:52,679 fields that link eye movement 127 00:04:52,680 --> 00:04:55,499 to something like attention, 128 00:04:55,500 --> 00:04:57,719 concentration, intention 129 00:04:57,720 --> 00:04:59,459 and higher cognitive tasks. 130 00:05:00,990 --> 00:05:03,479 But unfortunately, in most 131 00:05:03,480 --> 00:05:05,909 of the eyes, today is used 132 00:05:05,910 --> 00:05:07,829 for something like this or for 133 00:05:07,830 --> 00:05:10,259 entertainment or 134 00:05:10,260 --> 00:05:11,999 marketing. And now that we have mobile 135 00:05:12,000 --> 00:05:14,009 eye trackers, you know, you also do this 136 00:05:14,010 --> 00:05:16,109 in supermarkets and figure out 137 00:05:16,110 --> 00:05:18,149 how your product must look like that. 138 00:05:18,150 --> 00:05:19,409 People will pick it up. 139 00:05:19,410 --> 00:05:20,759 And I think that's a waste. 140 00:05:20,760 --> 00:05:22,779 And that's a little bit sad. 141 00:05:22,780 --> 00:05:23,780 Um. 142 00:05:29,470 --> 00:05:31,989 So last year, 143 00:05:31,990 --> 00:05:34,479 I gave a talk about tracking 144 00:05:34,480 --> 00:05:35,829 reading habits. 145 00:05:35,830 --> 00:05:38,229 So in this case, we used 146 00:05:38,230 --> 00:05:40,329 an mobile eye tracker from some 147 00:05:40,330 --> 00:05:42,669 eye. This is also an optical system 148 00:05:42,670 --> 00:05:44,109 using infrared lights. 149 00:05:44,110 --> 00:05:47,079 You see on the top the view on the eyes 150 00:05:47,080 --> 00:05:49,149 and infrared cameras underneath 151 00:05:49,150 --> 00:05:51,369 your eyes to detect your 152 00:05:51,370 --> 00:05:53,649 pupils. And with this, you can recognize 153 00:05:53,650 --> 00:05:55,749 your gaze and you get suckered so 154 00:05:55,750 --> 00:05:58,449 fast eye movements and fixations out. 155 00:05:58,450 --> 00:06:00,699 And with this, we can recognize 156 00:06:00,700 --> 00:06:02,829 reading and we can tell how many 157 00:06:02,830 --> 00:06:04,119 words you read. 158 00:06:04,120 --> 00:06:06,219 And not only this, but we can also 159 00:06:06,220 --> 00:06:08,619 figure out what kind of document types 160 00:06:08,620 --> 00:06:10,149 you're reading just by looking at the 161 00:06:10,150 --> 00:06:11,439 visual behavior. 162 00:06:11,440 --> 00:06:13,089 And that's important for Japanese 163 00:06:13,090 --> 00:06:15,609 students. We can distinguish 164 00:06:15,610 --> 00:06:18,009 science books versus mangas 165 00:06:18,010 --> 00:06:19,010 and 166 00:06:21,190 --> 00:06:23,589 and also we try or we slowly 167 00:06:23,590 --> 00:06:25,659 try to assess how much she understands 168 00:06:25,660 --> 00:06:27,909 or comprehension. But it's trickier or 169 00:06:27,910 --> 00:06:29,329 more difficult. 170 00:06:29,330 --> 00:06:30,909 That was last year. 171 00:06:30,910 --> 00:06:33,279 And you could already build 172 00:06:33,280 --> 00:06:34,989 some kind of Fitbit for the mind. 173 00:06:34,990 --> 00:06:36,669 I'm not really sure if you want to have 174 00:06:36,670 --> 00:06:38,589 this. You know, you could get your 175 00:06:38,590 --> 00:06:40,209 reading count during the day. 176 00:06:40,210 --> 00:06:42,519 How many words with what speed 177 00:06:42,520 --> 00:06:44,719 you're reading, how many Mangus 178 00:06:44,720 --> 00:06:46,419 versus science papers. 179 00:06:46,420 --> 00:06:48,729 And but I'm wondering myself, 180 00:06:48,730 --> 00:06:51,009 you know, what a healthy reading habits. 181 00:06:51,010 --> 00:06:53,359 How can I improve my reading habits? 182 00:06:53,360 --> 00:06:55,599 What happens if I copy 183 00:06:55,600 --> 00:06:57,819 somebody is reading over 184 00:06:57,820 --> 00:06:58,989 two or three years. 185 00:06:58,990 --> 00:07:00,939 Will this make me an expert in a certain 186 00:07:00,940 --> 00:07:01,839 field? 187 00:07:01,840 --> 00:07:04,089 The biggest problem I see right now 188 00:07:04,090 --> 00:07:06,309 with this approach using the technology 189 00:07:06,310 --> 00:07:08,559 I introduced last year is 190 00:07:08,560 --> 00:07:10,269 first of all, I mean, this just works for 191 00:07:10,270 --> 00:07:11,259 reading. 192 00:07:11,260 --> 00:07:13,239 So I would like also to have something 193 00:07:13,240 --> 00:07:15,309 more channels. So focusing on 194 00:07:15,310 --> 00:07:18,009 attention, concentration, focus 195 00:07:18,010 --> 00:07:19,659 in in real life settings. 196 00:07:19,660 --> 00:07:21,789 And the other thing is it should work 197 00:07:21,790 --> 00:07:23,709 during every day. 198 00:07:23,710 --> 00:07:25,509 And the biggest problem there is the 199 00:07:25,510 --> 00:07:27,919 mobile eye tracker you see over here. 200 00:07:27,920 --> 00:07:29,679 I mean, it's not something you would like 201 00:07:29,680 --> 00:07:31,779 to wear and 202 00:07:31,780 --> 00:07:33,799 maybe some people would like to wear it 203 00:07:33,800 --> 00:07:35,919 first. And then the next problem is then 204 00:07:35,920 --> 00:07:38,079 also the battery power or, you know, 205 00:07:38,080 --> 00:07:39,009 the device is not. 206 00:07:39,010 --> 00:07:41,109 So you can maybe record 207 00:07:41,110 --> 00:07:43,849 four hours or five hours in one go. 208 00:07:43,850 --> 00:07:46,360 And so we focused already on 209 00:07:47,410 --> 00:07:48,999 tablets and smartphones. 210 00:07:49,000 --> 00:07:51,069 But then the problem comes that, OK, 211 00:07:51,070 --> 00:07:53,709 if you turn this tablet of 212 00:07:53,710 --> 00:07:55,569 it's not working anymore, so you don't 213 00:07:55,570 --> 00:07:57,009 have a coverage of the whole day. 214 00:07:57,010 --> 00:07:59,109 So we try to implement eye tracking on 215 00:07:59,110 --> 00:08:00,969 these devices. It also doesn't work so 216 00:08:00,970 --> 00:08:02,679 well with a front facing camera. 217 00:08:02,680 --> 00:08:05,709 The computer vision is quite complicated, 218 00:08:05,710 --> 00:08:07,899 but we did also some work on on Google 219 00:08:07,900 --> 00:08:10,869 Glass, although, you know, it's not 220 00:08:10,870 --> 00:08:13,479 maybe socially acceptable everywhere. 221 00:08:13,480 --> 00:08:15,639 There are more problems in Germany if 222 00:08:15,640 --> 00:08:17,349 you walk around with Google Glass than 223 00:08:17,350 --> 00:08:18,669 they are in Japan. 224 00:08:18,670 --> 00:08:20,259 So it depends also a little bit on the 225 00:08:20,260 --> 00:08:22,419 cultural background, but you can do some 226 00:08:22,420 --> 00:08:24,339 stuff with it. And now I want to show a 227 00:08:24,340 --> 00:08:26,559 demo a little 228 00:08:26,560 --> 00:08:28,839 bit, an extension from last year's work 229 00:08:28,840 --> 00:08:30,339 on glass. 230 00:08:30,340 --> 00:08:32,529 So basically 231 00:08:32,530 --> 00:08:34,599 Glass doesn't have an eye tracker, but it 232 00:08:34,600 --> 00:08:37,689 has a small infrared sensor 233 00:08:37,690 --> 00:08:39,999 over here that's 234 00:08:40,000 --> 00:08:42,158 used to detect if you wear the device 235 00:08:42,159 --> 00:08:44,349 or not. And also you 236 00:08:44,350 --> 00:08:46,929 can have a long blink to take pictures, 237 00:08:46,930 --> 00:08:48,040 which are fun. Yeah. 238 00:08:50,090 --> 00:08:52,299 Uh, so what you see 239 00:08:52,300 --> 00:08:54,429 right now is the picture I see on 240 00:08:54,430 --> 00:08:56,169 the head mounted display in glass. 241 00:08:56,170 --> 00:08:57,879 And, you know, you have this touchpad on 242 00:08:57,880 --> 00:08:58,389 the side. 243 00:08:58,390 --> 00:09:00,999 So I can see 244 00:09:01,000 --> 00:09:02,259 all pictures I took. 245 00:09:02,260 --> 00:09:03,999 I taped over the camera. 246 00:09:04,000 --> 00:09:05,919 So if I take a picture right now. 247 00:09:08,380 --> 00:09:10,389 You should see it just shows black and a 248 00:09:10,390 --> 00:09:11,859 little white dot, because that's the 249 00:09:11,860 --> 00:09:13,569 light sensor, otherwise I cannot see 250 00:09:13,570 --> 00:09:14,649 anything on the display. 251 00:09:17,860 --> 00:09:19,960 So what we did with it, 252 00:09:22,810 --> 00:09:24,999 we have an open source class logger that 253 00:09:25,000 --> 00:09:28,059 locks all of the sensors 254 00:09:28,060 --> 00:09:30,399 from the device and we can 255 00:09:30,400 --> 00:09:32,709 also monitor the infrared 256 00:09:32,710 --> 00:09:33,699 sensor. 257 00:09:33,700 --> 00:09:35,919 So this is something I showed 258 00:09:35,920 --> 00:09:38,239 last year already. 259 00:09:38,240 --> 00:09:39,819 And this is a little bit an advanced 260 00:09:39,820 --> 00:09:41,979 version. And you can also recognize 261 00:09:41,980 --> 00:09:44,109 that if I move my pupil and 262 00:09:44,110 --> 00:09:46,299 don't blink. OK, some of the things that 263 00:09:46,300 --> 00:09:48,729 I detected as eye blinks, but you can see 264 00:09:48,730 --> 00:09:51,189 also the light, the infrared 265 00:09:51,190 --> 00:09:53,349 sensor moves so you can detect 266 00:09:53,350 --> 00:09:55,209 something like left and right movement of 267 00:09:55,210 --> 00:09:57,429 the eye from the change of 268 00:09:57,430 --> 00:10:00,129 the distance to glass. 269 00:10:00,130 --> 00:10:01,130 And now. 270 00:10:05,000 --> 00:10:07,159 A little bit more advanced demo, what can 271 00:10:07,160 --> 00:10:07,939 you do with it? 272 00:10:07,940 --> 00:10:10,159 For one, I can do some very limited 273 00:10:10,160 --> 00:10:11,839 activity recognition with it. 274 00:10:11,840 --> 00:10:13,909 So after maybe two or three seconds, it 275 00:10:13,910 --> 00:10:15,709 should show what I'm currently doing. 276 00:10:15,710 --> 00:10:18,439 This is just a two class segmentation, 277 00:10:18,440 --> 00:10:19,669 uh, problem. 278 00:10:19,670 --> 00:10:22,099 So currently it's talking. 279 00:10:22,100 --> 00:10:23,100 And now 280 00:10:24,500 --> 00:10:25,729 if I focus here. 281 00:10:33,990 --> 00:10:36,149 So we're reading this just 282 00:10:36,150 --> 00:10:38,219 uses my head 283 00:10:38,220 --> 00:10:40,559 motion as well as 284 00:10:41,850 --> 00:10:45,149 the eye blinks to detect, 285 00:10:45,150 --> 00:10:47,219 to distinguish talking and reading, 286 00:10:47,220 --> 00:10:49,469 talking works relatively stable 287 00:10:49,470 --> 00:10:51,419 during the day. Reading is a little bit 288 00:10:51,420 --> 00:10:54,119 troublesome with the infrared sensor 289 00:10:54,120 --> 00:10:56,459 because of, 290 00:10:56,460 --> 00:10:57,929 you know, if you're watching a movie or 291 00:10:57,930 --> 00:11:00,329 so on, it might also see talking, 292 00:11:00,330 --> 00:11:02,669 reading, because that's not working 293 00:11:02,670 --> 00:11:03,659 so well. 294 00:11:03,660 --> 00:11:05,749 But other things you can do with 295 00:11:05,750 --> 00:11:07,799 hyperlinks, as I mentioned, one is 296 00:11:07,800 --> 00:11:08,949 recognizing talking. 297 00:11:08,950 --> 00:11:11,129 So this is simple because your 298 00:11:11,130 --> 00:11:13,229 blink frequency will increase usually 299 00:11:13,230 --> 00:11:15,419 double, and 300 00:11:15,420 --> 00:11:17,579 there are also some correlation 301 00:11:17,580 --> 00:11:18,509 with focus. 302 00:11:18,510 --> 00:11:21,089 So usually if you focus on something, 303 00:11:21,090 --> 00:11:23,999 you blink, frequency will decrease 304 00:11:24,000 --> 00:11:25,949 and also with content. 305 00:11:25,950 --> 00:11:28,079 So what I observed is 306 00:11:28,080 --> 00:11:29,759 if you look at people reading from an 307 00:11:29,760 --> 00:11:32,009 eating display, you recognize 308 00:11:32,010 --> 00:11:34,019 that the blink when they turn the page. 309 00:11:36,030 --> 00:11:38,309 And I would like to use also 310 00:11:38,310 --> 00:11:40,019 the change blindness during blink. 311 00:11:40,020 --> 00:11:41,849 I haven't really gotten a good idea what 312 00:11:41,850 --> 00:11:43,859 I want to do with it, but you probably 313 00:11:43,860 --> 00:11:45,929 know that if you have some kind of 314 00:11:45,930 --> 00:11:48,329 stimulus or if you have some 315 00:11:48,330 --> 00:11:50,729 if you blink and somebody changes, 316 00:11:50,730 --> 00:11:52,889 maybe the color of the background 317 00:11:52,890 --> 00:11:54,389 or changes some people in here, it 318 00:11:54,390 --> 00:11:56,369 wouldn't recognize because I blinked in 319 00:11:56,370 --> 00:11:58,259 between. And you could use this, for 320 00:11:58,260 --> 00:12:00,859 example, for a display that changes 321 00:12:00,860 --> 00:12:02,219 just when you blink. 322 00:12:02,220 --> 00:12:04,289 So you wouldn't recognize that 323 00:12:04,290 --> 00:12:06,539 something in your eye, in your view, 324 00:12:06,540 --> 00:12:08,519 changed. But if you want to get the new 325 00:12:08,520 --> 00:12:10,409 information, it's already there. 326 00:12:10,410 --> 00:12:11,909 And some some ideas like this. 327 00:12:11,910 --> 00:12:14,339 And I think it would also be cool for 328 00:12:14,340 --> 00:12:16,859 horror games or interactive movies 329 00:12:16,860 --> 00:12:17,860 or so on. 330 00:12:24,460 --> 00:12:26,349 I think you can really scare somebody 331 00:12:26,350 --> 00:12:28,509 with that and another 332 00:12:28,510 --> 00:12:30,609 idea, something that's actually 333 00:12:30,610 --> 00:12:33,909 also fairly simple is fatigue detection. 334 00:12:33,910 --> 00:12:36,339 So just by the duration of blinks 335 00:12:36,340 --> 00:12:38,499 and the blink frequency again, so if 336 00:12:38,500 --> 00:12:41,139 it gets more, more, higher, 337 00:12:41,140 --> 00:12:43,749 you can detect how tired somebody is 338 00:12:43,750 --> 00:12:46,539 and how tired somebody will become. 339 00:12:46,540 --> 00:12:48,639 And I 340 00:12:48,640 --> 00:12:50,709 had the idea of equipping a batch 341 00:12:50,710 --> 00:12:53,259 of students with just simple 342 00:12:53,260 --> 00:12:55,509 Blinkx detectors for classes 343 00:12:55,510 --> 00:12:56,979 and then have a ranking system of 344 00:12:56,980 --> 00:12:59,499 lectures so you can get 345 00:12:59,500 --> 00:13:01,779 the most boring lecture versus the second 346 00:13:01,780 --> 00:13:03,220 boring lecture and so on. 347 00:13:04,710 --> 00:13:05,710 I tried. 348 00:13:09,770 --> 00:13:11,419 I discussed this with some professor, 349 00:13:11,420 --> 00:13:13,549 says Osaka University, 350 00:13:13,550 --> 00:13:15,689 the university was before hand and 351 00:13:15,690 --> 00:13:18,679 didn't really like this idea too much. 352 00:13:18,680 --> 00:13:20,809 I wonder why maybe we can do something 353 00:13:20,810 --> 00:13:23,059 that came with this and then 354 00:13:23,060 --> 00:13:24,449 other things you can do. 355 00:13:24,450 --> 00:13:26,569 Blinkx, together with emotion, helps 356 00:13:26,570 --> 00:13:28,429 you to distinguish closely related 357 00:13:28,430 --> 00:13:29,389 activities. 358 00:13:29,390 --> 00:13:31,609 In this case, we recorded reading, 359 00:13:31,610 --> 00:13:34,309 watching a video solving Sudoku, puzzle 360 00:13:34,310 --> 00:13:36,529 solving to have some 361 00:13:36,530 --> 00:13:38,779 physical activity and also talking and 362 00:13:38,780 --> 00:13:40,999 just blinking frequency you get already 363 00:13:41,000 --> 00:13:43,309 quite far. You get up to 70 percent 364 00:13:43,310 --> 00:13:44,310 and then if you 365 00:13:45,500 --> 00:13:47,689 include the motions, you get up 366 00:13:47,690 --> 00:13:49,019 to eighty two percent. 367 00:13:50,120 --> 00:13:52,189 However, there are some issues also 368 00:13:52,190 --> 00:13:54,559 with class and even more 369 00:13:54,560 --> 00:13:56,689 with the eye tracking 370 00:13:56,690 --> 00:13:57,739 hardware. 371 00:13:57,740 --> 00:14:00,589 It's first of all power, but also 372 00:14:00,590 --> 00:14:02,509 they don't really look so appealing. 373 00:14:02,510 --> 00:14:04,759 So it's not something somebody will 374 00:14:04,760 --> 00:14:07,609 wear during everyday 375 00:14:07,610 --> 00:14:10,009 tasks. So there are new systems from some 376 00:14:10,010 --> 00:14:12,199 eye and also taulbee 377 00:14:12,200 --> 00:14:14,539 and they look 378 00:14:14,540 --> 00:14:16,969 nicer. But you can already recognize 379 00:14:16,970 --> 00:14:19,099 there are things built by engineers 380 00:14:19,100 --> 00:14:20,989 and researchers for engineers and 381 00:14:20,990 --> 00:14:22,159 researchers. 382 00:14:22,160 --> 00:14:24,439 So for everyday people, 383 00:14:24,440 --> 00:14:26,899 for for normal people, maybe, 384 00:14:26,900 --> 00:14:28,669 uh, it's not possible. 385 00:14:28,670 --> 00:14:30,679 So I was quite happy when Nami sensei 386 00:14:30,680 --> 00:14:32,849 asked me to join for a project with 387 00:14:32,850 --> 00:14:35,059 Jin, uh, Japanese classes 388 00:14:35,060 --> 00:14:37,909 company and they make 389 00:14:37,910 --> 00:14:39,659 prototypes of genes meme. 390 00:14:39,660 --> 00:14:40,999 So the glasses I showed you in the 391 00:14:41,000 --> 00:14:42,919 beginning and they have a very different 392 00:14:42,920 --> 00:14:44,269 idea from Google Glass. 393 00:14:44,270 --> 00:14:47,449 So this is not a full fledged 394 00:14:47,450 --> 00:14:49,859 computer on your head. 395 00:14:49,860 --> 00:14:51,949 It has no display, it has 396 00:14:51,950 --> 00:14:54,649 no camera and 397 00:14:54,650 --> 00:14:56,899 the device is just connected to your 398 00:14:56,900 --> 00:14:58,849 phone. So currently it's just streaming 399 00:14:58,850 --> 00:15:00,439 data to your phone. 400 00:15:00,440 --> 00:15:02,629 One is the electromyography 401 00:15:02,630 --> 00:15:04,309 and the other one is motion sensor. 402 00:15:04,310 --> 00:15:05,929 So with one, you can detect eye 403 00:15:05,930 --> 00:15:08,569 movements, a left, right, up, down 404 00:15:08,570 --> 00:15:10,649 and also eye blinks. 405 00:15:10,650 --> 00:15:12,979 And the other one, accelerometer 406 00:15:12,980 --> 00:15:13,999 and Schriro. 407 00:15:14,000 --> 00:15:16,399 We are directly working with them 408 00:15:16,400 --> 00:15:17,929 in the research department. 409 00:15:17,930 --> 00:15:20,239 And, uh, on one thing, I was also happy. 410 00:15:20,240 --> 00:15:22,549 So on one side, I can work on smart 411 00:15:22,550 --> 00:15:24,409 glasses. On the other side for some 412 00:15:24,410 --> 00:15:26,179 promotional video, I could wear the 413 00:15:26,180 --> 00:15:28,589 optical camouflage from the sensor. 414 00:15:28,590 --> 00:15:30,260 So win win situation. 415 00:15:31,550 --> 00:15:33,649 Uh, yeah. The demo in insert the 416 00:15:33,650 --> 00:15:34,609 demo here. 417 00:15:34,610 --> 00:15:36,739 Uh, so you already saw it. 418 00:15:36,740 --> 00:15:38,989 And now I go into the principles, 419 00:15:38,990 --> 00:15:42,739 how it works. So we use electromyography. 420 00:15:42,740 --> 00:15:45,769 Basically your eye is a deep hole 421 00:15:45,770 --> 00:15:47,449 and it has a positive charge in the 422 00:15:47,450 --> 00:15:49,189 front, the negative in the back. 423 00:15:49,190 --> 00:15:51,319 And if you place electrodes around your 424 00:15:51,320 --> 00:15:53,389 eye, you can measure 425 00:15:53,390 --> 00:15:54,390 a movement. 426 00:15:55,290 --> 00:15:57,319 So the regular set up is like this. 427 00:15:57,320 --> 00:15:59,479 You use the, um, electrode 428 00:15:59,480 --> 00:16:01,699 on the top of your eye and 429 00:16:01,700 --> 00:16:03,889 electrode below your eye to get the 430 00:16:03,890 --> 00:16:06,259 vertical movement and one 431 00:16:06,260 --> 00:16:08,089 left and right to get the horizontal 432 00:16:08,090 --> 00:16:10,549 movement. And usually one reference, one 433 00:16:10,550 --> 00:16:12,769 the advantage compared to 434 00:16:12,770 --> 00:16:14,179 optical eye tracking. 435 00:16:14,180 --> 00:16:16,299 Um, you can run a 436 00:16:16,300 --> 00:16:17,989 very high sampling rates. 437 00:16:17,990 --> 00:16:19,969 You have no problems or not so much 438 00:16:19,970 --> 00:16:21,709 problems with battery power because you 439 00:16:21,710 --> 00:16:23,689 don't need so much processing. 440 00:16:23,690 --> 00:16:25,789 And some of the disadvantages 441 00:16:25,790 --> 00:16:27,979 are, uh, it's just relative 442 00:16:27,980 --> 00:16:30,049 eye movement and it can be 443 00:16:30,050 --> 00:16:30,949 sometimes noisy. 444 00:16:30,950 --> 00:16:33,049 So it depends also on the skin and so 445 00:16:33,050 --> 00:16:34,609 on and on the electrode placement. 446 00:16:34,610 --> 00:16:36,799 Of course, uh, there were some 447 00:16:36,800 --> 00:16:38,179 other people who worked on this. 448 00:16:38,180 --> 00:16:40,339 So you don't really have to use goggles. 449 00:16:40,340 --> 00:16:42,019 You can also use headphones. 450 00:16:42,020 --> 00:16:44,269 So monopolization from DoCoMo 451 00:16:44,270 --> 00:16:47,079 used headphones for that, uh, 452 00:16:47,080 --> 00:16:47,989 are spooling used. 453 00:16:47,990 --> 00:16:50,059 Also this regular set up with 454 00:16:50,060 --> 00:16:52,399 top and bottom for up and down and 455 00:16:52,400 --> 00:16:55,279 left and right here for the 456 00:16:55,280 --> 00:16:57,349 horizontal, um, 457 00:16:57,350 --> 00:16:59,509 potential measurement, what the 458 00:16:59,510 --> 00:17:01,609 chins mean now that, uh, 459 00:17:01,610 --> 00:17:03,949 they integrated it into normal 460 00:17:03,950 --> 00:17:06,858 eyeglasses. So they use a three point, 461 00:17:06,859 --> 00:17:08,838 uh, Yochi here. 462 00:17:08,839 --> 00:17:10,159 You see it in detail. 463 00:17:10,160 --> 00:17:12,199 And if you can go to the GoPro, I can 464 00:17:12,200 --> 00:17:14,328 also show it to you 465 00:17:14,329 --> 00:17:15,329 on the. 466 00:17:17,290 --> 00:17:18,290 Video. 467 00:17:22,450 --> 00:17:23,450 No, 468 00:17:25,540 --> 00:17:27,999 now, OK, so you see, these are the three 469 00:17:28,000 --> 00:17:30,099 electrodes, so use the left and 470 00:17:30,100 --> 00:17:31,819 the right one. 471 00:17:31,820 --> 00:17:33,889 For if I'm not shaking 472 00:17:33,890 --> 00:17:35,389 so much, I can actually also see 473 00:17:35,390 --> 00:17:37,579 something on the left and the right ones 474 00:17:37,580 --> 00:17:39,889 for the horizontal axis and the, 475 00:17:39,890 --> 00:17:42,169 uh, these two then for 476 00:17:42,170 --> 00:17:43,579 the vertical axis. 477 00:17:45,200 --> 00:17:46,849 OK, back to the slides. 478 00:17:49,530 --> 00:17:51,419 Right now, these are just first 479 00:17:51,420 --> 00:17:53,159 prototypes and the battery runtime is 480 00:17:53,160 --> 00:17:55,049 around eight hours streaming data. 481 00:17:55,050 --> 00:17:57,239 If we do, On-Board, processing also will 482 00:17:57,240 --> 00:17:59,219 get even better. So now you might wonder 483 00:17:59,220 --> 00:18:01,109 what you can do with it. 484 00:18:01,110 --> 00:18:03,239 Um, for one, we 485 00:18:03,240 --> 00:18:05,399 can recognize the eye 486 00:18:05,400 --> 00:18:07,049 movements as you see here. 487 00:18:07,050 --> 00:18:09,299 So blinks and left 488 00:18:09,300 --> 00:18:10,300 and right. 489 00:18:16,220 --> 00:18:17,959 This was one of the earlier prototypes, 490 00:18:17,960 --> 00:18:19,669 that's, by the way, ChawIa, he was here 491 00:18:19,670 --> 00:18:21,139 last year. Unfortunately, he couldn't 492 00:18:21,140 --> 00:18:22,789 troyen it this time. 493 00:18:22,790 --> 00:18:24,589 And you know, what is the first thing you 494 00:18:24,590 --> 00:18:25,590 try to implement? 495 00:18:29,330 --> 00:18:31,459 If you have a binary 496 00:18:31,460 --> 00:18:34,249 control system like I blinked 497 00:18:34,250 --> 00:18:36,799 Flappy Bird, this is really, really, 498 00:18:36,800 --> 00:18:37,999 really hard to play. 499 00:18:38,000 --> 00:18:39,920 You also see it in the face of Shroyer 500 00:18:41,780 --> 00:18:43,939 and I 501 00:18:43,940 --> 00:18:46,129 can try it out for later and maybe you 502 00:18:46,130 --> 00:18:48,229 can try it. But getting even a score of 503 00:18:48,230 --> 00:18:49,759 one or two is hard. 504 00:18:49,760 --> 00:18:51,949 What you can also do is, of course, 505 00:18:51,950 --> 00:18:53,749 more interesting activity recognition. 506 00:18:53,750 --> 00:18:56,299 In this case, we can detect reading 507 00:18:56,300 --> 00:18:58,849 and just by time we assign a word 508 00:18:58,850 --> 00:19:01,219 currently. So this is 15 to 20 percent 509 00:19:01,220 --> 00:19:03,079 error. But of course we can analyze the 510 00:19:03,080 --> 00:19:05,149 data further and get better 511 00:19:05,150 --> 00:19:07,219 results. So reading and also the 512 00:19:07,220 --> 00:19:09,409 talking or the head 513 00:19:09,410 --> 00:19:11,359 motion change, as well as the blinking 514 00:19:11,360 --> 00:19:13,489 frequency change easy to 515 00:19:13,490 --> 00:19:15,709 detect. So you can get an overview over 516 00:19:15,710 --> 00:19:18,109 your day how much physical 517 00:19:18,110 --> 00:19:20,179 activity you did, because it also has 518 00:19:20,180 --> 00:19:22,399 an excellent meter inside, but also 519 00:19:22,400 --> 00:19:24,529 social interaction and how 520 00:19:24,530 --> 00:19:26,489 much reading you did. 521 00:19:26,490 --> 00:19:28,579 Now, I think right 522 00:19:28,580 --> 00:19:31,069 now they mostly focus 523 00:19:31,070 --> 00:19:32,149 for the product or so on. 524 00:19:32,150 --> 00:19:34,129 They mostly focus on fatigue detection 525 00:19:34,130 --> 00:19:36,319 and they want to release sometime next 526 00:19:36,320 --> 00:19:37,429 year in September. 527 00:19:37,430 --> 00:19:39,859 But if you're not into consumer products 528 00:19:39,860 --> 00:19:41,569 and so on or don't want to wait till 529 00:19:41,570 --> 00:19:44,059 September and want to build it yourself, 530 00:19:44,060 --> 00:19:45,799 it's not so difficult. 531 00:19:45,800 --> 00:19:48,409 So there are a couple of instructions 532 00:19:48,410 --> 00:19:50,509 online. I was mostly astonished 533 00:19:50,510 --> 00:19:52,909 by the eyeballed by 534 00:19:52,910 --> 00:19:54,709 a Honduran teenager. 535 00:19:55,760 --> 00:19:57,859 And, you know, his his setup really 536 00:19:57,860 --> 00:20:00,229 works. So Musci, a 537 00:20:00,230 --> 00:20:02,629 student of mine, also built one. 538 00:20:02,630 --> 00:20:04,999 He took several of the Dowa 539 00:20:05,000 --> 00:20:07,339 instruction sets and he just uses 540 00:20:07,340 --> 00:20:09,579 two electrodes left and right to detect 541 00:20:09,580 --> 00:20:11,059 the horizontal movement. 542 00:20:11,060 --> 00:20:13,339 But as you see, it works 543 00:20:13,340 --> 00:20:14,399 relatively well. 544 00:20:14,400 --> 00:20:17,059 So even small 545 00:20:17,060 --> 00:20:19,489 eye movements or so on can be detected 546 00:20:19,490 --> 00:20:21,649 with a DIY set. 547 00:20:21,650 --> 00:20:23,879 And now all 548 00:20:23,880 --> 00:20:26,119 we have to hurry. So just five minutes 549 00:20:26,120 --> 00:20:28,499 left before questions 550 00:20:28,500 --> 00:20:29,579 and answers. 551 00:20:29,580 --> 00:20:31,699 And now we are also trying 552 00:20:31,700 --> 00:20:33,769 out other 553 00:20:33,770 --> 00:20:35,929 Yochi set up the electrode 554 00:20:35,930 --> 00:20:38,179 setups. What works best for reading 555 00:20:38,180 --> 00:20:39,560 or for for other 556 00:20:40,970 --> 00:20:42,319 cognitive tasks? 557 00:20:42,320 --> 00:20:44,629 And you might wonder, can we use 558 00:20:44,630 --> 00:20:46,819 Yochi also for 559 00:20:46,820 --> 00:20:48,679 this tracking of reading habits and how 560 00:20:48,680 --> 00:20:50,869 does it compare to the 561 00:20:50,870 --> 00:20:52,429 optical eye tracker? 562 00:20:52,430 --> 00:20:55,129 So in this case, 563 00:20:55,130 --> 00:20:57,319 we use a medical yochi with 564 00:20:57,320 --> 00:20:58,459 active electrodes. 565 00:20:58,460 --> 00:21:00,529 So the one I have equipped 566 00:21:00,530 --> 00:21:02,629 there and just looking at 567 00:21:02,630 --> 00:21:03,920 the horizontal 568 00:21:05,330 --> 00:21:07,969 component, again from the potential, 569 00:21:07,970 --> 00:21:10,459 uh, that's a graph you see here 570 00:21:10,460 --> 00:21:12,289 for me reading. 571 00:21:12,290 --> 00:21:14,509 And I just use a simple peak 572 00:21:14,510 --> 00:21:16,579 detection algorithm to detect that the 573 00:21:16,580 --> 00:21:17,569 line breaks. 574 00:21:17,570 --> 00:21:19,609 So these long peaks are the line breaks. 575 00:21:19,610 --> 00:21:21,739 And interestingly enough, you can also 576 00:21:21,740 --> 00:21:23,429 detect very small line breaks. 577 00:21:23,430 --> 00:21:25,649 So kind of two or three word things. 578 00:21:25,650 --> 00:21:28,009 So these are the not so 579 00:21:28,010 --> 00:21:29,010 deep peaks. 580 00:21:30,250 --> 00:21:32,019 And that's not possible with the optical 581 00:21:32,020 --> 00:21:34,119 eye tracker because the sampling 582 00:21:34,120 --> 00:21:36,309 rate is not high enough, so that's quite 583 00:21:36,310 --> 00:21:38,499 nice. Oh, and by the way, this is 584 00:21:38,500 --> 00:21:40,719 not published yet, but as soon as 585 00:21:40,720 --> 00:21:43,119 it gets hopefully published, 586 00:21:43,120 --> 00:21:45,189 also share the code also on and 587 00:21:45,190 --> 00:21:47,589 share how you can 588 00:21:47,590 --> 00:21:50,649 kind of filter. Also on the single. 589 00:21:50,650 --> 00:21:53,019 And then for the last part, 590 00:21:53,020 --> 00:21:55,119 for the last five minutes, I want to talk 591 00:21:55,120 --> 00:21:57,369 a little bit about how we 592 00:21:57,370 --> 00:21:59,649 can recognize more 593 00:21:59,650 --> 00:22:02,289 general cognitive states. 594 00:22:02,290 --> 00:22:03,459 In this case. 595 00:22:03,460 --> 00:22:05,919 We try to track cognitive 596 00:22:05,920 --> 00:22:08,199 load or brain sensing. 597 00:22:08,200 --> 00:22:10,269 So we used near 598 00:22:10,270 --> 00:22:12,489 the near infrared spectroscopy device 599 00:22:12,490 --> 00:22:14,679 from Ishimatsu Leibniz's and it's 600 00:22:14,680 --> 00:22:17,079 estimates the globin 601 00:22:17,080 --> 00:22:19,359 change in the Blatche, in 602 00:22:19,360 --> 00:22:21,909 this case in the prefrontal cortex. 603 00:22:21,910 --> 00:22:24,369 So the oxygen change 604 00:22:24,370 --> 00:22:26,589 in the blood we used the 605 00:22:26,590 --> 00:22:29,319 Lebanese the genes meme and 606 00:22:29,320 --> 00:22:31,659 an optical eye tracker, stationary 607 00:22:31,660 --> 00:22:32,660 eye tracker. 608 00:22:33,690 --> 00:22:36,059 Device, what do you get from the nurse is 609 00:22:36,060 --> 00:22:38,969 brain activation, so the 610 00:22:38,970 --> 00:22:41,129 red means high activation, blue 611 00:22:41,130 --> 00:22:43,759 or green is lower activation? 612 00:22:43,760 --> 00:22:45,959 Um, strange interface, but 613 00:22:45,960 --> 00:22:48,209 yeah, and this 614 00:22:48,210 --> 00:22:50,729 is the end, I guess, synched 615 00:22:50,730 --> 00:22:52,949 with the brain 616 00:22:52,950 --> 00:22:54,839 activation. So in this case, the reading 617 00:22:54,840 --> 00:22:55,840 task. 618 00:22:57,270 --> 00:22:59,489 Uh, but what I'm really interested 619 00:22:59,490 --> 00:23:02,249 in, we didn't only record reading, 620 00:23:02,250 --> 00:23:04,619 but also calculation and 621 00:23:04,620 --> 00:23:05,829 so memory games. 622 00:23:05,830 --> 00:23:08,289 So in this case, you see a memory task 623 00:23:08,290 --> 00:23:10,739 and back, uh, that's 624 00:23:10,740 --> 00:23:12,959 a common task to assess memory. 625 00:23:12,960 --> 00:23:15,509 And one back is relatively easy 626 00:23:15,510 --> 00:23:17,939 to back. It's already quite difficult 627 00:23:17,940 --> 00:23:19,799 if you're not trained through your back 628 00:23:19,800 --> 00:23:22,109 is more difficult and fall-back 629 00:23:22,110 --> 00:23:23,729 nearly impossible if you have not 630 00:23:23,730 --> 00:23:25,589 trained. And this is a recording from one 631 00:23:25,590 --> 00:23:27,659 user, uh, with the 632 00:23:27,660 --> 00:23:29,459 workload, you know, as I said, one was 633 00:23:29,460 --> 00:23:30,959 easy for him. 634 00:23:30,960 --> 00:23:33,179 Looking at the effort as 635 00:23:33,180 --> 00:23:35,249 too was a little bit more difficult. 636 00:23:35,250 --> 00:23:37,529 Three was most difficult 637 00:23:37,530 --> 00:23:39,630 and for the person gave up. 638 00:23:41,160 --> 00:23:42,839 So and I'm 639 00:23:44,400 --> 00:23:46,619 most interested in this state, 640 00:23:46,620 --> 00:23:48,809 if it's possible to detect the state, 641 00:23:48,810 --> 00:23:50,999 not with the Lebanese device, but with 642 00:23:51,000 --> 00:23:53,249 something like blinks, 643 00:23:53,250 --> 00:23:55,739 I guess, features or some other, 644 00:23:55,740 --> 00:23:57,959 uh, cheap sensors, 645 00:23:57,960 --> 00:24:00,329 we can implement something 646 00:24:00,330 --> 00:24:02,489 that keeps us challenged while 647 00:24:02,490 --> 00:24:05,519 learning and not frustrated. 648 00:24:05,520 --> 00:24:07,709 And that's something I would really 649 00:24:07,710 --> 00:24:09,659 like to go. We just recorded the data. 650 00:24:09,660 --> 00:24:11,619 I'm not sure if there's something there. 651 00:24:11,620 --> 00:24:14,219 I saw some correlations, pupil diameter 652 00:24:14,220 --> 00:24:15,749 or something, but that's not something I 653 00:24:15,750 --> 00:24:16,799 can get over. 654 00:24:16,800 --> 00:24:18,959 The Yochi Blink frequency is also 655 00:24:18,960 --> 00:24:20,969 interesting, but it's not good enough to 656 00:24:21,990 --> 00:24:23,389 detect the state yet. 657 00:24:23,390 --> 00:24:25,259 Also predicted, because what I really 658 00:24:25,260 --> 00:24:27,509 want to do is I want to predict that 659 00:24:27,510 --> 00:24:29,819 if I increase the difficulty for a task, 660 00:24:29,820 --> 00:24:31,019 the person will give up. 661 00:24:32,100 --> 00:24:33,869 And that brings me more or less already 662 00:24:33,870 --> 00:24:36,089 to the summary. So I believe that 663 00:24:36,090 --> 00:24:38,219 if you look at the last century is the 664 00:24:38,220 --> 00:24:40,079 biggest scientific breakthroughs where 665 00:24:40,080 --> 00:24:42,719 about our physical limitations 666 00:24:42,720 --> 00:24:45,179 so we can now travel faster, 667 00:24:45,180 --> 00:24:46,799 we can build higher, we live more 668 00:24:46,800 --> 00:24:49,139 comfortable and more along the longer 669 00:24:49,140 --> 00:24:51,479 lives. And I believe the future 670 00:24:51,480 --> 00:24:53,459 will be about overcoming all the biggest 671 00:24:53,460 --> 00:24:55,499 scientific breakthroughs, will be about 672 00:24:55,500 --> 00:24:58,379 overcoming our cognitive limitations. 673 00:24:58,380 --> 00:25:00,509 And let's work together with 674 00:25:00,510 --> 00:25:02,789 open tools and open, uh, a 675 00:25:02,790 --> 00:25:05,429 code to to achieve this. 676 00:25:05,430 --> 00:25:07,499 And that brings me also to a quick 677 00:25:07,500 --> 00:25:09,719 thank you slide just one to name two 678 00:25:09,720 --> 00:25:11,789 people. So especially Oliver, I'm often 679 00:25:11,790 --> 00:25:14,039 the cost of Cuba who enables 680 00:25:14,040 --> 00:25:16,229 part of the talk because of some hardware 681 00:25:16,230 --> 00:25:18,599 I got from him and a special thanks 682 00:25:18,600 --> 00:25:20,729 to the people who actually did 683 00:25:20,730 --> 00:25:22,109 the work. 684 00:25:22,110 --> 00:25:23,110 So. 685 00:25:32,400 --> 00:25:33,400 So 686 00:25:34,860 --> 00:25:37,719 you got to mind 687 00:25:37,720 --> 00:25:39,899 and now if you have questions, 688 00:25:39,900 --> 00:25:41,969 remarks or violent dissent 689 00:25:41,970 --> 00:25:43,739 and also if there are people interested 690 00:25:43,740 --> 00:25:45,869 in open eyewear platform or 691 00:25:45,870 --> 00:25:48,449 so on, please come and talk to me later. 692 00:25:48,450 --> 00:25:50,699 Thank you for the nice talk we 693 00:25:50,700 --> 00:25:51,700 have. 694 00:25:57,360 --> 00:25:59,699 We have five minutes for questions 695 00:25:59,700 --> 00:26:01,529 you can line about the microphones. 696 00:26:01,530 --> 00:26:03,899 We will start with microphone four. 697 00:26:03,900 --> 00:26:05,879 Could you explain the blink again? 698 00:26:05,880 --> 00:26:07,079 How do you capture that? 699 00:26:08,790 --> 00:26:11,049 So for the blink, 700 00:26:11,050 --> 00:26:13,379 I'm sorry for the interruption. 701 00:26:13,380 --> 00:26:15,299 Please move out quietly. 702 00:26:15,300 --> 00:26:17,609 No talking because there is a 703 00:26:17,610 --> 00:26:18,039 poll. 704 00:26:18,040 --> 00:26:20,189 You said it's a deep also it moves down 705 00:26:20,190 --> 00:26:22,319 when you blink and blink is actually 706 00:26:22,320 --> 00:26:24,119 muscle activity you get also. 707 00:26:25,640 --> 00:26:27,529 So what you see here is. 708 00:26:30,510 --> 00:26:32,789 Blink is relatively 709 00:26:32,790 --> 00:26:33,790 nice. 710 00:26:36,600 --> 00:26:38,669 So kind of this this signal 711 00:26:38,670 --> 00:26:40,889 in the end and you get it over 712 00:26:40,890 --> 00:26:42,659 the muscle movement. So this is not the 713 00:26:42,660 --> 00:26:43,559 deposal. 714 00:26:43,560 --> 00:26:44,999 So the eye does not move. 715 00:26:45,000 --> 00:26:46,949 In this case, you move your muscles and 716 00:26:46,950 --> 00:26:49,499 that's also something you can register 717 00:26:49,500 --> 00:26:50,669 over the electrodes. 718 00:26:50,670 --> 00:26:52,649 That's actually how do you say noise 719 00:26:52,650 --> 00:26:54,680 signal if you want to recognize 720 00:26:56,310 --> 00:26:57,249 Yochi. 721 00:26:57,250 --> 00:27:00,089 So you have one kind of sensor capturing 722 00:27:00,090 --> 00:27:02,249 muscles and the whole movement 723 00:27:02,250 --> 00:27:03,179 of the eye. 724 00:27:03,180 --> 00:27:05,189 Yeah. And you also see, I mean, you also 725 00:27:05,190 --> 00:27:06,389 have to do some filtering. 726 00:27:06,390 --> 00:27:08,609 So if I move my head and so on, of course 727 00:27:08,610 --> 00:27:10,319 you will get this also on the Yoji 728 00:27:11,400 --> 00:27:13,679 because of the small banks. 729 00:27:13,680 --> 00:27:15,509 Yeah. One question from the signal, 730 00:27:15,510 --> 00:27:16,510 Angel. 731 00:27:17,970 --> 00:27:20,219 Uh, the question 732 00:27:20,220 --> 00:27:22,499 was what sampling rate 733 00:27:22,500 --> 00:27:25,199 does the Yoji operate 734 00:27:25,200 --> 00:27:26,549 with for the line lengths? 735 00:27:28,140 --> 00:27:30,269 So for sampling rates, this 736 00:27:30,270 --> 00:27:32,879 depends on. So the urchins 737 00:27:32,880 --> 00:27:34,829 mean currently samples at around one 738 00:27:34,830 --> 00:27:36,259 hundred hertz. 739 00:27:36,260 --> 00:27:38,339 Uh, it can go, I think, up to two 740 00:27:38,340 --> 00:27:39,419 hundred hertz. 741 00:27:39,420 --> 00:27:41,249 This has nothing to do with the 742 00:27:41,250 --> 00:27:43,349 electrodes but with a chip inside 743 00:27:43,350 --> 00:27:45,689 and for the line lengths, the data 744 00:27:45,690 --> 00:27:47,579 you saw beforehand, that's a medical 745 00:27:47,580 --> 00:27:50,279 Yochi. So this is five hundred hertz. 746 00:27:50,280 --> 00:27:52,529 So here you need some some higher 747 00:27:52,530 --> 00:27:53,729 sampling rates. 748 00:27:53,730 --> 00:27:56,429 So for this you need five hundred hertz. 749 00:27:56,430 --> 00:27:58,529 But I think maybe with two hundred or one 750 00:27:58,530 --> 00:28:00,749 hundred fifty, you would also get 751 00:28:00,750 --> 00:28:02,939 reasonable results for for line break 752 00:28:02,940 --> 00:28:03,940 detection. 753 00:28:04,920 --> 00:28:05,309 Thank you. 754 00:28:05,310 --> 00:28:06,310 Microphone number three, 755 00:28:07,380 --> 00:28:09,809 I was wondering if there's any ethical 756 00:28:09,810 --> 00:28:12,659 reflection included in your studies 757 00:28:12,660 --> 00:28:15,149 and in your work because. 758 00:28:15,150 --> 00:28:17,009 Well, you don't have to have too much 759 00:28:17,010 --> 00:28:19,019 fantasy just to think how this 760 00:28:20,220 --> 00:28:22,169 thing, which is funny here, can be used 761 00:28:22,170 --> 00:28:24,549 as a perfect tool of control. 762 00:28:24,550 --> 00:28:25,619 Yeah, I agree. 763 00:28:25,620 --> 00:28:27,359 And I think there's already 764 00:28:28,680 --> 00:28:30,059 there's already a lot of work. 765 00:28:30,060 --> 00:28:31,469 That's why I had this advertised in 766 00:28:31,470 --> 00:28:33,890 Slate. I think we are already behind. 767 00:28:35,000 --> 00:28:37,159 You know, some companies that work 768 00:28:37,160 --> 00:28:39,229 in this field and have more knowledge 769 00:28:39,230 --> 00:28:41,899 about the relation between 770 00:28:41,900 --> 00:28:44,179 brain and movement 771 00:28:44,180 --> 00:28:46,279 and so on, and also kind of how 772 00:28:46,280 --> 00:28:48,349 you get this whole their whole study 773 00:28:48,350 --> 00:28:50,449 is on how to place products or 774 00:28:50,450 --> 00:28:52,639 how to design a supermarket or 775 00:28:52,640 --> 00:28:54,439 something like that that are not out 776 00:28:54,440 --> 00:28:56,839 there. And for me, the important 777 00:28:56,840 --> 00:28:59,209 part is that the data stays 778 00:28:59,210 --> 00:29:01,369 with you. You know, I don't want Google 779 00:29:01,370 --> 00:29:03,529 or Apple or somebody like 780 00:29:03,530 --> 00:29:05,599 this developing 781 00:29:05,600 --> 00:29:07,729 glasses and then getting 782 00:29:07,730 --> 00:29:10,009 all of our data, because I'm already 783 00:29:10,010 --> 00:29:12,409 scared from the data, 784 00:29:12,410 --> 00:29:14,299 you know, just the location data and 785 00:29:14,300 --> 00:29:16,549 other data. I'm giving it away for free. 786 00:29:16,550 --> 00:29:18,049 So there's definitely an issue. 787 00:29:18,050 --> 00:29:20,209 I think, you know, one thing we can do is 788 00:29:20,210 --> 00:29:22,399 just try to 789 00:29:22,400 --> 00:29:24,829 open this research up and show people 790 00:29:24,830 --> 00:29:27,049 what is possible to 791 00:29:27,050 --> 00:29:29,300 hopefully prevent somebody 792 00:29:30,470 --> 00:29:31,369 misusing this. 793 00:29:31,370 --> 00:29:32,749 But, yeah, that's that's definitely an 794 00:29:32,750 --> 00:29:33,799 issue. Thanks. 795 00:29:39,670 --> 00:29:41,439 My phone number two. 796 00:29:41,440 --> 00:29:43,689 Yep, yep. 797 00:29:43,690 --> 00:29:45,759 My question is, are you doing any 798 00:29:45,760 --> 00:29:48,639 research to connect your technology 799 00:29:48,640 --> 00:29:50,979 with virtual reality? 800 00:29:50,980 --> 00:29:53,559 For example, in the field of architecture 801 00:29:53,560 --> 00:29:54,640 or in arts or. 802 00:29:56,580 --> 00:29:58,679 I'm personally not 803 00:29:58,680 --> 00:30:00,869 so much, but there are some people 804 00:30:00,870 --> 00:30:02,519 that came through, especially not me, 805 00:30:02,520 --> 00:30:04,769 Sensi, who who worked in virtual 806 00:30:04,770 --> 00:30:06,959 reality and also thinks about this 807 00:30:06,960 --> 00:30:09,209 that I had not. 808 00:30:09,210 --> 00:30:11,579 So I think for especially for virtual 809 00:30:11,580 --> 00:30:13,709 reality, I think optical 810 00:30:13,710 --> 00:30:15,929 eye tracking is nice or nicer because 811 00:30:15,930 --> 00:30:17,939 you can get the relation of where you 812 00:30:17,940 --> 00:30:20,459 are, where where you're, where you're 813 00:30:20,460 --> 00:30:22,439 seeing. And you don't have this problem. 814 00:30:22,440 --> 00:30:24,389 If you're already wearing an Oculus Rift 815 00:30:24,390 --> 00:30:26,699 or a similar device, you don't 816 00:30:26,700 --> 00:30:28,979 have this problem of, 817 00:30:28,980 --> 00:30:31,619 you know, having something lightweight 818 00:30:31,620 --> 00:30:34,569 or so on to carry Yuchi. 819 00:30:34,570 --> 00:30:35,849 I'm not so sure. 820 00:30:35,850 --> 00:30:38,009 I think, you know, this Blinkx 821 00:30:38,010 --> 00:30:39,269 thing could be something. 822 00:30:39,270 --> 00:30:41,489 You know, the change Blanton's doing 823 00:30:41,490 --> 00:30:43,899 blinks could be interesting, but 824 00:30:43,900 --> 00:30:44,900 yeah. 825 00:30:46,140 --> 00:30:47,969 One last question from NO2. 826 00:30:49,840 --> 00:30:52,279 And I wanted to ask you about 827 00:30:52,280 --> 00:30:53,779 whether you have you thought about 828 00:30:53,780 --> 00:30:56,779 capturing brain waves 829 00:30:56,780 --> 00:30:59,089 from the frontal cortex? 830 00:30:59,090 --> 00:31:00,889 Uh, because it seems like, you know, 831 00:31:00,890 --> 00:31:02,180 something that is around, 832 00:31:03,850 --> 00:31:06,229 uh, so I tried 833 00:31:06,230 --> 00:31:08,099 each year beforehand, so I played with 834 00:31:08,100 --> 00:31:11,029 the motive. I have also the open BCI. 835 00:31:11,030 --> 00:31:13,249 So the dot com thing, not the 836 00:31:13,250 --> 00:31:14,449 thing. 837 00:31:14,450 --> 00:31:16,489 And I was interested in the talks 838 00:31:16,490 --> 00:31:17,929 yesterday, but unfortunately, I couldn't 839 00:31:17,930 --> 00:31:19,279 get in and I haven't really met the 840 00:31:19,280 --> 00:31:21,499 people from from my experience. 841 00:31:21,500 --> 00:31:24,169 As I said, Ichi is relatively noisy 842 00:31:24,170 --> 00:31:25,759 and it depends highly on users. 843 00:31:25,760 --> 00:31:27,709 So for me, surprisingly, it works very 844 00:31:27,710 --> 00:31:29,749 well. For a lot of my students, it 845 00:31:29,750 --> 00:31:32,119 doesn't really work well because 846 00:31:32,120 --> 00:31:34,309 of the placement of the electrodes and so 847 00:31:34,310 --> 00:31:36,379 on. So I'm interested and look 848 00:31:36,380 --> 00:31:37,339 into it. 849 00:31:37,340 --> 00:31:39,619 But for me, being 850 00:31:39,620 --> 00:31:42,019 lazy, I guess seems to be easier to 851 00:31:42,020 --> 00:31:44,429 come by and and use. 852 00:31:44,430 --> 00:31:46,309 Also, I find it funny and it's also 853 00:31:46,310 --> 00:31:48,469 interesting to some part because we 854 00:31:48,470 --> 00:31:50,599 saw now that maybe two 855 00:31:50,600 --> 00:31:52,609 or three channels could be enough to 856 00:31:52,610 --> 00:31:53,719 detect something. 857 00:31:53,720 --> 00:31:55,579 So it definitely look into it. 858 00:31:55,580 --> 00:31:56,719 But, uh, yeah. 859 00:31:56,720 --> 00:31:58,999 So so far it seems to be 860 00:31:59,000 --> 00:32:00,309 for me, more difficult. 861 00:32:01,430 --> 00:32:03,170 Then using, I guess, things 862 00:32:04,190 --> 00:32:06,020 one last round of applause for Kydd.