0 00:00:00,000 --> 00:00:30,000 Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/344 Thanks! 1 00:00:09,460 --> 00:00:11,559 Thank you for letting me have 2 00:00:11,560 --> 00:00:12,560 this talk today. 3 00:00:13,480 --> 00:00:15,699 I give you 30 more seconds to read 4 00:00:15,700 --> 00:00:18,159 the comic because it's essential 5 00:00:18,160 --> 00:00:20,039 to what I'm going to talk about today. 6 00:00:21,070 --> 00:00:23,229 So too 7 00:00:23,230 --> 00:00:24,230 small. 8 00:00:24,720 --> 00:00:27,069 OK, so in short, 9 00:00:27,070 --> 00:00:29,109 download the comic sometime later on. 10 00:00:29,110 --> 00:00:30,110 It's funny. 11 00:00:31,330 --> 00:00:33,699 The problem with robotics, either 12 00:00:33,700 --> 00:00:35,769 in academia or also in hacker 13 00:00:35,770 --> 00:00:37,929 spaces or in your private lab 14 00:00:37,930 --> 00:00:40,029 or in your basement or whatever you 15 00:00:40,030 --> 00:00:42,099 want to do. Robotics is that 16 00:00:42,100 --> 00:00:44,229 too many people start to reinvent what 17 00:00:44,230 --> 00:00:45,579 is already there. 18 00:00:45,580 --> 00:00:47,409 Of course, there are many reasons for 19 00:00:47,410 --> 00:00:49,689 that. I wrote them down in the 20 00:00:49,690 --> 00:00:51,879 talk summary, the AARP's 21 00:00:51,880 --> 00:00:54,489 blog that it's deprecated, not using C++ 22 00:00:54,490 --> 00:00:56,469 14 features and so on. 23 00:00:56,470 --> 00:00:58,929 But still, I want to make a point 24 00:00:58,930 --> 00:01:01,299 that if you want to go into robotic 25 00:01:01,300 --> 00:01:03,519 projects in your hackerspace, have 26 00:01:03,520 --> 00:01:05,679 a look at Ross, which is well established 27 00:01:05,680 --> 00:01:07,779 middleware for robotics, because it 28 00:01:07,780 --> 00:01:09,549 will give you many things for free that 29 00:01:09,550 --> 00:01:11,379 you would otherwise have to develop 30 00:01:11,380 --> 00:01:13,989 yourself so that you can more easily 31 00:01:13,990 --> 00:01:15,909 get to the point where you actually want 32 00:01:15,910 --> 00:01:16,910 to work on. 33 00:01:17,980 --> 00:01:19,899 So short overview. 34 00:01:19,900 --> 00:01:22,059 I'm going to give a technical 35 00:01:22,060 --> 00:01:23,409 introduction into Ross. 36 00:01:23,410 --> 00:01:25,659 I'm going to show some examples of 37 00:01:25,660 --> 00:01:27,999 technologies, packages that are already 38 00:01:28,000 --> 00:01:29,859 there and can be used out of the box in 39 00:01:29,860 --> 00:01:32,109 most cases that do not only work 40 00:01:32,110 --> 00:01:34,179 and this is the where I 41 00:01:34,180 --> 00:01:36,279 differ from most introduction to Ross 42 00:01:36,280 --> 00:01:39,159 talks can be also used with a webcam 43 00:01:39,160 --> 00:01:40,429 and a low budget. 44 00:01:40,430 --> 00:01:42,399 So if you have, like, I don't know, one 45 00:01:42,400 --> 00:01:43,899 hundred, two hundred, three hundred euros 46 00:01:43,900 --> 00:01:46,239 to build a robot, you can still use Ross 47 00:01:46,240 --> 00:01:47,889 to do it. You don't have to start with an 48 00:01:47,890 --> 00:01:50,529 ATM or Arduino and 49 00:01:50,530 --> 00:01:52,809 programing to register again. 50 00:01:52,810 --> 00:01:55,089 And the last part will be about some 51 00:01:55,090 --> 00:01:57,189 tools that you might 52 00:01:57,190 --> 00:01:59,319 be missing from your custom implemented 53 00:01:59,320 --> 00:02:01,089 middleware that are really great to have. 54 00:02:01,090 --> 00:02:03,159 And you can have them if 55 00:02:03,160 --> 00:02:04,160 you use. 56 00:02:04,570 --> 00:02:07,119 So first some text 57 00:02:07,120 --> 00:02:09,008 methods, then some images and funny 58 00:02:09,009 --> 00:02:12,039 videos and at the end some hyperlinks. 59 00:02:12,040 --> 00:02:14,319 So if you do robotics, 60 00:02:14,320 --> 00:02:15,459 what are you actually doing? 61 00:02:15,460 --> 00:02:17,929 Robots mainly consist of three parts. 62 00:02:17,930 --> 00:02:19,989 So you have some mechanical parts, some 63 00:02:19,990 --> 00:02:21,999 electrical parts and software. 64 00:02:22,000 --> 00:02:23,979 Today, I'm only going to talk about 65 00:02:23,980 --> 00:02:27,069 software. There's also many interesting 66 00:02:27,070 --> 00:02:29,319 projects how you can build modular 67 00:02:29,320 --> 00:02:31,449 reusable hardware, not only software, but 68 00:02:31,450 --> 00:02:33,279 today it's about software. 69 00:02:33,280 --> 00:02:35,439 So what kind of software is in a robot? 70 00:02:35,440 --> 00:02:38,079 Of course, there's drivers, cameras, 71 00:02:38,080 --> 00:02:39,080 motors, 72 00:02:40,150 --> 00:02:41,559 whatever else there is. 73 00:02:41,560 --> 00:02:43,659 You want to connect to your robot LEDs 74 00:02:43,660 --> 00:02:44,660 and so on. 75 00:02:45,250 --> 00:02:46,479 Then there's some core robot 76 00:02:46,480 --> 00:02:48,669 functionality you want to be able 77 00:02:48,670 --> 00:02:51,159 to do path planning, motion planning. 78 00:02:51,160 --> 00:02:53,019 You want to do some kind of image 79 00:02:53,020 --> 00:02:55,209 processing, 3D reconstruction 80 00:02:55,210 --> 00:02:57,249 from stereo cameras and so on and so 81 00:02:57,250 --> 00:02:59,379 forth. This is what I call core 82 00:02:59,380 --> 00:03:00,999 functionality here. 83 00:03:01,000 --> 00:03:03,189 Then usually your software does 84 00:03:03,190 --> 00:03:05,319 not run the way you wanted 85 00:03:05,320 --> 00:03:06,459 in the first place. 86 00:03:06,460 --> 00:03:09,039 So you need some debugging facilities. 87 00:03:09,040 --> 00:03:10,299 Also introspection. 88 00:03:10,300 --> 00:03:12,399 Your robot is doing something you did not 89 00:03:12,400 --> 00:03:14,439 expect. Instead of just hitting the 90 00:03:14,440 --> 00:03:16,719 emergency stop button, please always 91 00:03:16,720 --> 00:03:18,429 add one to your robots. 92 00:03:18,430 --> 00:03:20,889 You can have a look what is currently 93 00:03:20,890 --> 00:03:22,239 happening with the software. 94 00:03:22,240 --> 00:03:24,429 If you did not build a large binary 95 00:03:24,430 --> 00:03:26,199 blob that you are flashing on to your 96 00:03:26,200 --> 00:03:28,389 robot, then you need 97 00:03:28,390 --> 00:03:29,799 all kinds of algorithms. 98 00:03:29,800 --> 00:03:32,469 And here is where most academics 99 00:03:32,470 --> 00:03:34,359 work on like myself. 100 00:03:34,360 --> 00:03:36,339 You want to do, I don't know, motion 101 00:03:36,340 --> 00:03:38,979 compensation in cognitive 102 00:03:38,980 --> 00:03:41,289 surgery with robots or whatever. 103 00:03:42,580 --> 00:03:44,649 In any case, you need some very basic 104 00:03:44,650 --> 00:03:46,599 algorithms. Otherwise the robot will 105 00:03:46,600 --> 00:03:48,669 only, like, move around rather 106 00:03:48,670 --> 00:03:51,249 randomly. So if I want to 107 00:03:51,250 --> 00:03:53,949 make my robot grasp this microphone, 108 00:03:53,950 --> 00:03:56,109 I need to do motion planning with 109 00:03:56,110 --> 00:03:58,239 my many degrees of freedom arm. 110 00:03:58,240 --> 00:04:00,549 I need to do grass planning. 111 00:04:00,550 --> 00:04:02,379 I need to do collision avoidance and all 112 00:04:02,380 --> 00:04:03,819 of these things that are really hard 113 00:04:03,820 --> 00:04:05,409 problems. People are working on them 114 00:04:05,410 --> 00:04:07,659 since like thirty or forty years. 115 00:04:07,660 --> 00:04:09,429 They are not completely solved yet, but 116 00:04:09,430 --> 00:04:11,769 they are very good open source 117 00:04:11,770 --> 00:04:14,049 implementations that solve it for 118 00:04:14,050 --> 00:04:15,369 the usual cases. 119 00:04:15,370 --> 00:04:17,469 And the usual cases fortunately are the 120 00:04:17,470 --> 00:04:19,599 ones one usually encounters. 121 00:04:19,600 --> 00:04:21,669 That's why they are the usual ones. 122 00:04:21,670 --> 00:04:24,069 And then once your software 123 00:04:24,070 --> 00:04:26,259 gets large enough, you will not 124 00:04:26,260 --> 00:04:28,449 be able to run it on a single machine 125 00:04:28,450 --> 00:04:30,729 anymore. Made a single machine be some 126 00:04:30,730 --> 00:04:33,249 embedded device, like a beagle bone or, 127 00:04:33,250 --> 00:04:35,469 I don't know, a Sytek box or what 128 00:04:35,470 --> 00:04:37,629 all other arm boards there are around 129 00:04:37,630 --> 00:04:40,149 these days, not even an A7. 130 00:04:40,150 --> 00:04:42,369 You usually want to be able, 131 00:04:42,370 --> 00:04:44,379 if the computation demands it, to 132 00:04:44,380 --> 00:04:47,349 distribute your robot software 133 00:04:47,350 --> 00:04:49,539 to multiple computers that are connected 134 00:04:49,540 --> 00:04:51,759 via Internet, for example, 135 00:04:51,760 --> 00:04:52,809 and all us. 136 00:04:52,810 --> 00:04:55,209 All of this is already 137 00:04:55,210 --> 00:04:57,039 built into the core of us. 138 00:04:57,040 --> 00:04:59,259 So everything I'm going to show today 139 00:04:59,260 --> 00:05:01,719 can be run on my single laptop. 140 00:05:01,720 --> 00:05:04,059 Most things can be even run on something 141 00:05:04,060 --> 00:05:06,549 like the new Raspberry Pi, 142 00:05:06,550 --> 00:05:08,359 but it can. 143 00:05:08,360 --> 00:05:10,639 Also be started on a cluster 144 00:05:10,640 --> 00:05:12,319 of computers, so if I have like five 145 00:05:12,320 --> 00:05:14,749 desktop machines available, I can 146 00:05:14,750 --> 00:05:17,209 put the camera processing on one machine, 147 00:05:17,210 --> 00:05:19,339 the path planning on the other, and 148 00:05:19,340 --> 00:05:21,919 it will work together the same way as 149 00:05:21,920 --> 00:05:23,389 it would be running locally. 150 00:05:23,390 --> 00:05:24,949 And that is completely networked, 151 00:05:24,950 --> 00:05:27,139 transparent and happens during runtime. 152 00:05:28,600 --> 00:05:30,759 Then the this is also 153 00:05:30,760 --> 00:05:32,319 if you have multiple machines, 154 00:05:33,520 --> 00:05:35,709 you know that it's not as easy as 155 00:05:35,710 --> 00:05:37,539 compile, run it because you have to 156 00:05:37,540 --> 00:05:39,699 distribute it to the 157 00:05:39,700 --> 00:05:40,599 various machines. 158 00:05:40,600 --> 00:05:42,339 You have to start the components on 159 00:05:42,340 --> 00:05:43,389 various machines. 160 00:05:43,390 --> 00:05:45,489 So deployment and orchestration is 161 00:05:45,490 --> 00:05:47,799 also something you need to take care 162 00:05:47,800 --> 00:05:50,019 of once you have more complex robotic 163 00:05:50,020 --> 00:05:51,069 software. 164 00:05:51,070 --> 00:05:53,229 And of course, it's taken more or less 165 00:05:53,230 --> 00:05:54,669 care of already. 166 00:05:54,670 --> 00:05:56,019 So what do you want to work on? 167 00:05:56,020 --> 00:05:57,669 Usually just you want to build a cool 168 00:05:57,670 --> 00:06:00,309 application, I don't know, have a robot 169 00:06:00,310 --> 00:06:02,499 that gets the pieces out of your freshly 170 00:06:02,500 --> 00:06:04,779 built 3D printer so that it can continue 171 00:06:04,780 --> 00:06:06,849 printing instead of you having to go 172 00:06:06,850 --> 00:06:08,499 there, take the part and click 173 00:06:09,970 --> 00:06:12,519 and click onto print next one. 174 00:06:12,520 --> 00:06:14,709 So but where you usually end up working 175 00:06:14,710 --> 00:06:17,469 on is the very low level software. 176 00:06:17,470 --> 00:06:19,569 Get the motor to turn, 177 00:06:19,570 --> 00:06:21,669 get the motor to position itself 178 00:06:21,670 --> 00:06:23,949 and to a certain degree, get 179 00:06:23,950 --> 00:06:26,409 your camera running, get your camera 180 00:06:26,410 --> 00:06:28,569 data into open TV, back out 181 00:06:28,570 --> 00:06:31,149 of open TV, create point clouds, 182 00:06:31,150 --> 00:06:32,799 use the PCL and whatever. 183 00:06:32,800 --> 00:06:36,159 So you are doing a whole lot of pumping 184 00:06:36,160 --> 00:06:38,259 work when when you are working 185 00:06:38,260 --> 00:06:39,339 with robots. 186 00:06:39,340 --> 00:06:41,979 And this does not have to be the case. 187 00:06:41,980 --> 00:06:43,410 That is what I want to show now. 188 00:06:44,680 --> 00:06:45,999 So what is Ross? 189 00:06:46,000 --> 00:06:48,399 Ross stands for Robot Operating System. 190 00:06:48,400 --> 00:06:50,739 It consists of four major 191 00:06:50,740 --> 00:06:52,449 parts. One is the middleware, together 192 00:06:52,450 --> 00:06:53,529 with tools. 193 00:06:53,530 --> 00:06:55,329 Never underestimate the importance of 194 00:06:55,330 --> 00:06:57,099 having tools available for whatever you 195 00:06:57,100 --> 00:06:58,239 are using. 196 00:06:58,240 --> 00:07:00,099 Then the basic robotics software 197 00:07:00,100 --> 00:07:02,649 distributed processing 198 00:07:02,650 --> 00:07:04,989 of Cordona transformations. 199 00:07:04,990 --> 00:07:07,059 So, for example, if you have your 200 00:07:07,060 --> 00:07:09,129 robot and you want to tell it, move 201 00:07:09,130 --> 00:07:11,349 five centimeters to the left, you 202 00:07:11,350 --> 00:07:13,449 usually specify that within some kind of 203 00:07:13,450 --> 00:07:14,709 coordinate frame. 204 00:07:14,710 --> 00:07:16,539 So either it is the coordinate frame of 205 00:07:16,540 --> 00:07:18,489 the robot, it is the coordinate frame of 206 00:07:18,490 --> 00:07:20,169 my laptop and so on. 207 00:07:20,170 --> 00:07:22,299 And you in the background, you 208 00:07:22,300 --> 00:07:24,639 do just some matrix multiplications 209 00:07:24,640 --> 00:07:26,169 to get everything right. 210 00:07:26,170 --> 00:07:28,989 Perhaps you remember your algebra classes 211 00:07:28,990 --> 00:07:31,449 or not. It does not matter because 212 00:07:31,450 --> 00:07:32,979 here it's just an optical. 213 00:07:32,980 --> 00:07:35,409 Give me this position in this coordinate 214 00:07:35,410 --> 00:07:37,749 frame and it will do all the magic, 215 00:07:37,750 --> 00:07:39,970 algebra, algebra in the background. 216 00:07:41,050 --> 00:07:43,479 Um, then the cert. 217 00:07:43,480 --> 00:07:45,609 And I think most important part about 218 00:07:45,610 --> 00:07:47,499 Ross Ross is not better than any other 219 00:07:47,500 --> 00:07:49,629 middleware that was built and 220 00:07:49,630 --> 00:07:51,039 there were hundreds of it in the last 221 00:07:51,040 --> 00:07:53,139 thirty years, but it gained 222 00:07:53,140 --> 00:07:55,449 such a big momentum that a large 223 00:07:55,450 --> 00:07:57,699 ecosystem formed around it, kind 224 00:07:57,700 --> 00:07:59,889 of similar to Linux, perhaps 225 00:07:59,890 --> 00:08:02,169 compared to the BSD. 226 00:08:02,170 --> 00:08:04,809 Now, don't don't flatter me 227 00:08:04,810 --> 00:08:07,119 for that. But the important point 228 00:08:07,120 --> 00:08:09,279 is you can take Ross today. 229 00:08:10,330 --> 00:08:12,489 Install it, which is easy to 230 00:08:12,490 --> 00:08:14,769 get on a Bantus and then 231 00:08:14,770 --> 00:08:17,139 have a large collection of software 232 00:08:17,140 --> 00:08:19,449 available that I will be showing 233 00:08:19,450 --> 00:08:21,369 shortly, but it's already there. 234 00:08:21,370 --> 00:08:23,709 And these are the three main 235 00:08:23,710 --> 00:08:25,989 components that make Ross a great tool 236 00:08:25,990 --> 00:08:27,310 to use for robotics. 237 00:08:28,680 --> 00:08:31,139 So now we're going to 238 00:08:31,140 --> 00:08:33,239 describe some of the technical 239 00:08:33,240 --> 00:08:35,189 terminology used in. 240 00:08:35,190 --> 00:08:37,379 So first there's the ROSTKER, which 241 00:08:37,380 --> 00:08:39,719 is the only centralized 242 00:08:39,720 --> 00:08:41,548 part of Ross. So it's a well known 243 00:08:41,549 --> 00:08:43,709 instance, like your name server, 244 00:08:43,710 --> 00:08:46,169 for example, and it can point 245 00:08:46,170 --> 00:08:48,599 you to the other instances within your 246 00:08:48,600 --> 00:08:50,370 robot network. 247 00:08:53,380 --> 00:08:55,509 Notes are any 248 00:08:55,510 --> 00:08:57,639 process within your 249 00:08:57,640 --> 00:09:00,279 robotic network that implements 250 00:09:00,280 --> 00:09:02,439 the client up, which is 251 00:09:02,440 --> 00:09:05,199 usually used through raw CPR 252 00:09:05,200 --> 00:09:07,839 OSPI, which are the main programing 253 00:09:07,840 --> 00:09:09,849 languages used to interface Bifrost 254 00:09:09,850 --> 00:09:12,669 Systems. There are also many 255 00:09:12,670 --> 00:09:14,859 other interfaces, even for Motlop these 256 00:09:14,860 --> 00:09:17,169 days, even officially from MAFF 257 00:09:17,170 --> 00:09:18,439 works for Motlop these days. 258 00:09:18,440 --> 00:09:20,139 So it really got important, obviously, 259 00:09:21,310 --> 00:09:23,859 and did support all the nice features 260 00:09:23,860 --> 00:09:25,929 I'm going to show till the 261 00:09:25,930 --> 00:09:27,129 end of my talk. 262 00:09:28,670 --> 00:09:30,859 So initially you start out with 263 00:09:30,860 --> 00:09:33,209 an empty computational crap, 264 00:09:33,210 --> 00:09:35,479 so basically we are talking 265 00:09:35,480 --> 00:09:37,639 about notes and entries 266 00:09:37,640 --> 00:09:39,319 that connect these notes. 267 00:09:39,320 --> 00:09:42,259 So the note is some kind of process 268 00:09:42,260 --> 00:09:44,539 on an operating system running on any 269 00:09:44,540 --> 00:09:46,219 machines that are connected to your 270 00:09:46,220 --> 00:09:47,209 network. 271 00:09:47,210 --> 00:09:49,249 And then there's the central core running 272 00:09:49,250 --> 00:09:51,049 on one of these machines. 273 00:09:51,050 --> 00:09:52,969 So initially, there are no connections. 274 00:09:52,970 --> 00:09:55,219 We have just started the Rostker. 275 00:09:55,220 --> 00:09:57,349 Then we had a node A and A 276 00:09:57,350 --> 00:09:59,959 wants to publish some information 277 00:09:59,960 --> 00:10:02,419 under the name subtopic, 278 00:10:02,420 --> 00:10:04,549 either later on or previously. 279 00:10:04,550 --> 00:10:06,499 Timing does not matter here. 280 00:10:06,500 --> 00:10:08,659 Nobody comes along and wants to 281 00:10:08,660 --> 00:10:11,149 subscribe to this same topic. 282 00:10:11,150 --> 00:10:13,219 And so it also tells the Rostker, 283 00:10:13,220 --> 00:10:15,499 please tell me who is 284 00:10:15,500 --> 00:10:17,749 publishing information under this 285 00:10:17,750 --> 00:10:19,769 name. So it's a classical publisher, 286 00:10:19,770 --> 00:10:22,009 subscriber architecture for those people 287 00:10:22,010 --> 00:10:23,010 who read Tennenbaum, 288 00:10:25,160 --> 00:10:27,259 the two currencies with the IP address 289 00:10:27,260 --> 00:10:29,149 and part of this core. 290 00:10:29,150 --> 00:10:31,459 So it's all based on TCP IP. 291 00:10:31,460 --> 00:10:34,249 There's also UDP implementations, 292 00:10:34,250 --> 00:10:36,979 but the standard is TCP IP. 293 00:10:36,980 --> 00:10:39,319 After nobody knows about how 294 00:10:39,320 --> 00:10:41,539 to reach Node eight directly, it 295 00:10:41,540 --> 00:10:42,439 will contact him. 296 00:10:42,440 --> 00:10:43,730 This is all our. 297 00:10:45,140 --> 00:10:47,419 And from there on, not 298 00:10:47,420 --> 00:10:50,149 able to transmit only the 299 00:10:50,150 --> 00:10:52,219 bulk data to be whenever 300 00:10:52,220 --> 00:10:54,499 there's something new that it wants 301 00:10:54,500 --> 00:10:56,719 to tell its subscribers 302 00:10:56,720 --> 00:10:58,909 about. So as you can see, the ROSTKER 303 00:10:58,910 --> 00:11:00,979 only gets involved into setting 304 00:11:00,980 --> 00:11:02,509 up the connections, but it's not the 305 00:11:02,510 --> 00:11:04,619 central bottleneck in the network. 306 00:11:04,620 --> 00:11:06,919 So not all of the information is pushed 307 00:11:06,920 --> 00:11:09,049 through the ROSTKER, which is quite 308 00:11:09,050 --> 00:11:11,209 fortunate because if you have a couple of 309 00:11:11,210 --> 00:11:13,219 Kinect cameras, for example, on your 310 00:11:13,220 --> 00:11:15,439 robot, each one saturates your 311 00:11:15,440 --> 00:11:16,369 gigabit link. 312 00:11:16,370 --> 00:11:19,309 So you really do not want to 313 00:11:19,310 --> 00:11:21,529 add artificial bottlenecks 314 00:11:21,530 --> 00:11:22,629 to. 315 00:11:22,630 --> 00:11:23,869 And the nice thing about this 316 00:11:23,870 --> 00:11:26,119 architecture is that on the same 317 00:11:26,120 --> 00:11:27,120 topic. 318 00:11:28,310 --> 00:11:30,559 Any number of publishers can add 319 00:11:30,560 --> 00:11:32,749 information to this topic and any 320 00:11:32,750 --> 00:11:34,879 number of subscribers will be 321 00:11:34,880 --> 00:11:36,979 informed once there's new information 322 00:11:36,980 --> 00:11:39,199 available on 323 00:11:39,200 --> 00:11:40,159 a technical level. 324 00:11:40,160 --> 00:11:42,289 This is done at the moment 325 00:11:42,290 --> 00:11:43,629 with unicast. 326 00:11:43,630 --> 00:11:46,099 So you have multiple copies 327 00:11:46,100 --> 00:11:47,029 on your network. 328 00:11:47,030 --> 00:11:49,219 But in May of next year, 329 00:11:49,220 --> 00:11:51,409 it will also be able or 330 00:11:51,410 --> 00:11:54,139 at least that's the plan, to use IPv6 331 00:11:54,140 --> 00:11:56,539 multicast. So then it's even efficient 332 00:11:56,540 --> 00:11:57,540 what you are doing here. 333 00:11:58,970 --> 00:12:01,039 So topics are already introduced. 334 00:12:01,040 --> 00:12:02,809 Those were the names under which 335 00:12:02,810 --> 00:12:04,139 information is published. 336 00:12:04,140 --> 00:12:07,009 So it's just some way to 337 00:12:07,010 --> 00:12:09,029 define where to find information. 338 00:12:09,030 --> 00:12:11,029 And it's built up in a hierarchical 339 00:12:11,030 --> 00:12:12,979 structure you'll see shortly. 340 00:12:12,980 --> 00:12:15,079 So you can kind of imagine it 341 00:12:15,080 --> 00:12:17,389 like your your else 342 00:12:17,390 --> 00:12:20,179 in your file system, your file, 343 00:12:20,180 --> 00:12:22,369 whatever funny cat video that ABC 344 00:12:22,370 --> 00:12:24,499 is located in some directory and you 345 00:12:24,500 --> 00:12:26,359 can refer to it under that name. 346 00:12:26,360 --> 00:12:28,759 So these are the names and roles, 347 00:12:28,760 --> 00:12:30,979 but they are not locally but 348 00:12:30,980 --> 00:12:33,199 used within this network through 349 00:12:33,200 --> 00:12:34,200 the Rostker. 350 00:12:35,280 --> 00:12:37,779 A topic is a unidirectional connection, 351 00:12:37,780 --> 00:12:40,049 so you publish information and 352 00:12:40,050 --> 00:12:41,939 people subscribe to it, so it's not 353 00:12:41,940 --> 00:12:42,839 bidirectional. 354 00:12:42,840 --> 00:12:44,729 So you do not get feedback on who 355 00:12:44,730 --> 00:12:46,719 subscribe to it and you don't care about 356 00:12:46,720 --> 00:12:47,879 you just publish it. 357 00:12:47,880 --> 00:12:49,409 And if there is at least someone who's 358 00:12:49,410 --> 00:12:51,689 interested in it, this note 359 00:12:51,690 --> 00:12:52,830 will get the information. 360 00:12:54,600 --> 00:12:56,129 There's another kind of 361 00:12:57,870 --> 00:13:00,309 ipsi interspecies communication mechanism 362 00:13:00,310 --> 00:13:02,459 that are called services that are your 363 00:13:02,460 --> 00:13:04,499 classically remote procedure calls to 364 00:13:04,500 --> 00:13:07,259 your. You call this name 365 00:13:07,260 --> 00:13:09,149 has some parameters to it. 366 00:13:09,150 --> 00:13:10,859 It will be serialized through the 367 00:13:10,860 --> 00:13:12,029 network. 368 00:13:12,030 --> 00:13:14,039 The destination node will do some kind of 369 00:13:14,040 --> 00:13:16,199 computation and 370 00:13:16,200 --> 00:13:18,329 sort of reply 371 00:13:18,330 --> 00:13:19,919 and afterwards this kind of 372 00:13:19,920 --> 00:13:20,909 communications end. 373 00:13:20,910 --> 00:13:22,979 So it's not intended 374 00:13:22,980 --> 00:13:25,049 for streaming data. 375 00:13:25,050 --> 00:13:27,479 It's more for like, please, 376 00:13:27,480 --> 00:13:29,579 please reinitialize yourself or 377 00:13:29,580 --> 00:13:31,769 please do this calculation for me or get 378 00:13:31,770 --> 00:13:34,459 me your current set of parameters. 379 00:13:34,460 --> 00:13:36,119 And the good thing about these 380 00:13:36,120 --> 00:13:38,189 communications mechanisms is that they're 381 00:13:38,190 --> 00:13:40,049 kind of complementary. 382 00:13:40,050 --> 00:13:42,419 So you have your topics for streaming 383 00:13:42,420 --> 00:13:43,909 data sensors. 384 00:13:43,910 --> 00:13:46,559 Think of a camera or your robot, 385 00:13:46,560 --> 00:13:48,959 which publishes its current position 386 00:13:48,960 --> 00:13:52,439 of all of its angles and joins. 387 00:13:52,440 --> 00:13:54,569 The problem of it is that messages may 388 00:13:54,570 --> 00:13:56,789 be dropped due to overload without 389 00:13:56,790 --> 00:13:58,949 notice. So but if 390 00:13:58,950 --> 00:14:00,569 you think about it, if you have a camera 391 00:14:00,570 --> 00:14:02,999 publishing information, it's 33 392 00:14:03,000 --> 00:14:05,249 milliseconds, like it's 30 frames 393 00:14:05,250 --> 00:14:07,619 per second. You really want the frame 394 00:14:07,620 --> 00:14:09,689 to be dropped instead of a Q building up. 395 00:14:09,690 --> 00:14:12,689 If your Wi-Fi is too slow and you 396 00:14:12,690 --> 00:14:14,909 go further and further back into the, 397 00:14:14,910 --> 00:14:17,219 um, into the past and 398 00:14:17,220 --> 00:14:18,449 get a large latency. 399 00:14:18,450 --> 00:14:20,729 So that is why it was built 400 00:14:20,730 --> 00:14:22,649 with lots in mind. 401 00:14:22,650 --> 00:14:24,899 Services, on the other hand, are 402 00:14:24,900 --> 00:14:25,949 blocking calls. 403 00:14:25,950 --> 00:14:28,139 So you call the service, get blocked 404 00:14:28,140 --> 00:14:30,149 within the current node will be 405 00:14:30,150 --> 00:14:32,789 transferred to the distant node 406 00:14:32,790 --> 00:14:34,799 until the reply is received. 407 00:14:34,800 --> 00:14:36,959 You are blocked, which is nice to do some 408 00:14:36,960 --> 00:14:39,599 kind of tasks. 409 00:14:39,600 --> 00:14:41,639 And then there are also actions which 410 00:14:41,640 --> 00:14:44,369 built and on the first two mechanisms 411 00:14:44,370 --> 00:14:46,919 and they are allow you to 412 00:14:46,920 --> 00:14:49,649 deal with long term 413 00:14:49,650 --> 00:14:51,299 tasks. For example, if you tell your 414 00:14:51,300 --> 00:14:53,669 robot move from here to here, 415 00:14:53,670 --> 00:14:55,469 you really do not want to have that as a 416 00:14:55,470 --> 00:14:57,659 function call like in your programing 417 00:14:57,660 --> 00:15:00,389 language, because it will take time 418 00:15:00,390 --> 00:15:02,549 until the robot goes from some point 419 00:15:02,550 --> 00:15:03,779 to another point. 420 00:15:03,780 --> 00:15:05,789 And you might want to get feedback in 421 00:15:05,790 --> 00:15:06,849 between as well. 422 00:15:06,850 --> 00:15:08,939 So you have actions, you tell your 423 00:15:08,940 --> 00:15:11,099 robot, go there and then you 424 00:15:11,100 --> 00:15:13,229 are unplugged when this request 425 00:15:13,230 --> 00:15:15,359 was received by the 426 00:15:15,360 --> 00:15:17,309 other node and then you can get 427 00:15:17,310 --> 00:15:19,409 intermediate feedback and at the end you 428 00:15:19,410 --> 00:15:21,449 get the result. For example, OK, I 429 00:15:21,450 --> 00:15:23,999 reached a position or I collided 430 00:15:24,000 --> 00:15:25,340 with your cat or whatever, 431 00:15:27,120 --> 00:15:29,339 and all these kinds of mechanisms 432 00:15:29,340 --> 00:15:31,569 are used in what I'm going to show for 433 00:15:31,570 --> 00:15:32,570 it on. 434 00:15:33,920 --> 00:15:36,049 The last terminology I 435 00:15:36,050 --> 00:15:38,149 have to introduce our parameters, so the 436 00:15:38,150 --> 00:15:41,029 Rostker also act as parameter so 437 00:15:41,030 --> 00:15:43,189 you can kind of at it's 438 00:15:43,190 --> 00:15:45,829 a bit similar to rest up piece, you can 439 00:15:45,830 --> 00:15:48,109 just add names there with some 440 00:15:48,110 --> 00:15:50,299 data attached to it and also with 441 00:15:50,300 --> 00:15:51,529 various kinds of data. 442 00:15:51,530 --> 00:15:53,779 So not only strings, but also arrays, 443 00:15:53,780 --> 00:15:55,759 whatever you can serialize in YAML. 444 00:15:56,760 --> 00:15:58,559 And this is usually used to do 445 00:15:58,560 --> 00:16:00,629 distributed parameterization 446 00:16:00,630 --> 00:16:02,009 of your notes. 447 00:16:02,010 --> 00:16:04,319 Imagine you have the classical solution 448 00:16:04,320 --> 00:16:06,239 of configuration files and you want to 449 00:16:06,240 --> 00:16:08,309 run your note 450 00:16:08,310 --> 00:16:10,559 today on a different machine than 451 00:16:10,560 --> 00:16:12,449 you would have to make sure that your 452 00:16:12,450 --> 00:16:14,609 configuration file on this machine is 453 00:16:14,610 --> 00:16:17,279 up to date with the parameter server. 454 00:16:17,280 --> 00:16:19,559 Whoever in the network starts this node 455 00:16:19,560 --> 00:16:21,569 pushes the current set of parameters to 456 00:16:21,570 --> 00:16:23,459 the server, and all nodes, no matter on 457 00:16:23,460 --> 00:16:25,679 which machine they're running, uses these 458 00:16:25,680 --> 00:16:28,989 parameters to set up their 459 00:16:28,990 --> 00:16:30,600 their algorithms, for example. 460 00:16:32,860 --> 00:16:34,959 OK, I've talked about 461 00:16:34,960 --> 00:16:37,629 names for everyone who 462 00:16:37,630 --> 00:16:39,969 ever used your L's or 463 00:16:39,970 --> 00:16:42,129 perhaps Unix OS, it will be quite 464 00:16:42,130 --> 00:16:44,649 familiar that you have absolute paths 465 00:16:44,650 --> 00:16:47,049 that begin with a slash and 466 00:16:47,050 --> 00:16:50,019 you have relative paths that are not 467 00:16:50,020 --> 00:16:51,939 anchored kind of at the root of your 468 00:16:51,940 --> 00:16:54,579 system. And this is quite nice 469 00:16:54,580 --> 00:16:57,429 because you can at runtime 470 00:16:57,430 --> 00:17:00,039 tell the note, OK, don't publish 471 00:17:00,040 --> 00:17:02,619 your topics at the 472 00:17:02,620 --> 00:17:04,689 kind of the root of your namespace, but 473 00:17:04,690 --> 00:17:06,909 push it one space 474 00:17:06,910 --> 00:17:08,979 down like kind of moving something 475 00:17:08,980 --> 00:17:10,209 to a subfolder. 476 00:17:10,210 --> 00:17:12,489 If you have relative addresses, 477 00:17:12,490 --> 00:17:14,348 everything will continue to work within 478 00:17:14,349 --> 00:17:16,779 that sub fold, even if you push it down. 479 00:17:16,780 --> 00:17:19,439 So, for example, if you have two webcams, 480 00:17:19,440 --> 00:17:21,909 you simply start your driver 481 00:17:21,910 --> 00:17:24,368 node twice in separate 482 00:17:24,369 --> 00:17:26,348 namespace is, for example, camera left 483 00:17:26,349 --> 00:17:27,459 and camera right. 484 00:17:27,460 --> 00:17:29,679 And then software does not need 485 00:17:29,680 --> 00:17:31,359 to know about that. 486 00:17:31,360 --> 00:17:33,339 It currently is being used together with 487 00:17:33,340 --> 00:17:34,359 another camera. 488 00:17:34,360 --> 00:17:36,219 That is just the runtime decision when 489 00:17:36,220 --> 00:17:37,240 you start your nodes. 490 00:17:38,930 --> 00:17:41,269 So a small example 491 00:17:41,270 --> 00:17:43,369 here, we have three notes within our 492 00:17:43,370 --> 00:17:45,589 network and our Ruska, for example, after 493 00:17:45,590 --> 00:17:47,959 you see Cammarano publishing images 494 00:17:47,960 --> 00:17:50,179 from some webcam, we have 495 00:17:50,180 --> 00:17:52,009 some processing note and does, for 496 00:17:52,010 --> 00:17:54,619 example, an air filter 497 00:17:54,620 --> 00:17:56,029 filtering with the images. 498 00:17:56,030 --> 00:17:58,309 And then we have our Arcudi GUI, 499 00:17:58,310 --> 00:18:01,069 which displays these images 500 00:18:01,070 --> 00:18:03,079 for the user to look at what the robot is 501 00:18:03,080 --> 00:18:04,159 currently perceiving. 502 00:18:04,160 --> 00:18:06,559 So we have here our parameters. 503 00:18:06,560 --> 00:18:07,699 We have our nodes. 504 00:18:07,700 --> 00:18:10,249 We have our services that can be used 505 00:18:10,250 --> 00:18:12,649 as our PCs and we have our topics 506 00:18:12,650 --> 00:18:14,660 where streaming data is moving. 507 00:18:16,290 --> 00:18:18,779 Now, we have all kinds of tools for 508 00:18:18,780 --> 00:18:20,259 all of these things I talked about, 509 00:18:20,260 --> 00:18:22,559 there's the command line tool that allows 510 00:18:22,560 --> 00:18:24,659 you to have a look at what is 511 00:18:24,660 --> 00:18:26,699 currently running into your network. 512 00:18:26,700 --> 00:18:28,649 For example, if you want to find out all 513 00:18:28,650 --> 00:18:30,329 the notes that you have started on your 514 00:18:30,330 --> 00:18:32,429 various machines, you just type in 515 00:18:32,430 --> 00:18:34,319 your command line node list and you will 516 00:18:34,320 --> 00:18:36,389 get the list and you can get additional 517 00:18:36,390 --> 00:18:37,679 information on each node. 518 00:18:37,680 --> 00:18:39,869 The same for topic services, 519 00:18:39,870 --> 00:18:42,119 messages for the different data types 520 00:18:42,120 --> 00:18:43,120 that are used 521 00:18:44,250 --> 00:18:45,539 for the parameters of. 522 00:18:45,540 --> 00:18:47,930 And also there are CREF, there are 523 00:18:49,080 --> 00:18:51,539 three the or 524 00:18:51,540 --> 00:18:53,669 GUI and 3D tools you 525 00:18:53,670 --> 00:18:55,769 can use to fuel the current state of 526 00:18:55,770 --> 00:18:57,869 your robot. You'll see them short. 527 00:18:57,870 --> 00:18:59,999 But so just here 528 00:19:00,000 --> 00:19:02,969 to glance over these are two 529 00:19:02,970 --> 00:19:05,219 code snippets, one for Python and one for 530 00:19:05,220 --> 00:19:07,529 C++ to publish something 531 00:19:07,530 --> 00:19:08,939 on the network. 532 00:19:08,940 --> 00:19:09,899 So it's quite simple. 533 00:19:09,900 --> 00:19:12,219 You initialize your node, you create 534 00:19:12,220 --> 00:19:14,369 the publisher, you fill your 535 00:19:14,370 --> 00:19:16,589 data type with information, and then 536 00:19:16,590 --> 00:19:18,869 you call publish and you don't 537 00:19:18,870 --> 00:19:20,939 have to do anything else. 538 00:19:20,940 --> 00:19:22,349 Then it will be already 539 00:19:23,610 --> 00:19:26,099 available. If you type topic list, 540 00:19:26,100 --> 00:19:28,619 you can already see the topic 541 00:19:28,620 --> 00:19:31,259 as a new topic available. 542 00:19:31,260 --> 00:19:33,449 And if you go to the next slide, 543 00:19:33,450 --> 00:19:35,609 you can write your subscriber with a few 544 00:19:35,610 --> 00:19:36,059 lines. 545 00:19:36,060 --> 00:19:39,209 You create the subscriber, tell it 546 00:19:39,210 --> 00:19:40,829 what is the name of the topic it should 547 00:19:40,830 --> 00:19:42,989 subscribe to what data is being sent 548 00:19:42,990 --> 00:19:45,539 on it, um, provided for callbacks. 549 00:19:45,540 --> 00:19:47,249 So this is event driven. 550 00:19:47,250 --> 00:19:49,919 And afterwards this callback will be 551 00:19:49,920 --> 00:19:52,109 called whenever some node on 552 00:19:52,110 --> 00:19:54,329 the network publishes something 553 00:19:54,330 --> 00:19:56,729 under the name a topic. 554 00:19:56,730 --> 00:19:57,749 And that's about it. 555 00:19:57,750 --> 00:20:00,119 You have to know to use most 556 00:20:00,120 --> 00:20:02,529 parts of the middleware 557 00:20:02,530 --> 00:20:03,530 of Ross. 558 00:20:04,890 --> 00:20:07,109 OK, yes, Ross has 559 00:20:07,110 --> 00:20:09,419 its own built system, as perhaps all 560 00:20:09,420 --> 00:20:10,529 large software 561 00:20:11,970 --> 00:20:14,579 frameworks have come up with. 562 00:20:14,580 --> 00:20:16,349 The good thing is it's only a very thin 563 00:20:16,350 --> 00:20:17,599 wrap around. 564 00:20:17,600 --> 00:20:19,739 If you have ever worked with Semak, you 565 00:20:19,740 --> 00:20:22,049 will be fine. Otherwise, learn 566 00:20:22,050 --> 00:20:23,050 c make. 567 00:20:24,150 --> 00:20:27,149 But don't worry, this is really no 568 00:20:27,150 --> 00:20:28,409 game changer. 569 00:20:28,410 --> 00:20:30,629 It's just you have 570 00:20:30,630 --> 00:20:32,759 to look at what is written on this slide, 571 00:20:32,760 --> 00:20:34,109 this slide and this slide. 572 00:20:34,110 --> 00:20:36,389 And then you are in C McWorld again and 573 00:20:36,390 --> 00:20:38,339 it will take care of resolving the 574 00:20:38,340 --> 00:20:40,469 dependencies between the 575 00:20:40,470 --> 00:20:42,809 packages you are using 576 00:20:42,810 --> 00:20:44,279 that they are get built in the right 577 00:20:44,280 --> 00:20:46,259 order. So it actually helps to work with 578 00:20:46,260 --> 00:20:48,599 it. Also, it's of course annoying 579 00:20:48,600 --> 00:20:50,759 as but I've never seen a built system 580 00:20:50,760 --> 00:20:53,409 that is not annoying, so no change here. 581 00:20:53,410 --> 00:20:55,679 OK, so now I've talked about what 582 00:20:55,680 --> 00:20:57,059 is in technical terms. 583 00:20:57,060 --> 00:20:59,219 So what do you need to use 584 00:20:59,220 --> 00:21:01,379 or do you need always nice seven PC 585 00:21:01,380 --> 00:21:02,609 purpose. You don't have one on your 586 00:21:02,610 --> 00:21:04,949 quadcopter or on the International Space 587 00:21:04,950 --> 00:21:06,779 Station, both places where Ross is 588 00:21:06,780 --> 00:21:08,009 currently running? 589 00:21:08,010 --> 00:21:10,199 No, you simply need anything that can 590 00:21:10,200 --> 00:21:12,629 run Debian, 591 00:21:12,630 --> 00:21:14,949 Linux or preferably Ubuntu. 592 00:21:14,950 --> 00:21:17,279 I'm saying Bunta 14 for 593 00:21:17,280 --> 00:21:19,379 here because that's the easiest 594 00:21:19,380 --> 00:21:21,539 way there. Just type SUDEP, Cardross, 595 00:21:21,540 --> 00:21:23,819 whatever, and it's installed. 596 00:21:23,820 --> 00:21:25,979 You can of course, build it from source 597 00:21:25,980 --> 00:21:28,169 on all other platforms, but I don't 598 00:21:28,170 --> 00:21:30,449 think you want to. It has a couple 599 00:21:30,450 --> 00:21:31,590 thousand dependencies. 600 00:21:34,630 --> 00:21:36,849 So what kind of machine do you need? 601 00:21:36,850 --> 00:21:39,069 Preferably some powerful desktop 602 00:21:39,070 --> 00:21:42,249 or laptop machine so you can get started. 603 00:21:42,250 --> 00:21:44,649 A good solution if you are power 604 00:21:44,650 --> 00:21:46,869 constrained is to have a small machine on 605 00:21:46,870 --> 00:21:49,269 board of the robot, like a beagle bone 606 00:21:49,270 --> 00:21:51,899 or a raspberry or knock 607 00:21:51,900 --> 00:21:54,549 knock and a faster machine 608 00:21:54,550 --> 00:21:56,139 of the robot that does the heavy 609 00:21:56,140 --> 00:21:58,029 processing. But you can always, as I 610 00:21:58,030 --> 00:22:00,369 said, use it locally and you don't get a 611 00:22:00,370 --> 00:22:02,079 large performance penalty. 612 00:22:03,220 --> 00:22:05,619 But it depends on 613 00:22:05,620 --> 00:22:06,849 what application you're running. 614 00:22:06,850 --> 00:22:09,220 Of course, if you have like four connects 615 00:22:10,240 --> 00:22:12,009 that stream data and you want to do 616 00:22:12,010 --> 00:22:15,249 Pontarlier processing on them, you can 617 00:22:15,250 --> 00:22:17,559 put some desktop machine 618 00:22:17,560 --> 00:22:19,989 into your rack for each Kinect because 619 00:22:19,990 --> 00:22:22,839 that is just computationally intensive. 620 00:22:22,840 --> 00:22:24,669 If you just want to have a mobile robot 621 00:22:24,670 --> 00:22:26,619 like a Roomba moving around, it's 622 00:22:26,620 --> 00:22:28,779 probably enough to strap a beagle bone to 623 00:22:28,780 --> 00:22:29,780 it. 624 00:22:30,220 --> 00:22:32,379 OK, now let's come to the part 625 00:22:32,380 --> 00:22:34,599 with more images and perhaps 626 00:22:34,600 --> 00:22:36,669 more interesting if you are not yet sure 627 00:22:36,670 --> 00:22:38,109 you want to use us. 628 00:22:38,110 --> 00:22:40,209 So far, all the usual 629 00:22:40,210 --> 00:22:42,639 kind of camera sensors 630 00:22:42,640 --> 00:22:45,399 you find on a robot like Molnau cameras, 631 00:22:45,400 --> 00:22:47,619 for example, a single webcam, stereo 632 00:22:47,620 --> 00:22:49,689 cameras, for example, to webcams 633 00:22:49,690 --> 00:22:52,089 and Archibong cameras, for example, 634 00:22:52,090 --> 00:22:54,329 a Kinect or an Atlas 635 00:22:54,330 --> 00:22:56,559 action or whatever you want 636 00:22:56,560 --> 00:22:58,059 to spell it. 637 00:22:58,060 --> 00:23:00,159 Um, you need drivers for all of 638 00:23:00,160 --> 00:23:02,139 this. You need to calibrate them 639 00:23:02,140 --> 00:23:03,759 intrinsically and extrinsically. 640 00:23:03,760 --> 00:23:05,529 You want to visualize what's currently 641 00:23:05,530 --> 00:23:08,439 happening and you need processing 642 00:23:08,440 --> 00:23:10,899 so that your raw data 643 00:23:10,900 --> 00:23:13,479 do something useful for your application. 644 00:23:13,480 --> 00:23:15,579 And now I want to go through all of 645 00:23:15,580 --> 00:23:17,709 these steps for all three camera types 646 00:23:17,710 --> 00:23:18,710 very quickly. 647 00:23:19,630 --> 00:23:21,519 I'm not going to talk about object 648 00:23:21,520 --> 00:23:22,419 recognition here. 649 00:23:22,420 --> 00:23:24,639 There are some rather specific frameworks 650 00:23:24,640 --> 00:23:26,919 to facilitate object recognition, but 651 00:23:26,920 --> 00:23:29,199 basically they come down to 652 00:23:29,200 --> 00:23:31,419 using open CSV and upon 653 00:23:31,420 --> 00:23:32,420 cloud library. 654 00:23:33,940 --> 00:23:36,069 But still, it might be easier if you want 655 00:23:36,070 --> 00:23:38,259 to do object recognition to look 656 00:23:38,260 --> 00:23:40,479 what is already there in 657 00:23:40,480 --> 00:23:42,939 what packages have been built by others 658 00:23:42,940 --> 00:23:45,009 instead of rolling your own from the 659 00:23:45,010 --> 00:23:46,010 onset. 660 00:23:47,200 --> 00:23:49,269 So that's the final launch file to 661 00:23:49,270 --> 00:23:50,380 start my webcam 662 00:23:51,490 --> 00:23:53,769 launch files are just a convenient way 663 00:23:53,770 --> 00:23:55,569 for deployment and orchestration. 664 00:23:55,570 --> 00:23:57,609 It's just an extreme Elfa, starting with 665 00:23:57,610 --> 00:24:00,339 the highest level tech launch. 666 00:24:00,340 --> 00:24:02,439 Then you can add any number of note 667 00:24:02,440 --> 00:24:04,180 techs where you simply tell. 668 00:24:05,470 --> 00:24:07,749 Ross, from which package, what 669 00:24:07,750 --> 00:24:10,329 to load, what process 670 00:24:10,330 --> 00:24:12,549 or what binary, you can give it some 671 00:24:12,550 --> 00:24:15,259 name and define some parameters. 672 00:24:15,260 --> 00:24:17,709 For example, here I said namespace 673 00:24:17,710 --> 00:24:19,959 cams so everything will be pushed 674 00:24:19,960 --> 00:24:22,179 down into the subfolder camera 675 00:24:22,180 --> 00:24:24,489 so that I don't have a whole mess 676 00:24:24,490 --> 00:24:27,009 of topics at the top level namespace. 677 00:24:27,010 --> 00:24:30,099 And then you add some parameters for 678 00:24:30,100 --> 00:24:32,109 specific, of course, to your node 679 00:24:32,110 --> 00:24:35,109 resolution of the camera frame rate, 680 00:24:35,110 --> 00:24:37,059 disable autofocus and so on. 681 00:24:37,060 --> 00:24:38,060 And that's about it. 682 00:24:39,230 --> 00:24:41,389 Um, then you tie launch, 683 00:24:41,390 --> 00:24:43,819 I prepared a screen capture videos 684 00:24:43,820 --> 00:24:46,009 for this because life demos, as 685 00:24:46,010 --> 00:24:47,689 we saw in the previous hour, is too 686 00:24:47,690 --> 00:24:48,789 dangerous. 687 00:24:48,790 --> 00:24:50,959 Um, so let's have a look at this. 688 00:24:50,960 --> 00:24:51,960 I hope it works. 689 00:24:53,600 --> 00:24:55,699 So I started my Rostker, this is all now 690 00:24:55,700 --> 00:24:57,559 closed, I'll start the launch for you 691 00:24:57,560 --> 00:24:59,659 just saw then I run the 692 00:24:59,660 --> 00:25:01,849 Arctic we know I trust. 693 00:25:01,850 --> 00:25:03,949 OK, first I want to have a look on 694 00:25:03,950 --> 00:25:05,749 the command line. I am perhaps I don't 695 00:25:05,750 --> 00:25:07,819 like this. So let's have a look at 696 00:25:07,820 --> 00:25:09,289 the command line, what notes I have 697 00:25:09,290 --> 00:25:11,510 started, what topics they created. 698 00:25:12,530 --> 00:25:14,719 That's all good and all 699 00:25:14,720 --> 00:25:16,909 nice because you can also get more 700 00:25:16,910 --> 00:25:18,679 information about the topic number of 701 00:25:18,680 --> 00:25:21,049 publishers, what type the topics 702 00:25:21,050 --> 00:25:23,149 and so on. But I just started 703 00:25:23,150 --> 00:25:25,249 a cammarano, so I might want to see the 704 00:25:25,250 --> 00:25:26,629 camera images. 705 00:25:26,630 --> 00:25:28,249 That is where I start the Arctic. 706 00:25:28,250 --> 00:25:30,439 We here I select 707 00:25:30,440 --> 00:25:32,929 as a plugin visualization 708 00:25:32,930 --> 00:25:33,930 image. 709 00:25:35,970 --> 00:25:38,279 And then you can have a drop down 710 00:25:38,280 --> 00:25:40,589 menu, which gives you all the choices 711 00:25:40,590 --> 00:25:43,139 of topics that Kerry sends some messages 712 00:25:43,140 --> 00:25:44,309 image as their type. 713 00:25:44,310 --> 00:25:46,499 So these are normal, two dimensional 714 00:25:46,500 --> 00:25:47,849 images in rows. 715 00:25:47,850 --> 00:25:50,069 And I can just select that and 716 00:25:50,070 --> 00:25:52,469 have a live feed of my camera data. 717 00:25:52,470 --> 00:25:53,700 And this is. 718 00:25:55,370 --> 00:25:57,259 During runtime, so I did not have to 719 00:25:57,260 --> 00:25:59,479 recompile my camera, drive a note 720 00:25:59,480 --> 00:26:02,629 adding Dipak output, for example, 721 00:26:02,630 --> 00:26:03,879 open KVI image. 722 00:26:03,880 --> 00:26:06,169 So I just can at runtime 723 00:26:06,170 --> 00:26:08,359 start my GUI and it will 724 00:26:10,880 --> 00:26:13,069 connect as a subscriber to my 725 00:26:13,070 --> 00:26:15,199 data stream and also get a copy 726 00:26:15,200 --> 00:26:16,969 of the information all other nodes keep 727 00:26:16,970 --> 00:26:17,970 on running. 728 00:26:20,870 --> 00:26:23,569 And this would have been working 729 00:26:23,570 --> 00:26:25,679 exactly the same way, whether I started 730 00:26:25,680 --> 00:26:28,189 the camera locally, the Arctic, 731 00:26:28,190 --> 00:26:30,289 we locally or one of those remote 732 00:26:30,290 --> 00:26:33,559 and the Rostker on the third computer, 733 00:26:33,560 --> 00:26:34,949 don't ask about security. 734 00:26:34,950 --> 00:26:36,949 Ross was not built with security in mind. 735 00:26:36,950 --> 00:26:39,139 So the first 736 00:26:39,140 --> 00:26:40,699 thing you should do when you start to use 737 00:26:40,700 --> 00:26:42,889 Ross is do pseudo 738 00:26:42,890 --> 00:26:45,289 IP tables menas flush 739 00:26:45,290 --> 00:26:47,539 because otherwise you will have a very 740 00:26:47,540 --> 00:26:49,849 weird connection, problems 741 00:26:49,850 --> 00:26:51,109 within your network. 742 00:26:51,110 --> 00:26:52,879 So you might not connected to the 743 00:26:52,880 --> 00:26:54,410 Internet during that time. 744 00:26:56,200 --> 00:26:58,479 OK, now we have a camera 745 00:26:58,480 --> 00:27:00,849 who has ever worked with image processing 746 00:27:00,850 --> 00:27:02,979 here in the room, first raise your hand 747 00:27:02,980 --> 00:27:04,479 if you've ever done some kind of image 748 00:27:04,480 --> 00:27:05,789 processing. 749 00:27:05,790 --> 00:27:07,899 OK, so then a short 750 00:27:07,900 --> 00:27:10,149 reminder, if you have a camera, 751 00:27:10,150 --> 00:27:12,219 you need to have 752 00:27:12,220 --> 00:27:14,289 a mapping between the pixels and 753 00:27:14,290 --> 00:27:15,729 the outside world. 754 00:27:15,730 --> 00:27:17,769 So if you just have the image, you do not 755 00:27:17,770 --> 00:27:19,959 really know a relation between 756 00:27:19,960 --> 00:27:21,459 pixel values and 757 00:27:22,510 --> 00:27:25,119 see units, for example, you do not know 758 00:27:25,120 --> 00:27:27,519 how large this object 759 00:27:27,520 --> 00:27:30,069 five meters away in pixels 760 00:27:30,070 --> 00:27:31,719 if you do not have an intrinsic 761 00:27:31,720 --> 00:27:32,949 calibration of the camera. 762 00:27:32,950 --> 00:27:35,019 So intrinsic calibration gives 763 00:27:35,020 --> 00:27:37,209 you a mapping between the 764 00:27:37,210 --> 00:27:40,479 outside world in terms of meters 765 00:27:40,480 --> 00:27:42,699 to what you get from 766 00:27:42,700 --> 00:27:45,069 the camera in terms of pixels 767 00:27:45,070 --> 00:27:47,679 in a very simplified manner. 768 00:27:47,680 --> 00:27:49,929 Um, there's a utility for that. 769 00:27:49,930 --> 00:27:51,549 It's using open sea. 770 00:27:51,550 --> 00:27:53,739 But if you have ever wrote your own 771 00:27:53,740 --> 00:27:56,019 open CV calibration for 772 00:27:56,020 --> 00:27:58,479 camera, um, software, 773 00:27:58,480 --> 00:28:00,279 you know that there are many weird 774 00:28:00,280 --> 00:28:01,569 Koronakis and so on. 775 00:28:01,570 --> 00:28:03,849 And it's really nice to simply 776 00:28:03,850 --> 00:28:06,009 have a GUI for that, um 777 00:28:06,010 --> 00:28:07,229 and show the video and. 778 00:28:10,250 --> 00:28:12,499 So basically, we have everything running 779 00:28:12,500 --> 00:28:14,689 from the previous screen capture. 780 00:28:14,690 --> 00:28:16,759 Now, additionally, start this 781 00:28:16,760 --> 00:28:19,129 calibration note, which 782 00:28:19,130 --> 00:28:21,650 will give me a GUI. 783 00:28:23,880 --> 00:28:25,949 And now if I update on the left 784 00:28:25,950 --> 00:28:28,039 and the plotting of my 785 00:28:28,040 --> 00:28:30,329 Rosebraugh, which is also very nice 786 00:28:30,330 --> 00:28:32,339 introspection utility that comes for 787 00:28:32,340 --> 00:28:34,649 Ross, you can ask 788 00:28:34,650 --> 00:28:36,239 Ross to plot itself. 789 00:28:36,240 --> 00:28:38,339 So to tell you which nodes 790 00:28:38,340 --> 00:28:40,079 are running on which machine, who is 791 00:28:40,080 --> 00:28:42,209 connected to whom, what amount of 792 00:28:42,210 --> 00:28:44,039 data is being transferred in between and 793 00:28:44,040 --> 00:28:45,839 so on. So now that I started this 794 00:28:45,840 --> 00:28:48,059 calibration node and I update this 795 00:28:48,060 --> 00:28:50,249 graph, it appeared here as 796 00:28:50,250 --> 00:28:52,679 a node subscribing to the image, 797 00:28:52,680 --> 00:28:55,829 of course, of the camera and outputting 798 00:28:55,830 --> 00:28:56,830 to the out. 799 00:28:59,110 --> 00:29:01,599 OK, then you do the usual thing, run 800 00:29:01,600 --> 00:29:03,759 around in front of your camera with a 801 00:29:03,760 --> 00:29:06,459 checkerboard, don't have to watch that. 802 00:29:06,460 --> 00:29:08,799 Um, and then in the 803 00:29:08,800 --> 00:29:11,199 end, you get this screen 804 00:29:11,200 --> 00:29:13,479 where you are told 805 00:29:13,480 --> 00:29:16,149 whether you have enough, um, posters 806 00:29:16,150 --> 00:29:18,099 of your checkerboard captured, which 807 00:29:18,100 --> 00:29:20,199 opens if it does not tell you by default, 808 00:29:20,200 --> 00:29:22,269 odds are again a nice tool to 809 00:29:22,270 --> 00:29:24,939 have available instead of, um, 810 00:29:24,940 --> 00:29:26,169 writing your own. 811 00:29:26,170 --> 00:29:28,239 And then finally, if everything is 812 00:29:28,240 --> 00:29:30,579 screen you hit calibrate, it will 813 00:29:30,580 --> 00:29:32,799 do the intrinsic calibration 814 00:29:32,800 --> 00:29:35,199 using open KVI and you get the camera 815 00:29:35,200 --> 00:29:37,689 metrics, the distortion coefficients, 816 00:29:37,690 --> 00:29:39,279 the protection metrics and so on. 817 00:29:39,280 --> 00:29:40,989 So for those of you who know what it is, 818 00:29:40,990 --> 00:29:43,239 fine. For those that don't, uh, doesn't 819 00:29:43,240 --> 00:29:44,919 matter. You can have your camera 820 00:29:44,920 --> 00:29:46,839 intrinsically calibrated and you can use 821 00:29:46,840 --> 00:29:48,969 that down stream. 822 00:29:48,970 --> 00:29:51,099 And it will also automatically start 823 00:29:51,100 --> 00:29:53,709 a topic, um, that's called, 824 00:29:53,710 --> 00:29:55,929 um, camera info. 825 00:29:55,930 --> 00:29:58,239 And there you can get 826 00:29:58,240 --> 00:30:00,009 the parameters of your camera. 827 00:30:00,010 --> 00:30:01,779 And this is used in all different parts 828 00:30:01,780 --> 00:30:03,879 of us, wherever you need 829 00:30:03,880 --> 00:30:06,009 the mapping of cameras to, 830 00:30:06,010 --> 00:30:08,349 um, external values. 831 00:30:09,740 --> 00:30:11,629 What is also written down here on the 832 00:30:11,630 --> 00:30:13,879 command line with the column 833 00:30:13,880 --> 00:30:15,979 Equal Sign is runtime 834 00:30:15,980 --> 00:30:18,079 time name, runtime 835 00:30:18,080 --> 00:30:19,309 name remapping. 836 00:30:19,310 --> 00:30:21,379 So the camera calibrator PI, if 837 00:30:21,380 --> 00:30:23,359 you look into the source code, it 838 00:30:23,360 --> 00:30:25,879 subscribes to the topic named 839 00:30:25,880 --> 00:30:26,880 image. 840 00:30:27,960 --> 00:30:30,149 And now I would have to go about 841 00:30:30,150 --> 00:30:32,309 and change this string in 842 00:30:32,310 --> 00:30:34,439 the code in order to change to camera 843 00:30:34,440 --> 00:30:36,999 in, because that is what in my 844 00:30:37,000 --> 00:30:39,209 network on my robot, the image 845 00:30:39,210 --> 00:30:41,609 is called or what is the topic 846 00:30:41,610 --> 00:30:42,779 under which the image stream is 847 00:30:42,780 --> 00:30:43,979 published. 848 00:30:43,980 --> 00:30:45,569 But the nice thing about Ross is that you 849 00:30:45,570 --> 00:30:48,089 can do this remapping during startup 850 00:30:48,090 --> 00:30:50,369 or even during runtime where just 851 00:30:50,370 --> 00:30:52,380 held it's not OK, whatever is called. 852 00:30:53,460 --> 00:30:55,889 The name image in the road network 853 00:30:55,890 --> 00:30:58,199 for you now is camera image SRAW 854 00:30:58,200 --> 00:31:00,389 and everything will start 855 00:31:00,390 --> 00:31:02,789 working. So that's really nice way 856 00:31:02,790 --> 00:31:05,129 of flexibility in your software 857 00:31:05,130 --> 00:31:07,889 and also how you can write quite modular 858 00:31:07,890 --> 00:31:10,229 meaning, loosely coupled notes, 859 00:31:10,230 --> 00:31:12,089 because they don't have to know about 860 00:31:12,090 --> 00:31:14,039 each other up front. 861 00:31:14,040 --> 00:31:16,139 They just talk to each other using 862 00:31:16,140 --> 00:31:18,809 standardized formats and the naming 863 00:31:18,810 --> 00:31:20,669 the topics on which they talk, you can 864 00:31:20,670 --> 00:31:22,710 specify at orchestration time. 865 00:31:27,320 --> 00:31:30,139 OK, then, once you have your 866 00:31:30,140 --> 00:31:32,179 camera calibrated, you might want to do 867 00:31:32,180 --> 00:31:34,309 some post-processing, for example, if 868 00:31:34,310 --> 00:31:36,319 you have an industrial camera. 869 00:31:36,320 --> 00:31:38,449 Usually it does not 870 00:31:38,450 --> 00:31:40,639 output color images, rather 871 00:31:40,640 --> 00:31:43,129 does output grayscale by your images 872 00:31:43,130 --> 00:31:45,109 and you have to reverse that process and 873 00:31:45,110 --> 00:31:47,899 so on. And for all those low level 874 00:31:47,900 --> 00:31:50,239 processing steps, there's there are notes 875 00:31:50,240 --> 00:31:52,549 available. You just start them and they 876 00:31:52,550 --> 00:31:54,799 take the raw data as input and 877 00:31:54,800 --> 00:31:56,869 output the process data and you 878 00:31:56,870 --> 00:31:59,359 can connect your higher level algorithms 879 00:31:59,360 --> 00:32:01,130 with these data streams. 880 00:32:02,760 --> 00:32:04,889 If we want to write our own 881 00:32:04,890 --> 00:32:07,109 processing note 882 00:32:07,110 --> 00:32:09,209 on Ross, so we want to 883 00:32:09,210 --> 00:32:11,489 take in an image as input 884 00:32:11,490 --> 00:32:13,769 data, we want to apply some open copy 885 00:32:13,770 --> 00:32:15,839 filter in this case and each filter, 886 00:32:15,840 --> 00:32:17,909 and we want to output this data to 887 00:32:17,910 --> 00:32:20,009 our US network, perhaps to 888 00:32:20,010 --> 00:32:22,409 be used as input by a higher 889 00:32:22,410 --> 00:32:24,449 level processing node. 890 00:32:24,450 --> 00:32:26,849 Furthermore, we want to be able during 891 00:32:26,850 --> 00:32:29,099 runtime without restarting anything 892 00:32:29,100 --> 00:32:31,559 to change the parameters of this 893 00:32:31,560 --> 00:32:33,269 filter. We are running on the input 894 00:32:33,270 --> 00:32:35,309 images and this is called dynamic 895 00:32:35,310 --> 00:32:37,449 reconfiguring. And I'll show you 896 00:32:37,450 --> 00:32:38,450 a video. 897 00:32:40,110 --> 00:32:42,470 How all of this works together. 898 00:32:43,830 --> 00:32:46,769 So I start to note, I programed 899 00:32:46,770 --> 00:32:49,229 if I update a glamorous graph, it will 900 00:32:49,230 --> 00:32:51,569 show up as a new note subscribing 901 00:32:51,570 --> 00:32:53,130 to the image topic. 902 00:32:54,240 --> 00:32:56,399 And now I have available a 903 00:32:56,400 --> 00:32:58,499 new topic with a filter applied 904 00:32:58,500 --> 00:33:00,659 to it, and then 905 00:33:00,660 --> 00:33:02,879 I can start in Arctic with the 906 00:33:02,880 --> 00:33:04,529 dynamic reconfigure plugin 907 00:33:06,150 --> 00:33:08,219 where I get all these parameters 908 00:33:08,220 --> 00:33:10,349 I defined as being 909 00:33:10,350 --> 00:33:12,209 dynamically reconfigurable in my 910 00:33:12,210 --> 00:33:13,210 algorithm. 911 00:33:15,070 --> 00:33:17,169 And I can play with 912 00:33:17,170 --> 00:33:19,689 them during runtime and this is all 913 00:33:19,690 --> 00:33:22,029 something out of the box available. 914 00:33:22,030 --> 00:33:23,709 You don't have to sit down and write a 915 00:33:23,710 --> 00:33:25,479 library that can do something like that 916 00:33:25,480 --> 00:33:26,480 for you again. 917 00:33:28,600 --> 00:33:30,729 And this is really nice to get 918 00:33:30,730 --> 00:33:32,919 to quickly get to 919 00:33:32,920 --> 00:33:34,150 something that does 920 00:33:35,800 --> 00:33:37,749 something interesting from a point of 921 00:33:37,750 --> 00:33:39,069 view of the application. 922 00:33:41,990 --> 00:33:44,359 And all of this takes just 36 923 00:33:44,360 --> 00:33:46,970 lines of C++ code, I think this is quite. 924 00:33:48,330 --> 00:33:50,369 Few code in order to have something 925 00:33:50,370 --> 00:33:52,349 that's networked, transparent, that can 926 00:33:52,350 --> 00:33:54,899 be dynamically reconfigured 927 00:33:54,900 --> 00:33:57,059 remapped at runtime and so on, so 928 00:33:57,060 --> 00:33:59,639 it might be better than linking libraries 929 00:33:59,640 --> 00:34:01,680 to each other to change something. 930 00:34:03,680 --> 00:34:05,939 OK, so we had mono 931 00:34:05,940 --> 00:34:06,919 Skype cameras. 932 00:34:06,920 --> 00:34:09,138 Now let's do stereo, if you like, we 933 00:34:09,139 --> 00:34:11,419 humans, we have two eyes so we can 934 00:34:11,420 --> 00:34:12,749 process the disparity. 935 00:34:12,750 --> 00:34:15,109 For example, I see this microphone more 936 00:34:15,110 --> 00:34:16,968 to the right of my right eye than to my 937 00:34:16,969 --> 00:34:18,589 left eye. And from that, I can 938 00:34:18,590 --> 00:34:20,388 triangulate the three dimensional 939 00:34:20,389 --> 00:34:21,948 position of this microphone. 940 00:34:21,949 --> 00:34:24,109 And also you can do that with robots 941 00:34:24,110 --> 00:34:25,759 and you usually want to perceive the 942 00:34:25,760 --> 00:34:27,888 world in three dimensions in order to 943 00:34:27,889 --> 00:34:30,199 figure out where can the robot move along 944 00:34:31,219 --> 00:34:33,468 to two object detection in three 945 00:34:33,469 --> 00:34:35,059 dimensions and so on. 946 00:34:35,060 --> 00:34:37,189 So again, we write a launch file. 947 00:34:37,190 --> 00:34:39,349 The note this time is called 948 00:34:39,350 --> 00:34:42,229 UFOs. You know, those are all 949 00:34:42,230 --> 00:34:43,230 available. 950 00:34:44,040 --> 00:34:46,198 We again, at some parameters to it 951 00:34:46,199 --> 00:34:48,419 this time, we have two devices left 952 00:34:48,420 --> 00:34:50,519 camera and the right camera 953 00:34:50,520 --> 00:34:53,099 define which webcam is attached to which 954 00:34:53,100 --> 00:34:56,129 Linux video device, and 955 00:34:56,130 --> 00:34:57,749 then we can start that. 956 00:34:57,750 --> 00:35:00,239 We have this note publishes two topics. 957 00:35:00,240 --> 00:35:02,609 Under those two names, we can again start 958 00:35:02,610 --> 00:35:04,139 ArchCity, go and have a look at them. 959 00:35:05,280 --> 00:35:07,709 If we want to do serious calibration, 960 00:35:07,710 --> 00:35:09,689 which not only does the intrinsic 961 00:35:09,690 --> 00:35:11,759 calibration, but also correlates what 962 00:35:11,760 --> 00:35:13,949 the right camera sees to what the left 963 00:35:13,950 --> 00:35:16,349 camera perceives at the same moment, 964 00:35:16,350 --> 00:35:18,429 and also the synchronization 965 00:35:18,430 --> 00:35:20,879 and some other algorithmic 966 00:35:20,880 --> 00:35:22,829 transformations of the images. 967 00:35:22,830 --> 00:35:25,349 And there's also this calibrate 968 00:35:25,350 --> 00:35:27,179 the note for it pretty much looks the 969 00:35:27,180 --> 00:35:29,159 same way as previously for the mono 970 00:35:29,160 --> 00:35:30,719 camera, only with two images. 971 00:35:32,530 --> 00:35:34,959 OK, once we have our stereo 972 00:35:34,960 --> 00:35:37,089 camera calibrated, we 973 00:35:37,090 --> 00:35:39,429 can use the stereo image 974 00:35:39,430 --> 00:35:42,189 to reconstruct a three dimensional 975 00:35:42,190 --> 00:35:44,409 image from our two cameras and all 976 00:35:44,410 --> 00:35:46,809 of that this takes is a launch 977 00:35:46,810 --> 00:35:49,659 of about five lines of XML 978 00:35:49,660 --> 00:35:51,579 in addition to what we wrote down to 979 00:35:51,580 --> 00:35:53,439 start our cameras. 980 00:35:53,440 --> 00:35:55,689 And this note also 981 00:35:55,690 --> 00:35:58,299 has available dynamic reconfigure GUI 982 00:35:58,300 --> 00:35:59,799 in order to change the exterior 983 00:35:59,800 --> 00:36:01,839 reconstruction parameters. 984 00:36:01,840 --> 00:36:04,509 So if you get into Steria reconstruction, 985 00:36:04,510 --> 00:36:06,369 there are a whole lot of parameters you 986 00:36:06,370 --> 00:36:07,329 have to set. 987 00:36:07,330 --> 00:36:09,459 You can do that during runtime and don't 988 00:36:09,460 --> 00:36:11,649 have to write a single line of code 989 00:36:11,650 --> 00:36:12,789 for it. 990 00:36:12,790 --> 00:36:15,129 And then the result, our point clouds 991 00:36:15,130 --> 00:36:17,109 and point clouds are really nice format 992 00:36:17,110 --> 00:36:19,989 because there's a library available 993 00:36:19,990 --> 00:36:21,849 which does what open S.V. 994 00:36:21,850 --> 00:36:24,099 did for image processing four point 995 00:36:24,100 --> 00:36:24,519 clouds. 996 00:36:24,520 --> 00:36:26,919 So a really large 997 00:36:26,920 --> 00:36:29,859 close to state of the art library 998 00:36:29,860 --> 00:36:32,139 with all kinds of algorithms that 999 00:36:32,140 --> 00:36:34,359 allow you to get 1000 00:36:34,360 --> 00:36:36,609 with only a few lines of code 1001 00:36:36,610 --> 00:36:39,219 to something that's applicable 1002 00:36:39,220 --> 00:36:40,839 to what you actually want to do instead 1003 00:36:40,840 --> 00:36:42,629 of writing low level. 1004 00:36:42,630 --> 00:36:44,799 Take this point, compare it with the one 1005 00:36:44,800 --> 00:36:46,030 to the right and do something. 1006 00:36:48,470 --> 00:36:50,599 OK, then the last kind of 1007 00:36:50,600 --> 00:36:52,969 cameras are the Kinect 1008 00:36:52,970 --> 00:36:55,489 cameras, unfortunately, open 1009 00:36:55,490 --> 00:36:57,619 the night was bought by Apple and 1010 00:36:57,620 --> 00:36:59,179 they closed it down, which is a bit 1011 00:36:59,180 --> 00:37:01,489 unfortunate because those devices 1012 00:37:01,490 --> 00:37:03,949 replaced in academia research 1013 00:37:03,950 --> 00:37:06,229 for, I think one hundred fifty euros, 1014 00:37:06,230 --> 00:37:08,299 replaced devices that cost about 1015 00:37:08,300 --> 00:37:10,489 20000 euros and they're good 1016 00:37:10,490 --> 00:37:12,859 enough compared to laser range 1017 00:37:12,860 --> 00:37:14,419 finders and so on. 1018 00:37:14,420 --> 00:37:16,489 And to start this device 1019 00:37:16,490 --> 00:37:18,229 heavy debate, I have to point Claude 1020 00:37:18,230 --> 00:37:19,399 available. 1021 00:37:19,400 --> 00:37:22,159 You just type in this line at the 1022 00:37:22,160 --> 00:37:24,229 command prompt. You start 1023 00:37:24,230 --> 00:37:26,209 launch for someone else has written for 1024 00:37:26,210 --> 00:37:28,499 you and then you got a point 1025 00:37:28,500 --> 00:37:31,309 cloud you can process in your network. 1026 00:37:31,310 --> 00:37:32,689 So let's have a look at this. 1027 00:37:39,210 --> 00:37:41,339 So this is the Archabbey image from 1028 00:37:41,340 --> 00:37:43,469 this year, and this is the 1029 00:37:43,470 --> 00:37:46,679 tough image, which is all 1030 00:37:46,680 --> 00:37:48,899 quite nice, but what 1031 00:37:48,900 --> 00:37:50,969 you really want to have, if you get a 1032 00:37:50,970 --> 00:37:52,649 point cloud out of something, have a look 1033 00:37:52,650 --> 00:37:55,379 at the cloud and that interactively. 1034 00:37:55,380 --> 00:37:58,169 And this is what you can do with our. 1035 00:37:58,170 --> 00:38:00,239 So it's just not obvious at 1036 00:38:00,240 --> 00:38:02,819 this point. Cloud, plug in 1037 00:38:02,820 --> 00:38:05,459 to it, tell it on which topic to listen 1038 00:38:05,460 --> 00:38:07,529 how to do the color transform. 1039 00:38:07,530 --> 00:38:09,589 And then you have three dimensional 1040 00:38:09,590 --> 00:38:11,879 clouds, introspective, built 1041 00:38:11,880 --> 00:38:14,129 by the human developer who needs to debug 1042 00:38:14,130 --> 00:38:15,839 what's going wrong at the moment. 1043 00:38:15,840 --> 00:38:17,939 And writing something like this viewer by 1044 00:38:17,940 --> 00:38:20,129 hand, I can tell you, takes 1045 00:38:20,130 --> 00:38:22,199 you more time than you planned to 1046 00:38:22,200 --> 00:38:24,899 spend on your whole hobby robotics. 1047 00:38:24,900 --> 00:38:26,280 Unload my printer project, 1048 00:38:27,390 --> 00:38:29,579 so perhaps have a 1049 00:38:29,580 --> 00:38:32,099 few minutes time to trying 1050 00:38:32,100 --> 00:38:33,599 to do this life. 1051 00:38:34,890 --> 00:38:36,960 So let's start our Oscar, let's 1052 00:38:38,040 --> 00:38:39,119 launch. 1053 00:38:42,540 --> 00:38:44,189 I'm not sure if it will be working 1054 00:38:44,190 --> 00:38:46,349 because my laptop has not really 1055 00:38:46,350 --> 00:38:48,449 enough power for this, but we'll see. 1056 00:38:57,420 --> 00:39:00,629 So now at this plugin, that specifically 1057 00:39:00,630 --> 00:39:01,920 for porn clouds. 1058 00:39:07,780 --> 00:39:10,269 And of course, it's 1059 00:39:10,270 --> 00:39:11,920 perhaps not working. 1060 00:39:18,710 --> 00:39:20,809 I won't spend too much time on 1061 00:39:20,810 --> 00:39:21,810 it. 1062 00:39:27,960 --> 00:39:30,299 Let me have one look at the driver's side 1063 00:39:30,300 --> 00:39:31,300 of things. 1064 00:39:34,230 --> 00:39:36,389 OK, I forgot to check register def 1065 00:39:36,390 --> 00:39:37,390 images. 1066 00:39:44,010 --> 00:39:45,629 The purpose I should look at something 1067 00:39:45,630 --> 00:39:48,030 that's not out of range of the camera. 1068 00:39:52,720 --> 00:39:54,669 There we go. So now we have a three 1069 00:39:54,670 --> 00:39:56,960 dimensional view here of our war. 1070 00:40:07,280 --> 00:40:09,349 OK, and this was without writing 1071 00:40:09,350 --> 00:40:11,479 a single line of code and 1072 00:40:11,480 --> 00:40:13,309 also you can do all the other nifty 1073 00:40:13,310 --> 00:40:14,630 things I talked about to it. 1074 00:40:16,100 --> 00:40:17,300 OK. That was the Ronke. 1075 00:40:28,480 --> 00:40:30,909 Yep, obviously, PowerPoint 1076 00:40:30,910 --> 00:40:32,829 and connecting Beamer's to laptops is 1077 00:40:32,830 --> 00:40:34,750 more complicated in robotics these days. 1078 00:40:36,700 --> 00:40:38,889 OK, um, where 1079 00:40:38,890 --> 00:40:40,149 did we leave off? 1080 00:40:40,150 --> 00:40:41,929 I think that was here. 1081 00:40:41,930 --> 00:40:44,049 OK, and the nice thing about it 1082 00:40:44,050 --> 00:40:46,299 is if you have a stereo camera part and 1083 00:40:46,300 --> 00:40:47,859 the Kinect on your robot, which is a 1084 00:40:47,860 --> 00:40:50,439 quite common setup, they output 1085 00:40:50,440 --> 00:40:52,569 the same kind of data 1086 00:40:52,570 --> 00:40:55,329 format. So you can switch in between 1087 00:40:55,330 --> 00:40:57,819 between your Kinect style device 1088 00:40:57,820 --> 00:40:59,979 and your stereo camera pair. 1089 00:40:59,980 --> 00:41:02,259 And this is highly valuable because 1090 00:41:02,260 --> 00:41:04,989 Kinect are much better 1091 00:41:04,990 --> 00:41:07,359 at reconstructing what is in the range 1092 00:41:07,360 --> 00:41:10,239 of like half a meter to four meters, 1093 00:41:10,240 --> 00:41:12,639 even in the dark stereo 1094 00:41:12,640 --> 00:41:14,649 cameras. Obviously, you need light like 1095 00:41:14,650 --> 00:41:16,809 ourselves to see 1096 00:41:16,810 --> 00:41:18,639 anything. But on the other hand, there 1097 00:41:18,640 --> 00:41:20,559 are much better at doing long distance 1098 00:41:20,560 --> 00:41:22,989 reconstruction or if there 1099 00:41:22,990 --> 00:41:25,389 would be some shining 1100 00:41:25,390 --> 00:41:27,399 into this room. OK, I know this doesn't 1101 00:41:27,400 --> 00:41:28,690 happen that hacker spaces, but 1102 00:41:29,800 --> 00:41:31,749 then Kinect style cameras do not really 1103 00:41:31,750 --> 00:41:32,739 work well. 1104 00:41:32,740 --> 00:41:34,839 So if you have 1105 00:41:34,840 --> 00:41:37,029 your middleware set up in a way 1106 00:41:37,030 --> 00:41:38,469 that it does, no matter where you get the 1107 00:41:38,470 --> 00:41:40,899 input from, you can trust at runtime 1108 00:41:40,900 --> 00:41:43,239 switch between the two devices 1109 00:41:43,240 --> 00:41:45,499 that feed you the data and all your high 1110 00:41:45,500 --> 00:41:46,539 level algorithms. 1111 00:41:46,540 --> 00:41:48,609 For example, collision avoidance 1112 00:41:48,610 --> 00:41:50,800 will continue to work the way they did. 1113 00:41:52,540 --> 00:41:54,759 OK, now I was talking a whole 1114 00:41:54,760 --> 00:41:56,619 lot about sensors, so cameras are 1115 00:41:56,620 --> 00:41:58,839 obviously only one example of sensors 1116 00:41:58,840 --> 00:42:00,309 you might find on your robot. 1117 00:42:00,310 --> 00:42:02,499 They are drivers for laser 1118 00:42:02,500 --> 00:42:04,419 scanners, for laser line scanners, for 1119 00:42:04,420 --> 00:42:06,339 distance sensors, for what? 1120 00:42:06,340 --> 00:42:08,799 I do not know at all 1121 00:42:08,800 --> 00:42:10,330 what there is these days. 1122 00:42:11,680 --> 00:42:13,899 There's a listing on the Ross 1123 00:42:13,900 --> 00:42:15,369 wiki page. 1124 00:42:15,370 --> 00:42:17,469 The link is at the end of the slides. 1125 00:42:17,470 --> 00:42:19,059 Have a look. Perhaps your device is 1126 00:42:19,060 --> 00:42:20,559 already supported. 1127 00:42:20,560 --> 00:42:22,239 Otherwise, ask me reimplemented. 1128 00:42:22,240 --> 00:42:24,190 Quite some drivers, for example, for the, 1129 00:42:26,070 --> 00:42:28,569 uh, the Archibold elitist, 1130 00:42:28,570 --> 00:42:29,619 uh, w. 1131 00:42:29,620 --> 00:42:32,229 S a twelve moderate 1132 00:42:32,230 --> 00:42:34,809 to sixteen or whatever those leads 1133 00:42:34,810 --> 00:42:36,789 you find a lot of in the basement. 1134 00:42:36,790 --> 00:42:38,169 We have a Ross driver for that. 1135 00:42:38,170 --> 00:42:40,329 So just connect them to a beagle bone and 1136 00:42:40,330 --> 00:42:42,819 you can have a display 1137 00:42:42,820 --> 00:42:44,439 with us. 1138 00:42:44,440 --> 00:42:46,599 So it's also Ross does not only 1139 00:42:46,600 --> 00:42:48,879 work nice for robots, of course it 1140 00:42:48,880 --> 00:42:50,949 is targeted towards robots, but 1141 00:42:50,950 --> 00:42:52,869 whether you are looking for a flexible 1142 00:42:52,870 --> 00:42:55,149 middleware that does not need hard, 1143 00:42:55,150 --> 00:42:56,949 real time constraints and if you don't 1144 00:42:56,950 --> 00:42:59,169 know what hard time constraints are, 1145 00:42:59,170 --> 00:43:00,399 you might not need them. 1146 00:43:01,480 --> 00:43:03,729 So if if if best effort 1147 00:43:03,730 --> 00:43:05,799 is good enough, so if you can just check 1148 00:43:05,800 --> 00:43:07,749 whether your data is being processed fast 1149 00:43:07,750 --> 00:43:10,089 enough and if it is, it's fine, then 1150 00:43:10,090 --> 00:43:12,309 Ross is quite nice for a whole lot 1151 00:43:12,310 --> 00:43:14,529 of Hekker and 1152 00:43:14,530 --> 00:43:16,689 make our project OK. 1153 00:43:16,690 --> 00:43:18,219 But now let's get into robots. 1154 00:43:18,220 --> 00:43:20,499 We want to model our robot for this. 1155 00:43:20,500 --> 00:43:22,569 There's some Elbaz description 1156 00:43:22,570 --> 00:43:24,669 format called You are D.F.. 1157 00:43:24,670 --> 00:43:26,499 You start out with the top level tech 1158 00:43:26,500 --> 00:43:28,659 robot, give it a name, then you add 1159 00:43:28,660 --> 00:43:30,969 links. Links are the rigid 1160 00:43:30,970 --> 00:43:33,639 objects that Connect joins together. 1161 00:43:33,640 --> 00:43:35,379 Like, for example, if I want to model my 1162 00:43:35,380 --> 00:43:37,899 arm, perhaps I can model letters to links 1163 00:43:37,900 --> 00:43:40,269 and to Chines in between the shoulder 1164 00:43:40,270 --> 00:43:42,399 and the elbow, and then 1165 00:43:42,400 --> 00:43:43,629 you have to give them a visual 1166 00:43:43,630 --> 00:43:45,699 appearance. This can be either geometric 1167 00:43:45,700 --> 00:43:47,769 primitives or meshes and meshes are 1168 00:43:47,770 --> 00:43:49,389 nice because you can create them of 1169 00:43:49,390 --> 00:43:51,459 Blendr, you can export them from your 1170 00:43:51,460 --> 00:43:53,619 cat application and you can 1171 00:43:53,620 --> 00:43:55,929 just tell your use mesh 1172 00:43:55,930 --> 00:43:58,449 for this link and it's there 1173 00:43:58,450 --> 00:44:00,639 and there it is where you can easily 1174 00:44:00,640 --> 00:44:02,739 either model your robot before 1175 00:44:02,740 --> 00:44:05,439 you built it, which is quite nice 1176 00:44:05,440 --> 00:44:07,089 because then you can explore it's 1177 00:44:07,090 --> 00:44:08,919 kinematics before you find out they're 1178 00:44:08,920 --> 00:44:09,920 wrong. 1179 00:44:11,140 --> 00:44:13,269 Or you could just model your robot to 1180 00:44:13,270 --> 00:44:15,219 do all the things I'm going to show in 1181 00:44:15,220 --> 00:44:16,869 the next or ten minutes. 1182 00:44:16,870 --> 00:44:19,179 Um, the links 1183 00:44:19,180 --> 00:44:21,129 are connected by train stay. 1184 00:44:21,130 --> 00:44:22,209 They are different types. 1185 00:44:22,210 --> 00:44:23,889 So if you have electrical motor, for 1186 00:44:23,890 --> 00:44:26,019 example, a cerebral motor, this is 1187 00:44:26,020 --> 00:44:29,079 a type revolution and it has 1188 00:44:29,080 --> 00:44:30,699 joined limits. You cannot move it 1189 00:44:30,700 --> 00:44:32,409 endlessly in one direction. 1190 00:44:32,410 --> 00:44:34,779 And you specify all of these kinds 1191 00:44:34,780 --> 00:44:35,780 in here. 1192 00:44:36,520 --> 00:44:38,619 OK, once you're done with 1193 00:44:38,620 --> 00:44:39,909 that, you might want to have a look at 1194 00:44:39,910 --> 00:44:41,589 it. And again, the nice thing about is 1195 00:44:41,590 --> 00:44:43,929 other people already built 1196 00:44:43,930 --> 00:44:46,299 software so that you can focus 1197 00:44:46,300 --> 00:44:48,329 on what you're interested in. 1198 00:44:50,690 --> 00:44:52,969 So I wrote down that's about 1199 00:44:52,970 --> 00:44:54,189 as in one hundred lines, long 1200 00:44:54,190 --> 00:44:55,789 description, and now here's my very 1201 00:44:55,790 --> 00:44:57,889 simple robot consisting of three links, 1202 00:44:57,890 --> 00:45:01,189 a bass cylinder and 1203 00:45:01,190 --> 00:45:03,259 two arms segments connected 1204 00:45:03,260 --> 00:45:04,339 by two joints. 1205 00:45:04,340 --> 00:45:06,589 And I can immediately move them around 1206 00:45:06,590 --> 00:45:08,809 without writing any code through 1207 00:45:08,810 --> 00:45:10,549 this GUI. 1208 00:45:10,550 --> 00:45:13,369 And this way you can also explore 1209 00:45:13,370 --> 00:45:15,259 how you want to build your real robot. 1210 00:45:15,260 --> 00:45:17,599 For example, if you go the usual hobbyist 1211 00:45:17,600 --> 00:45:20,599 way and you servo motors and 1212 00:45:20,600 --> 00:45:22,610 aluminum profiles to build it. 1213 00:45:24,290 --> 00:45:26,659 OK, now comes something really 1214 00:45:26,660 --> 00:45:28,819 cool, if you have a robot that 1215 00:45:28,820 --> 00:45:31,039 has a couple of joints, it gets 1216 00:45:31,040 --> 00:45:33,349 really complicated to define 1217 00:45:33,350 --> 00:45:35,479 in which position, which point has 1218 00:45:35,480 --> 00:45:37,489 to move in order for the robot to get to 1219 00:45:37,490 --> 00:45:38,839 a certain point. 1220 00:45:38,840 --> 00:45:40,339 This is the problem of inverse 1221 00:45:40,340 --> 00:45:41,539 kinematics. 1222 00:45:41,540 --> 00:45:43,999 And usually you do that by 1223 00:45:44,000 --> 00:45:46,099 calculating something called the inverse 1224 00:45:46,100 --> 00:45:47,149 Jacobean metrics. 1225 00:45:47,150 --> 00:45:48,859 That already sounds like a lot of math 1226 00:45:48,860 --> 00:45:51,049 math. It's much more than that. 1227 00:45:51,050 --> 00:45:52,050 And. 1228 00:45:53,230 --> 00:45:55,359 You might not want to get too deep into 1229 00:45:55,360 --> 00:45:57,189 this if you just want to unload your 3D 1230 00:45:57,190 --> 00:45:58,509 printer, right? 1231 00:45:58,510 --> 00:46:00,819 So there's something 1232 00:46:00,820 --> 00:46:03,039 since I think the beginning of this 1233 00:46:03,040 --> 00:46:05,499 year called the move, it set up a system 1234 00:46:05,500 --> 00:46:07,719 and you just fit your description into 1235 00:46:07,720 --> 00:46:08,720 that. 1236 00:46:12,320 --> 00:46:14,839 It creates some nice GWI 1237 00:46:14,840 --> 00:46:16,730 where you can load your robot 1238 00:46:17,990 --> 00:46:20,479 again, it's immediately visualized 1239 00:46:20,480 --> 00:46:22,129 writing such a GUI again. 1240 00:46:22,130 --> 00:46:24,229 What has taken some people? 1241 00:46:24,230 --> 00:46:25,969 I am very grateful for that. 1242 00:46:25,970 --> 00:46:28,159 They did it a lot of time to do that. 1243 00:46:28,160 --> 00:46:30,349 Then you do all kinds of 1244 00:46:30,350 --> 00:46:32,179 configuration, but it only takes you 1245 00:46:32,180 --> 00:46:33,859 about two to three minutes. 1246 00:46:33,860 --> 00:46:35,419 Can more or less use the default 1247 00:46:35,420 --> 00:46:37,519 parameters if you have a robot 1248 00:46:37,520 --> 00:46:39,319 with multiple arms to define which 1249 00:46:39,320 --> 00:46:41,299 kinematic change our planning groups and 1250 00:46:41,300 --> 00:46:42,300 so on. 1251 00:46:42,800 --> 00:46:45,019 And in the end, you 1252 00:46:45,020 --> 00:46:45,929 start all of this. 1253 00:46:45,930 --> 00:46:47,209 This is very well described. 1254 00:46:47,210 --> 00:46:49,369 So this talk obviously is not 1255 00:46:49,370 --> 00:46:50,719 a tutorial. 1256 00:46:50,720 --> 00:46:52,939 You can just look at the website. 1257 00:46:52,940 --> 00:46:55,279 It's really easy to figure out, OK, 1258 00:46:55,280 --> 00:46:57,049 afterwards you have your 1259 00:46:58,370 --> 00:47:00,679 kinematic description of your robot 1260 00:47:00,680 --> 00:47:02,899 and you can use 1261 00:47:02,900 --> 00:47:05,479 this. They provide a demo 1262 00:47:05,480 --> 00:47:07,579 launch file for each robot you 1263 00:47:07,580 --> 00:47:09,049 have exported from the GUI. 1264 00:47:09,050 --> 00:47:10,050 I just showed. 1265 00:47:11,000 --> 00:47:12,709 And with this, you can do something 1266 00:47:12,710 --> 00:47:13,969 amazing out of the box. 1267 00:47:15,940 --> 00:47:17,770 You start to see more lunchpail. 1268 00:47:19,110 --> 00:47:20,909 It starts our August for you, there's a 1269 00:47:20,910 --> 00:47:23,009 robot and you can 1270 00:47:23,010 --> 00:47:25,469 do immediately do motion 1271 00:47:25,470 --> 00:47:27,539 planning at various state 1272 00:47:27,540 --> 00:47:29,309 of the art sampling based motion 1273 00:47:29,310 --> 00:47:31,469 planners. So, for example, I just told 1274 00:47:31,470 --> 00:47:33,539 it, go to some valid position and now it 1275 00:47:33,540 --> 00:47:34,949 plan the path to go there. 1276 00:47:34,950 --> 00:47:37,049 Of course, this seems trivial with such 1277 00:47:37,050 --> 00:47:39,089 a robot, but if you have a robot with six 1278 00:47:39,090 --> 00:47:41,399 degrees of freedom or even seven degrees 1279 00:47:41,400 --> 00:47:43,619 of freedom, you really 1280 00:47:43,620 --> 00:47:45,089 can't do that in your head. 1281 00:47:45,090 --> 00:47:47,189 It's very, very counterintuitive 1282 00:47:47,190 --> 00:47:49,199 what these drones are doing in order for 1283 00:47:49,200 --> 00:47:51,299 the robot to simply move along a straight 1284 00:47:51,300 --> 00:47:53,819 line in our Cartesian three dimensional 1285 00:47:53,820 --> 00:47:55,949 space until you get 1286 00:47:55,950 --> 00:47:58,139 it out of the box. And the nice thing is 1287 00:47:58,140 --> 00:48:00,359 this is not just the dummy robot, but 1288 00:48:00,360 --> 00:48:02,789 if you would connect your servo motors 1289 00:48:02,790 --> 00:48:05,789 to a controller through some low level 1290 00:48:05,790 --> 00:48:08,369 set up so that you can specify 1291 00:48:08,370 --> 00:48:10,769 what number do I have to send where, 1292 00:48:10,770 --> 00:48:12,419 so that your servo moves into this 1293 00:48:12,420 --> 00:48:14,789 position, you can immediately use 1294 00:48:14,790 --> 00:48:16,709 this GUI to move your real robot to do 1295 00:48:16,710 --> 00:48:18,689 path planning. If you read a robot and 1296 00:48:18,690 --> 00:48:21,089 you can even put a Kinect next 1297 00:48:21,090 --> 00:48:23,369 to your robot and do collision 1298 00:48:23,370 --> 00:48:25,469 free path planning using 1299 00:48:25,470 --> 00:48:26,939 the movie framework. 1300 00:48:26,940 --> 00:48:29,129 So this is really, Chris, 1301 00:48:29,130 --> 00:48:31,679 great that this is available nowadays. 1302 00:48:31,680 --> 00:48:33,899 That took, um, 1303 00:48:33,900 --> 00:48:36,749 I know of many projects in academia 1304 00:48:36,750 --> 00:48:39,029 that took like a year just to get 1305 00:48:39,030 --> 00:48:42,179 this basic motion planning running 1306 00:48:42,180 --> 00:48:44,909 stable and reliably with their robots. 1307 00:48:44,910 --> 00:48:47,189 And now you get it like in 10 1308 00:48:47,190 --> 00:48:49,379 to 20 minutes, which is 1309 00:48:49,380 --> 00:48:51,089 then a whole day if you consider 1310 00:48:51,090 --> 00:48:52,090 debugging. 1311 00:48:52,830 --> 00:48:54,689 But that's really, really a large 1312 00:48:54,690 --> 00:48:55,690 improvement. 1313 00:48:57,450 --> 00:49:00,119 And if you hate Cui's, that's fine. 1314 00:49:00,120 --> 00:49:02,219 All everything I'm showing 1315 00:49:02,220 --> 00:49:04,469 is well separated between the GUI 1316 00:49:04,470 --> 00:49:06,149 components and the code. 1317 00:49:06,150 --> 00:49:08,669 So if you want to do path planning, not 1318 00:49:08,670 --> 00:49:10,739 through a GUI, but from your code 1319 00:49:10,740 --> 00:49:12,599 because you recognize an object in your 1320 00:49:12,600 --> 00:49:14,669 image, of course there's a 1321 00:49:14,670 --> 00:49:16,799 python and a C++ app to tell 1322 00:49:16,800 --> 00:49:18,209 the robot, OK, go to this point. 1323 00:49:18,210 --> 00:49:19,889 You don't need to go through the GUI. 1324 00:49:21,660 --> 00:49:23,729 OK, if you want to go even further and 1325 00:49:23,730 --> 00:49:25,949 want to simulate your real robot 1326 00:49:25,950 --> 00:49:28,079 or your robot, you might want to 1327 00:49:28,080 --> 00:49:31,199 build someday a gazebo for it 1328 00:49:31,200 --> 00:49:33,209 physically based simulator that's 1329 00:49:33,210 --> 00:49:35,429 integrated quite tightly with these 1330 00:49:35,430 --> 00:49:36,749 days. 1331 00:49:36,750 --> 00:49:39,179 And there are a couple of details 1332 00:49:39,180 --> 00:49:40,539 I leave out due to time. 1333 00:49:40,540 --> 00:49:42,209 There are different formats, but you can 1334 00:49:42,210 --> 00:49:44,039 convert them in the end. 1335 00:49:45,550 --> 00:49:47,679 And since 1336 00:49:47,680 --> 00:49:50,349 Ross is built the way I thought 1337 00:49:50,350 --> 00:49:52,209 told you about in the beginning of the 1338 00:49:52,210 --> 00:49:54,339 topics, the dynamic renaming and 1339 00:49:54,340 --> 00:49:56,829 some mechanisms for us 1340 00:49:56,830 --> 00:49:58,989 to not use the real 1341 00:49:58,990 --> 00:50:01,059 progress of time like what we call 1342 00:50:01,060 --> 00:50:03,429 wall time, usually in simulation work, 1343 00:50:03,430 --> 00:50:05,739 but your simulated time. 1344 00:50:05,740 --> 00:50:07,809 So if your computer, for example, 1345 00:50:07,810 --> 00:50:09,969 is too slow to run the simulation in real 1346 00:50:09,970 --> 00:50:12,699 time, so it is only it 1347 00:50:12,700 --> 00:50:14,919 can progress one with 30 1348 00:50:14,920 --> 00:50:17,139 percent of the speed, the real 1349 00:50:17,140 --> 00:50:19,299 world time progresses, all 1350 00:50:19,300 --> 00:50:21,489 of your notes will automatically adapt 1351 00:50:21,490 --> 00:50:23,109 to that. So if you say sleep ten 1352 00:50:23,110 --> 00:50:25,179 milliseconds through the rosapepe, 1353 00:50:25,180 --> 00:50:28,089 it will sleep ten seconds 1354 00:50:28,090 --> 00:50:30,369 or ten simulated seconds 1355 00:50:32,650 --> 00:50:33,459 all by itself. 1356 00:50:33,460 --> 00:50:35,049 You don't have to recompile anything to 1357 00:50:35,050 --> 00:50:37,419 run your notes against simulation or 1358 00:50:37,420 --> 00:50:38,470 on your robot. 1359 00:50:39,550 --> 00:50:41,709 And of course, you can do all kinds of 1360 00:50:41,710 --> 00:50:43,059 software engineering, test driven 1361 00:50:43,060 --> 00:50:44,709 development, continuous integration, 1362 00:50:44,710 --> 00:50:46,359 there are people who integrated this with 1363 00:50:46,360 --> 00:50:48,639 the conserva and so on. 1364 00:50:48,640 --> 00:50:50,559 You can do distributed development in 1365 00:50:50,560 --> 00:50:52,479 your hackerspace because perhaps not 1366 00:50:52,480 --> 00:50:54,429 everyone has the same robot available. 1367 00:50:54,430 --> 00:50:56,679 You can still develop 1368 00:50:56,680 --> 00:50:59,349 algorithms, test them on the simulator, 1369 00:50:59,350 --> 00:51:00,429 and then once you're back in the 1370 00:51:00,430 --> 00:51:02,739 hackerspace tested on a real one, 1371 00:51:02,740 --> 00:51:04,780 stay close to the button in that moment. 1372 00:51:06,220 --> 00:51:08,409 OK, let's assume we attached 1373 00:51:08,410 --> 00:51:10,599 a webcam and an Nazi video camera to 1374 00:51:10,600 --> 00:51:13,479 the robot we modeled earlier 1375 00:51:13,480 --> 00:51:16,119 and converted the format 1376 00:51:16,120 --> 00:51:17,979 into each other. 1377 00:51:17,980 --> 00:51:20,619 Now, I started the gazebo simulator 1378 00:51:20,620 --> 00:51:21,969 at this robot here. 1379 00:51:21,970 --> 00:51:23,709 And now there is the physics engine, the 1380 00:51:23,710 --> 00:51:25,629 background, so you can throw objects 1381 00:51:25,630 --> 00:51:27,249 around and if they hit each other, it 1382 00:51:27,250 --> 00:51:29,199 will happen more or less physically 1383 00:51:29,200 --> 00:51:31,389 plausible. So they're using 1384 00:51:31,390 --> 00:51:33,579 different game physics engines like 1385 00:51:33,580 --> 00:51:35,769 old and bullet in the background. 1386 00:51:35,770 --> 00:51:37,899 And so I could flip over this book 1387 00:51:37,900 --> 00:51:39,789 bookshelf if I hit it hard enough in the 1388 00:51:39,790 --> 00:51:42,039 simulation so you can really do some 1389 00:51:42,040 --> 00:51:44,049 kind of advanced manipulation. 1390 00:51:44,050 --> 00:51:46,269 And as I said, we just added a simulated 1391 00:51:46,270 --> 00:51:48,369 camera and you can get images 1392 00:51:48,370 --> 00:51:50,199 from the simulated camera in the same 1393 00:51:50,200 --> 00:51:52,149 manner you get it from the real camera, 1394 00:51:52,150 --> 00:51:54,339 which gives you opportunities to do what 1395 00:51:54,340 --> 00:51:56,529 I started to call a robot unit 1396 00:51:56,530 --> 00:51:58,929 testing. So you can use your. 1397 00:52:00,060 --> 00:52:02,309 Software one to one, to use 1398 00:52:02,310 --> 00:52:04,019 it in your real robot, run it against a 1399 00:52:04,020 --> 00:52:07,019 simulated environment Wolfen, 1400 00:52:07,020 --> 00:52:09,149 run it against the simulated robot with 1401 00:52:09,150 --> 00:52:11,609 simulated environment, and thereby ensure 1402 00:52:11,610 --> 00:52:13,589 that if you change the code, nothing 1403 00:52:13,590 --> 00:52:15,659 breaks. And nothing, not only on the 1404 00:52:15,660 --> 00:52:17,429 level of it still compiles, but on the 1405 00:52:17,430 --> 00:52:19,199 level of the robot still does the correct 1406 00:52:19,200 --> 00:52:21,929 thing. So, OK, here's the simulated 1407 00:52:21,930 --> 00:52:22,919 cloud again. 1408 00:52:22,920 --> 00:52:23,920 I have to skip at 1409 00:52:25,050 --> 00:52:26,789 debugging tools. I showed you the command 1410 00:52:26,790 --> 00:52:28,979 line tools. I showed you a tiny bit 1411 00:52:28,980 --> 00:52:31,089 of the plugins of the Arctic, 1412 00:52:31,090 --> 00:52:33,569 we and our three Bush 1413 00:52:33,570 --> 00:52:34,570 station. 1414 00:52:35,810 --> 00:52:38,329 Now we are at the outlook, so I talked 1415 00:52:38,330 --> 00:52:40,759 very, very briefly about 1416 00:52:40,760 --> 00:52:43,399 gazebo and move it simulation 1417 00:52:43,400 --> 00:52:45,769 and motion planning, there's a 1418 00:52:45,770 --> 00:52:47,869 whole new framework to do low level 1419 00:52:47,870 --> 00:52:50,449 control, for example, um, 1420 00:52:50,450 --> 00:52:52,759 control loops, PID 1421 00:52:52,760 --> 00:52:54,019 control and so on. 1422 00:52:54,020 --> 00:52:55,020 That is. 1423 00:52:55,890 --> 00:52:58,019 Already integrated Bifrost, So 1424 00:52:58,020 --> 00:53:00,179 if you use race control on your 1425 00:53:00,180 --> 00:53:02,279 microcontroller to get your motor to 1426 00:53:02,280 --> 00:53:03,719 stay in the correct position, if you're 1427 00:53:03,720 --> 00:53:06,389 Talita, it can automatically 1428 00:53:06,390 --> 00:53:08,819 interface with move it with the 1429 00:53:08,820 --> 00:53:10,319 visualizations and so on. 1430 00:53:10,320 --> 00:53:11,209 This is really nice. 1431 00:53:11,210 --> 00:53:13,589 So perhaps have a look at it 1432 00:53:13,590 --> 00:53:15,089 in the beginning and then start 1433 00:53:15,090 --> 00:53:16,599 implementing instead of the other way 1434 00:53:16,600 --> 00:53:17,600 around. 1435 00:53:18,530 --> 00:53:20,779 And of course, open cvn 1436 00:53:20,780 --> 00:53:22,969 the PC Alara to 1437 00:53:22,970 --> 00:53:24,760 talks for itself. 1438 00:53:26,150 --> 00:53:28,129 Then there's also the possibility of 1439 00:53:28,130 --> 00:53:30,499 using shared memory transparently within 1440 00:53:30,500 --> 00:53:32,839 the same house, instead of using 1441 00:53:32,840 --> 00:53:34,999 local loop back for 1442 00:53:35,000 --> 00:53:37,459 efficiency reasons, there's CPU 1443 00:53:37,460 --> 00:53:40,159 processing you can do slam simultaneous 1444 00:53:40,160 --> 00:53:42,289 localization mapping, which means you 1445 00:53:42,290 --> 00:53:44,749 put your robot here, it drives around 1446 00:53:44,750 --> 00:53:45,949 while it drives around. 1447 00:53:45,950 --> 00:53:48,379 It builds a map and localize itself 1448 00:53:48,380 --> 00:53:50,719 within the map it just built. 1449 00:53:50,720 --> 00:53:53,299 And this is a really huge 1450 00:53:53,300 --> 00:53:55,369 scientific research question for 1451 00:53:55,370 --> 00:53:56,809 the last 10 years. 1452 00:53:56,810 --> 00:53:59,149 And now we have open source software 1453 00:53:59,150 --> 00:54:01,399 that does this reliably for 1454 00:54:01,400 --> 00:54:02,959 indoor scenarios. 1455 00:54:02,960 --> 00:54:05,779 So basically you have your Roomba 1456 00:54:05,780 --> 00:54:07,729 stick, connect to it, configure the 1457 00:54:07,730 --> 00:54:09,919 navigation stack and you have slam, 1458 00:54:09,920 --> 00:54:12,289 which like many, many PhD students, 1459 00:54:12,290 --> 00:54:14,359 worked hard on 1460 00:54:14,360 --> 00:54:15,840 for the last decade. 1461 00:54:17,030 --> 00:54:19,339 OK, um, of course you can 1462 00:54:19,340 --> 00:54:21,619 integrate Beamer's for augmented 1463 00:54:21,620 --> 00:54:23,749 reality, like project something 1464 00:54:23,750 --> 00:54:25,069 onto the robot. 1465 00:54:25,070 --> 00:54:27,419 OK, now I'm really out of time. 1466 00:54:27,420 --> 00:54:29,509 Um, there's no Rob. 1467 00:54:29,510 --> 00:54:31,009 So if you want to do knowledge 1468 00:54:31,010 --> 00:54:33,199 processing, artificial intelligence 1469 00:54:33,200 --> 00:54:35,539 on ontologies, 1470 00:54:35,540 --> 00:54:37,639 um, all kinds of 1471 00:54:37,640 --> 00:54:39,729 world perception, world understanding, we 1472 00:54:39,730 --> 00:54:41,809 froster people from the tomb have 1473 00:54:41,810 --> 00:54:43,090 worked on that quite a while. 1474 00:54:44,150 --> 00:54:45,739 Have a look at these two pages. 1475 00:54:45,740 --> 00:54:48,050 What people are already using for, 1476 00:54:49,190 --> 00:54:51,409 um, ROS 2.0 is coming up next 1477 00:54:51,410 --> 00:54:51,529 year. 1478 00:54:51,530 --> 00:54:53,269 I hope this will be create. 1479 00:54:53,270 --> 00:54:55,429 The negative thing about it is 1480 00:54:55,430 --> 00:54:57,469 it will deprecate some things. 1481 00:54:57,470 --> 00:54:59,299 And as you know, if something gets 1482 00:54:59,300 --> 00:55:01,579 deprecated, the person who once 1483 00:55:01,580 --> 00:55:03,379 wrote it might not be there anymore to 1484 00:55:03,380 --> 00:55:04,309 fix it. 1485 00:55:04,310 --> 00:55:05,719 So there might be some breakage. 1486 00:55:05,720 --> 00:55:07,939 But they're basing Ross on a much 1487 00:55:07,940 --> 00:55:09,829 more solid middleware. 1488 00:55:09,830 --> 00:55:11,959 So it feels that everything will stay the 1489 00:55:11,960 --> 00:55:14,179 same from an AP point 1490 00:55:14,180 --> 00:55:16,549 of view, from a user point of view, 1491 00:55:16,550 --> 00:55:18,619 but as the lowest level, 1492 00:55:18,620 --> 00:55:20,659 they will be using Dede's data 1493 00:55:20,660 --> 00:55:22,819 distribution services that you can add 1494 00:55:22,820 --> 00:55:25,249 real time guarantees, channel resources 1495 00:55:25,250 --> 00:55:28,459 and all other kinds of things and 1496 00:55:28,460 --> 00:55:30,170 transparency between. 1497 00:55:31,690 --> 00:55:33,879 Network based communication of nodes 1498 00:55:33,880 --> 00:55:35,949 and shared memory based communication 1499 00:55:35,950 --> 00:55:38,229 of nodes will be completely 1500 00:55:38,230 --> 00:55:40,449 so, you don't know whether 1501 00:55:40,450 --> 00:55:42,609 you run your nodes on the same 1502 00:55:42,610 --> 00:55:43,959 house values shared memory. 1503 00:55:43,960 --> 00:55:46,089 If you run it on multiple hosts, it will 1504 00:55:46,090 --> 00:55:48,369 do network serialization. 1505 00:55:48,370 --> 00:55:50,589 So this will be great if some 1506 00:55:50,590 --> 00:55:52,739 of you will start working 1507 00:55:52,740 --> 00:55:54,759 with us and help make it great. 1508 00:55:54,760 --> 00:55:56,349 That would be even better. 1509 00:55:56,350 --> 00:55:58,479 So I thank you very much for your 1510 00:55:58,480 --> 00:55:59,799 attention. 1511 00:55:59,800 --> 00:56:01,869 I hope someone will 1512 00:56:01,870 --> 00:56:03,789 read the line at the very bottom. 1513 00:56:03,790 --> 00:56:05,769 If you use drugs for any of your hobby 1514 00:56:05,770 --> 00:56:08,169 projects or in your hackerspace 1515 00:56:08,170 --> 00:56:10,089 plan to use it now or just get 1516 00:56:10,090 --> 00:56:11,979 interested, find me at the conference, 1517 00:56:11,980 --> 00:56:12,939 talk to me. 1518 00:56:12,940 --> 00:56:15,069 And really interesting to hear what 1519 00:56:15,070 --> 00:56:17,369 other people are using for. 1520 00:56:17,370 --> 00:56:19,119 Just sent me an email if you don't catch 1521 00:56:19,120 --> 00:56:20,120 me. Thank you. 1522 00:56:33,540 --> 00:56:34,739 OK, thank you, Andrea. 1523 00:56:34,740 --> 00:56:36,839 So far, we still have maybe one or 1524 00:56:36,840 --> 00:56:39,149 two minutes left for a few quick 1525 00:56:39,150 --> 00:56:41,609 questions if anyone has a question 1526 00:56:41,610 --> 00:56:43,739 and just come to one 1527 00:56:43,740 --> 00:56:46,019 of the seismic microphones. 1528 00:56:56,690 --> 00:56:58,729 OK, so that doesn't seem to be the case 1529 00:56:58,730 --> 00:57:01,699 then. Thank you so much, Andreas. 1530 00:57:01,700 --> 00:57:03,500 Thank you for listening and attending.