Dear viewer, these subtitles were generated by a machine via the service Trint and therefore are (very) buggy. If you are capable, please help us to create good quality subtitles: https://c3subtitles.de/talk/344 Thanks!
Thank you for letting me have
this talk today.
I give you 30 more seconds to read
the comic because it's essential
to what I'm going to talk about today.
So too
small.
OK, so in short,
download the comic sometime later on.
It's funny.
The problem with robotics, either
in academia or also in hacker
spaces or in your private lab
or in your basement or whatever you
want to do. Robotics is that
too many people start to reinvent what
is already there.
Of course, there are many reasons for
that. I wrote them down in the
talk summary, the AARP's
blog that it's deprecated, not using C++
14 features and so on.
But still, I want to make a point
that if you want to go into robotic
projects in your hackerspace, have
a look at Ross, which is well established
middleware for robotics, because it
will give you many things for free that
you would otherwise have to develop
yourself so that you can more easily
get to the point where you actually want
to work on.
So short overview.
I'm going to give a technical
introduction into Ross.
I'm going to show some examples of
technologies, packages that are already
there and can be used out of the box in
most cases that do not only work
and this is the where I
differ from most introduction to Ross
talks can be also used with a webcam
and a low budget.
So if you have, like, I don't know, one
hundred, two hundred, three hundred euros
to build a robot, you can still use Ross
to do it. You don't have to start with an
ATM or Arduino and
programing to register again.
And the last part will be about some
tools that you might
be missing from your custom implemented
middleware that are really great to have.
And you can have them if
you use.
So first some text
methods, then some images and funny
videos and at the end some hyperlinks.
So if you do robotics,
what are you actually doing?
Robots mainly consist of three parts.
So you have some mechanical parts, some
electrical parts and software.
Today, I'm only going to talk about
software. There's also many interesting
projects how you can build modular
reusable hardware, not only software, but
today it's about software.
So what kind of software is in a robot?
Of course, there's drivers, cameras,
motors,
whatever else there is.
You want to connect to your robot LEDs
and so on.
Then there's some core robot
functionality you want to be able
to do path planning, motion planning.
You want to do some kind of image
processing, 3D reconstruction
from stereo cameras and so on and so
forth. This is what I call core
functionality here.
Then usually your software does
not run the way you wanted
in the first place.
So you need some debugging facilities.
Also introspection.
Your robot is doing something you did not
expect. Instead of just hitting the
emergency stop button, please always
add one to your robots.
You can have a look what is currently
happening with the software.
If you did not build a large binary
blob that you are flashing on to your
robot, then you need
all kinds of algorithms.
And here is where most academics
work on like myself.
You want to do, I don't know, motion
compensation in cognitive
surgery with robots or whatever.
In any case, you need some very basic
algorithms. Otherwise the robot will
only, like, move around rather
randomly. So if I want to
make my robot grasp this microphone,
I need to do motion planning with
my many degrees of freedom arm.
I need to do grass planning.
I need to do collision avoidance and all
of these things that are really hard
problems. People are working on them
since like thirty or forty years.
They are not completely solved yet, but
they are very good open source
implementations that solve it for
the usual cases.
And the usual cases fortunately are the
ones one usually encounters.
That's why they are the usual ones.
And then once your software
gets large enough, you will not
be able to run it on a single machine
anymore. Made a single machine be some
embedded device, like a beagle bone or,
I don't know, a Sytek box or what
all other arm boards there are around
these days, not even an A7.
You usually want to be able,
if the computation demands it, to
distribute your robot software
to multiple computers that are connected
via Internet, for example,
and all us.
All of this is already
built into the core of us.
So everything I'm going to show today
can be run on my single laptop.
Most things can be even run on something
like the new Raspberry Pi,
but it can.
Also be started on a cluster
of computers, so if I have like five
desktop machines available, I can
put the camera processing on one machine,
the path planning on the other, and
it will work together the same way as
it would be running locally.
And that is completely networked,
transparent and happens during runtime.
Then the this is also
if you have multiple machines,
you know that it's not as easy as
compile, run it because you have to
distribute it to the
various machines.
You have to start the components on
various machines.
So deployment and orchestration is
also something you need to take care
of once you have more complex robotic
software.
And of course, it's taken more or less
care of already.
So what do you want to work on?
Usually just you want to build a cool
application, I don't know, have a robot
that gets the pieces out of your freshly
built 3D printer so that it can continue
printing instead of you having to go
there, take the part and click
and click onto print next one.
So but where you usually end up working
on is the very low level software.
Get the motor to turn,
get the motor to position itself
and to a certain degree, get
your camera running, get your camera
data into open TV, back out
of open TV, create point clouds,
use the PCL and whatever.
So you are doing a whole lot of pumping
work when when you are working
with robots.
And this does not have to be the case.
That is what I want to show now.
So what is Ross?
Ross stands for Robot Operating System.
It consists of four major
parts. One is the middleware, together
with tools.
Never underestimate the importance of
having tools available for whatever you
are using.
Then the basic robotics software
distributed processing
of Cordona transformations.
So, for example, if you have your
robot and you want to tell it, move
five centimeters to the left, you
usually specify that within some kind of
coordinate frame.
So either it is the coordinate frame of
the robot, it is the coordinate frame of
my laptop and so on.
And you in the background, you
do just some matrix multiplications
to get everything right.
Perhaps you remember your algebra classes
or not. It does not matter because
here it's just an optical.
Give me this position in this coordinate
frame and it will do all the magic,
algebra, algebra in the background.
Um, then the cert.
And I think most important part about
Ross Ross is not better than any other
middleware that was built and
there were hundreds of it in the last
thirty years, but it gained
such a big momentum that a large
ecosystem formed around it, kind
of similar to Linux, perhaps
compared to the BSD.
Now, don't don't flatter me
for that. But the important point
is you can take Ross today.
Install it, which is easy to
get on a Bantus and then
have a large collection of software
available that I will be showing
shortly, but it's already there.
And these are the three main
components that make Ross a great tool
to use for robotics.
So now we're going to
describe some of the technical
terminology used in.
So first there's the ROSTKER, which
is the only centralized
part of Ross. So it's a well known
instance, like your name server,
for example, and it can point
you to the other instances within your
robot network.
Notes are any
process within your
robotic network that implements
the client up, which is
usually used through raw CPR
OSPI, which are the main programing
languages used to interface Bifrost
Systems. There are also many
other interfaces, even for Motlop these
days, even officially from MAFF
works for Motlop these days.
So it really got important, obviously,
and did support all the nice features
I'm going to show till the
end of my talk.
So initially you start out with
an empty computational crap,
so basically we are talking
about notes and entries
that connect these notes.
So the note is some kind of process
on an operating system running on any
machines that are connected to your
network.
And then there's the central core running
on one of these machines.
So initially, there are no connections.
We have just started the Rostker.
Then we had a node A and A
wants to publish some information
under the name subtopic,
either later on or previously.
Timing does not matter here.
Nobody comes along and wants to
subscribe to this same topic.
And so it also tells the Rostker,
please tell me who is
publishing information under this
name. So it's a classical publisher,
subscriber architecture for those people
who read Tennenbaum,
the two currencies with the IP address
and part of this core.
So it's all based on TCP IP.
There's also UDP implementations,
but the standard is TCP IP.
After nobody knows about how
to reach Node eight directly, it
will contact him.
This is all our.
And from there on, not
able to transmit only the
bulk data to be whenever
there's something new that it wants
to tell its subscribers
about. So as you can see, the ROSTKER
only gets involved into setting
up the connections, but it's not the
central bottleneck in the network.
So not all of the information is pushed
through the ROSTKER, which is quite
fortunate because if you have a couple of
Kinect cameras, for example, on your
robot, each one saturates your
gigabit link.
So you really do not want to
add artificial bottlenecks
to.
And the nice thing about this
architecture is that on the same
topic.
Any number of publishers can add
information to this topic and any
number of subscribers will be
informed once there's new information
available on
a technical level.
This is done at the moment
with unicast.
So you have multiple copies
on your network.
But in May of next year,
it will also be able or
at least that's the plan, to use IPv6
multicast. So then it's even efficient
what you are doing here.
So topics are already introduced.
Those were the names under which
information is published.
So it's just some way to
define where to find information.
And it's built up in a hierarchical
structure you'll see shortly.
So you can kind of imagine it
like your your else
in your file system, your file,
whatever funny cat video that ABC
is located in some directory and you
can refer to it under that name.
So these are the names and roles,
but they are not locally but
used within this network through
the Rostker.
A topic is a unidirectional connection,
so you publish information and
people subscribe to it, so it's not
bidirectional.
So you do not get feedback on who
subscribe to it and you don't care about
you just publish it.
And if there is at least someone who's
interested in it, this note
will get the information.
There's another kind of
ipsi interspecies communication mechanism
that are called services that are your
classically remote procedure calls to
your. You call this name
has some parameters to it.
It will be serialized through the
network.
The destination node will do some kind of
computation and
sort of reply
and afterwards this kind of
communications end.
So it's not intended
for streaming data.
It's more for like, please,
please reinitialize yourself or
please do this calculation for me or get
me your current set of parameters.
And the good thing about these
communications mechanisms is that they're
kind of complementary.
So you have your topics for streaming
data sensors.
Think of a camera or your robot,
which publishes its current position
of all of its angles and joins.
The problem of it is that messages may
be dropped due to overload without
notice. So but if
you think about it, if you have a camera
publishing information, it's 33
milliseconds, like it's 30 frames
per second. You really want the frame
to be dropped instead of a Q building up.
If your Wi-Fi is too slow and you
go further and further back into the,
um, into the past and
get a large latency.
So that is why it was built
with lots in mind.
Services, on the other hand, are
blocking calls.
So you call the service, get blocked
within the current node will be
transferred to the distant node
until the reply is received.
You are blocked, which is nice to do some
kind of tasks.
And then there are also actions which
built and on the first two mechanisms
and they are allow you to
deal with long term
tasks. For example, if you tell your
robot move from here to here,
you really do not want to have that as a
function call like in your programing
language, because it will take time
until the robot goes from some point
to another point.
And you might want to get feedback in
between as well.
So you have actions, you tell your
robot, go there and then you
are unplugged when this request
was received by the
other node and then you can get
intermediate feedback and at the end you
get the result. For example, OK, I
reached a position or I collided
with your cat or whatever,
and all these kinds of mechanisms
are used in what I'm going to show for
it on.
The last terminology I
have to introduce our parameters, so the
Rostker also act as parameter so
you can kind of at it's
a bit similar to rest up piece, you can
just add names there with some
data attached to it and also with
various kinds of data.
So not only strings, but also arrays,
whatever you can serialize in YAML.
And this is usually used to do
distributed parameterization
of your notes.
Imagine you have the classical solution
of configuration files and you want to
run your note
today on a different machine than
you would have to make sure that your
configuration file on this machine is
up to date with the parameter server.
Whoever in the network starts this node
pushes the current set of parameters to
the server, and all nodes, no matter on
which machine they're running, uses these
parameters to set up their
their algorithms, for example.
OK, I've talked about
names for everyone who
ever used your L's or
perhaps Unix OS, it will be quite
familiar that you have absolute paths
that begin with a slash and
you have relative paths that are not
anchored kind of at the root of your
system. And this is quite nice
because you can at runtime
tell the note, OK, don't publish
your topics at the
kind of the root of your namespace, but
push it one space
down like kind of moving something
to a subfolder.
If you have relative addresses,
everything will continue to work within
that sub fold, even if you push it down.
So, for example, if you have two webcams,
you simply start your driver
node twice in separate
namespace is, for example, camera left
and camera right.
And then software does not need
to know about that.
It currently is being used together with
another camera.
That is just the runtime decision when
you start your nodes.
So a small example
here, we have three notes within our
network and our Ruska, for example, after
you see Cammarano publishing images
from some webcam, we have
some processing note and does, for
example, an air filter
filtering with the images.
And then we have our Arcudi GUI,
which displays these images
for the user to look at what the robot is
currently perceiving.
So we have here our parameters.
We have our nodes.
We have our services that can be used
as our PCs and we have our topics
where streaming data is moving.
Now, we have all kinds of tools for
all of these things I talked about,
there's the command line tool that allows
you to have a look at what is
currently running into your network.
For example, if you want to find out all
the notes that you have started on your
various machines, you just type in
your command line node list and you will
get the list and you can get additional
information on each node.
The same for topic services,
messages for the different data types
that are used
for the parameters of.
And also there are CREF, there are
three the or
GUI and 3D tools you
can use to fuel the current state of
your robot. You'll see them short.
But so just here
to glance over these are two
code snippets, one for Python and one for
C++ to publish something
on the network.
So it's quite simple.
You initialize your node, you create
the publisher, you fill your
data type with information, and then
you call publish and you don't
have to do anything else.
Then it will be already
available. If you type topic list,
you can already see the topic
as a new topic available.
And if you go to the next slide,
you can write your subscriber with a few
lines.
You create the subscriber, tell it
what is the name of the topic it should
subscribe to what data is being sent
on it, um, provided for callbacks.
So this is event driven.
And afterwards this callback will be
called whenever some node on
the network publishes something
under the name a topic.
And that's about it.
You have to know to use most
parts of the middleware
of Ross.
OK, yes, Ross has
its own built system, as perhaps all
large software
frameworks have come up with.
The good thing is it's only a very thin
wrap around.
If you have ever worked with Semak, you
will be fine. Otherwise, learn
c make.
But don't worry, this is really no
game changer.
It's just you have
to look at what is written on this slide,
this slide and this slide.
And then you are in C McWorld again and
it will take care of resolving the
dependencies between the
packages you are using
that they are get built in the right
order. So it actually helps to work with
it. Also, it's of course annoying
as but I've never seen a built system
that is not annoying, so no change here.
OK, so now I've talked about what
is in technical terms.
So what do you need to use
or do you need always nice seven PC
purpose. You don't have one on your
quadcopter or on the International Space
Station, both places where Ross is
currently running?
No, you simply need anything that can
run Debian,
Linux or preferably Ubuntu.
I'm saying Bunta 14 for
here because that's the easiest
way there. Just type SUDEP, Cardross,
whatever, and it's installed.
You can of course, build it from source
on all other platforms, but I don't
think you want to. It has a couple
thousand dependencies.
So what kind of machine do you need?
Preferably some powerful desktop
or laptop machine so you can get started.
A good solution if you are power
constrained is to have a small machine on
board of the robot, like a beagle bone
or a raspberry or knock
knock and a faster machine
of the robot that does the heavy
processing. But you can always, as I
said, use it locally and you don't get a
large performance penalty.
But it depends on
what application you're running.
Of course, if you have like four connects
that stream data and you want to do
Pontarlier processing on them, you can
put some desktop machine
into your rack for each Kinect because
that is just computationally intensive.
If you just want to have a mobile robot
like a Roomba moving around, it's
probably enough to strap a beagle bone to
it.
OK, now let's come to the part
with more images and perhaps
more interesting if you are not yet sure
you want to use us.
So far, all the usual
kind of camera sensors
you find on a robot like Molnau cameras,
for example, a single webcam, stereo
cameras, for example, to webcams
and Archibong cameras, for example,
a Kinect or an Atlas
action or whatever you want
to spell it.
Um, you need drivers for all of
this. You need to calibrate them
intrinsically and extrinsically.
You want to visualize what's currently
happening and you need processing
so that your raw data
do something useful for your application.
And now I want to go through all of
these steps for all three camera types
very quickly.
I'm not going to talk about object
recognition here.
There are some rather specific frameworks
to facilitate object recognition, but
basically they come down to
using open CSV and upon
cloud library.
But still, it might be easier if you want
to do object recognition to look
what is already there in
what packages have been built by others
instead of rolling your own from the
onset.
So that's the final launch file to
start my webcam
launch files are just a convenient way
for deployment and orchestration.
It's just an extreme Elfa, starting with
the highest level tech launch.
Then you can add any number of note
techs where you simply tell.
Ross, from which package, what
to load, what process
or what binary, you can give it some
name and define some parameters.
For example, here I said namespace
cams so everything will be pushed
down into the subfolder camera
so that I don't have a whole mess
of topics at the top level namespace.
And then you add some parameters for
specific, of course, to your node
resolution of the camera frame rate,
disable autofocus and so on.
And that's about it.
Um, then you tie launch,
I prepared a screen capture videos
for this because life demos, as
we saw in the previous hour, is too
dangerous.
Um, so let's have a look at this.
I hope it works.
So I started my Rostker, this is all now
closed, I'll start the launch for you
just saw then I run the
Arctic we know I trust.
OK, first I want to have a look on
the command line. I am perhaps I don't
like this. So let's have a look at
the command line, what notes I have
started, what topics they created.
That's all good and all
nice because you can also get more
information about the topic number of
publishers, what type the topics
and so on. But I just started
a cammarano, so I might want to see the
camera images.
That is where I start the Arctic.
We here I select
as a plugin visualization
image.
And then you can have a drop down
menu, which gives you all the choices
of topics that Kerry sends some messages
image as their type.
So these are normal, two dimensional
images in rows.
And I can just select that and
have a live feed of my camera data.
And this is.
During runtime, so I did not have to
recompile my camera, drive a note
adding Dipak output, for example,
open KVI image.
So I just can at runtime
start my GUI and it will
connect as a subscriber to my
data stream and also get a copy
of the information all other nodes keep
on running.
And this would have been working
exactly the same way, whether I started
the camera locally, the Arctic,
we locally or one of those remote
and the Rostker on the third computer,
don't ask about security.
Ross was not built with security in mind.
So the first
thing you should do when you start to use
Ross is do pseudo
IP tables menas flush
because otherwise you will have a very
weird connection, problems
within your network.
So you might not connected to the
Internet during that time.
OK, now we have a camera
who has ever worked with image processing
here in the room, first raise your hand
if you've ever done some kind of image
processing.
OK, so then a short
reminder, if you have a camera,
you need to have
a mapping between the pixels and
the outside world.
So if you just have the image, you do not
really know a relation between
pixel values and
see units, for example, you do not know
how large this object
five meters away in pixels
if you do not have an intrinsic
calibration of the camera.
So intrinsic calibration gives
you a mapping between the
outside world in terms of meters
to what you get from
the camera in terms of pixels
in a very simplified manner.
Um, there's a utility for that.
It's using open sea.
But if you have ever wrote your own
open CV calibration for
camera, um, software,
you know that there are many weird
Koronakis and so on.
And it's really nice to simply
have a GUI for that, um
and show the video and.
So basically, we have everything running
from the previous screen capture.
Now, additionally, start this
calibration note, which
will give me a GUI.
And now if I update on the left
and the plotting of my
Rosebraugh, which is also very nice
introspection utility that comes for
Ross, you can ask
Ross to plot itself.
So to tell you which nodes
are running on which machine, who is
connected to whom, what amount of
data is being transferred in between and
so on. So now that I started this
calibration node and I update this
graph, it appeared here as
a node subscribing to the image,
of course, of the camera and outputting
to the out.
OK, then you do the usual thing, run
around in front of your camera with a
checkerboard, don't have to watch that.
Um, and then in the
end, you get this screen
where you are told
whether you have enough, um, posters
of your checkerboard captured, which
opens if it does not tell you by default,
odds are again a nice tool to
have available instead of, um,
writing your own.
And then finally, if everything is
screen you hit calibrate, it will
do the intrinsic calibration
using open KVI and you get the camera
metrics, the distortion coefficients,
the protection metrics and so on.
So for those of you who know what it is,
fine. For those that don't, uh, doesn't
matter. You can have your camera
intrinsically calibrated and you can use
that down stream.
And it will also automatically start
a topic, um, that's called,
um, camera info.
And there you can get
the parameters of your camera.
And this is used in all different parts
of us, wherever you need
the mapping of cameras to,
um, external values.
What is also written down here on the
command line with the column
Equal Sign is runtime
time name, runtime
name remapping.
So the camera calibrator PI, if
you look into the source code, it
subscribes to the topic named
image.
And now I would have to go about
and change this string in
the code in order to change to camera
in, because that is what in my
network on my robot, the image
is called or what is the topic
under which the image stream is
published.
But the nice thing about Ross is that you
can do this remapping during startup
or even during runtime where just
held it's not OK, whatever is called.
The name image in the road network
for you now is camera image SRAW
and everything will start
working. So that's really nice way
of flexibility in your software
and also how you can write quite modular
meaning, loosely coupled notes,
because they don't have to know about
each other up front.
They just talk to each other using
standardized formats and the naming
the topics on which they talk, you can
specify at orchestration time.
OK, then, once you have your
camera calibrated, you might want to do
some post-processing, for example, if
you have an industrial camera.
Usually it does not
output color images, rather
does output grayscale by your images
and you have to reverse that process and
so on. And for all those low level
processing steps, there's there are notes
available. You just start them and they
take the raw data as input and
output the process data and you
can connect your higher level algorithms
with these data streams.
If we want to write our own
processing note
on Ross, so we want to
take in an image as input
data, we want to apply some open copy
filter in this case and each filter,
and we want to output this data to
our US network, perhaps to
be used as input by a higher
level processing node.
Furthermore, we want to be able during
runtime without restarting anything
to change the parameters of this
filter. We are running on the input
images and this is called dynamic
reconfiguring. And I'll show you
a video.
How all of this works together.
So I start to note, I programed
if I update a glamorous graph, it will
show up as a new note subscribing
to the image topic.
And now I have available a
new topic with a filter applied
to it, and then
I can start in Arctic with the
dynamic reconfigure plugin
where I get all these parameters
I defined as being
dynamically reconfigurable in my
algorithm.
And I can play with
them during runtime and this is all
something out of the box available.
You don't have to sit down and write a
library that can do something like that
for you again.
And this is really nice to get
to quickly get to
something that does
something interesting from a point of
view of the application.
And all of this takes just 36
lines of C++ code, I think this is quite.
Few code in order to have something
that's networked, transparent, that can
be dynamically reconfigured
remapped at runtime and so on, so
it might be better than linking libraries
to each other to change something.
OK, so we had mono
Skype cameras.
Now let's do stereo, if you like, we
humans, we have two eyes so we can
process the disparity.
For example, I see this microphone more
to the right of my right eye than to my
left eye. And from that, I can
triangulate the three dimensional
position of this microphone.
And also you can do that with robots
and you usually want to perceive the
world in three dimensions in order to
figure out where can the robot move along
to two object detection in three
dimensions and so on.
So again, we write a launch file.
The note this time is called
UFOs. You know, those are all
available.
We again, at some parameters to it
this time, we have two devices left
camera and the right camera
define which webcam is attached to which
Linux video device, and
then we can start that.
We have this note publishes two topics.
Under those two names, we can again start
ArchCity, go and have a look at them.
If we want to do serious calibration,
which not only does the intrinsic
calibration, but also correlates what
the right camera sees to what the left
camera perceives at the same moment,
and also the synchronization
and some other algorithmic
transformations of the images.
And there's also this calibrate
the note for it pretty much looks the
same way as previously for the mono
camera, only with two images.
OK, once we have our stereo
camera calibrated, we
can use the stereo image
to reconstruct a three dimensional
image from our two cameras and all
of that this takes is a launch
of about five lines of XML
in addition to what we wrote down to
start our cameras.
And this note also
has available dynamic reconfigure GUI
in order to change the exterior
reconstruction parameters.
So if you get into Steria reconstruction,
there are a whole lot of parameters you
have to set.
You can do that during runtime and don't
have to write a single line of code
for it.
And then the result, our point clouds
and point clouds are really nice format
because there's a library available
which does what open S.V.
did for image processing four point
clouds.
So a really large
close to state of the art library
with all kinds of algorithms that
allow you to get
with only a few lines of code
to something that's applicable
to what you actually want to do instead
of writing low level.
Take this point, compare it with the one
to the right and do something.
OK, then the last kind of
cameras are the Kinect
cameras, unfortunately, open
the night was bought by Apple and
they closed it down, which is a bit
unfortunate because those devices
replaced in academia research
for, I think one hundred fifty euros,
replaced devices that cost about
20000 euros and they're good
enough compared to laser range
finders and so on.
And to start this device
heavy debate, I have to point Claude
available.
You just type in this line at the
command prompt. You start
launch for someone else has written for
you and then you got a point
cloud you can process in your network.
So let's have a look at this.
So this is the Archabbey image from
this year, and this is the
tough image, which is all
quite nice, but what
you really want to have, if you get a
point cloud out of something, have a look
at the cloud and that interactively.
And this is what you can do with our.
So it's just not obvious at
this point. Cloud, plug in
to it, tell it on which topic to listen
how to do the color transform.
And then you have three dimensional
clouds, introspective, built
by the human developer who needs to debug
what's going wrong at the moment.
And writing something like this viewer by
hand, I can tell you, takes
you more time than you planned to
spend on your whole hobby robotics.
Unload my printer project,
so perhaps have a
few minutes time to trying
to do this life.
So let's start our Oscar, let's
launch.
I'm not sure if it will be working
because my laptop has not really
enough power for this, but we'll see.
So now at this plugin, that specifically
for porn clouds.
And of course, it's
perhaps not working.
I won't spend too much time on
it.
Let me have one look at the driver's side
of things.
OK, I forgot to check register def
images.
The purpose I should look at something
that's not out of range of the camera.
There we go. So now we have a three
dimensional view here of our war.
OK, and this was without writing
a single line of code and
also you can do all the other nifty
things I talked about to it.
OK. That was the Ronke.
Yep, obviously, PowerPoint
and connecting Beamer's to laptops is
more complicated in robotics these days.
OK, um, where
did we leave off?
I think that was here.
OK, and the nice thing about it
is if you have a stereo camera part and
the Kinect on your robot, which is a
quite common setup, they output
the same kind of data
format. So you can switch in between
between your Kinect style device
and your stereo camera pair.
And this is highly valuable because
Kinect are much better
at reconstructing what is in the range
of like half a meter to four meters,
even in the dark stereo
cameras. Obviously, you need light like
ourselves to see
anything. But on the other hand, there
are much better at doing long distance
reconstruction or if there
would be some shining
into this room. OK, I know this doesn't
happen that hacker spaces, but
then Kinect style cameras do not really
work well.
So if you have
your middleware set up in a way
that it does, no matter where you get the
input from, you can trust at runtime
switch between the two devices
that feed you the data and all your high
level algorithms.
For example, collision avoidance
will continue to work the way they did.
OK, now I was talking a whole
lot about sensors, so cameras are
obviously only one example of sensors
you might find on your robot.
They are drivers for laser
scanners, for laser line scanners, for
distance sensors, for what?
I do not know at all
what there is these days.
There's a listing on the Ross
wiki page.
The link is at the end of the slides.
Have a look. Perhaps your device is
already supported.
Otherwise, ask me reimplemented.
Quite some drivers, for example, for the,
uh, the Archibold elitist,
uh, w.
S a twelve moderate
to sixteen or whatever those leads
you find a lot of in the basement.
We have a Ross driver for that.
So just connect them to a beagle bone and
you can have a display
with us.
So it's also Ross does not only
work nice for robots, of course it
is targeted towards robots, but
whether you are looking for a flexible
middleware that does not need hard,
real time constraints and if you don't
know what hard time constraints are,
you might not need them.
So if if if best effort
is good enough, so if you can just check
whether your data is being processed fast
enough and if it is, it's fine, then
Ross is quite nice for a whole lot
of Hekker and
make our project OK.
But now let's get into robots.
We want to model our robot for this.
There's some Elbaz description
format called You are D.F..
You start out with the top level tech
robot, give it a name, then you add
links. Links are the rigid
objects that Connect joins together.
Like, for example, if I want to model my
arm, perhaps I can model letters to links
and to Chines in between the shoulder
and the elbow, and then
you have to give them a visual
appearance. This can be either geometric
primitives or meshes and meshes are
nice because you can create them of
Blendr, you can export them from your
cat application and you can
just tell your use mesh
for this link and it's there
and there it is where you can easily
either model your robot before
you built it, which is quite nice
because then you can explore it's
kinematics before you find out they're
wrong.
Or you could just model your robot to
do all the things I'm going to show in
the next or ten minutes.
Um, the links
are connected by train stay.
They are different types.
So if you have electrical motor, for
example, a cerebral motor, this is
a type revolution and it has
joined limits. You cannot move it
endlessly in one direction.
And you specify all of these kinds
in here.
OK, once you're done with
that, you might want to have a look at
it. And again, the nice thing about is
other people already built
software so that you can focus
on what you're interested in.
So I wrote down that's about
as in one hundred lines, long
description, and now here's my very
simple robot consisting of three links,
a bass cylinder and
two arms segments connected
by two joints.
And I can immediately move them around
without writing any code through
this GUI.
And this way you can also explore
how you want to build your real robot.
For example, if you go the usual hobbyist
way and you servo motors and
aluminum profiles to build it.
OK, now comes something really
cool, if you have a robot that
has a couple of joints, it gets
really complicated to define
in which position, which point has
to move in order for the robot to get to
a certain point.
This is the problem of inverse
kinematics.
And usually you do that by
calculating something called the inverse
Jacobean metrics.
That already sounds like a lot of math
math. It's much more than that.
And.
You might not want to get too deep into
this if you just want to unload your 3D
printer, right?
So there's something
since I think the beginning of this
year called the move, it set up a system
and you just fit your description into
that.
It creates some nice GWI
where you can load your robot
again, it's immediately visualized
writing such a GUI again.
What has taken some people?
I am very grateful for that.
They did it a lot of time to do that.
Then you do all kinds of
configuration, but it only takes you
about two to three minutes.
Can more or less use the default
parameters if you have a robot
with multiple arms to define which
kinematic change our planning groups and
so on.
And in the end, you
start all of this.
This is very well described.
So this talk obviously is not
a tutorial.
You can just look at the website.
It's really easy to figure out, OK,
afterwards you have your
kinematic description of your robot
and you can use
this. They provide a demo
launch file for each robot you
have exported from the GUI.
I just showed.
And with this, you can do something
amazing out of the box.
You start to see more lunchpail.
It starts our August for you, there's a
robot and you can
do immediately do motion
planning at various state
of the art sampling based motion
planners. So, for example, I just told
it, go to some valid position and now it
plan the path to go there.
Of course, this seems trivial with such
a robot, but if you have a robot with six
degrees of freedom or even seven degrees
of freedom, you really
can't do that in your head.
It's very, very counterintuitive
what these drones are doing in order for
the robot to simply move along a straight
line in our Cartesian three dimensional
space until you get
it out of the box. And the nice thing is
this is not just the dummy robot, but
if you would connect your servo motors
to a controller through some low level
set up so that you can specify
what number do I have to send where,
so that your servo moves into this
position, you can immediately use
this GUI to move your real robot to do
path planning. If you read a robot and
you can even put a Kinect next
to your robot and do collision
free path planning using
the movie framework.
So this is really, Chris,
great that this is available nowadays.
That took, um,
I know of many projects in academia
that took like a year just to get
this basic motion planning running
stable and reliably with their robots.
And now you get it like in 10
to 20 minutes, which is
then a whole day if you consider
debugging.
But that's really, really a large
improvement.
And if you hate Cui's, that's fine.
All everything I'm showing
is well separated between the GUI
components and the code.
So if you want to do path planning, not
through a GUI, but from your code
because you recognize an object in your
image, of course there's a
python and a C++ app to tell
the robot, OK, go to this point.
You don't need to go through the GUI.
OK, if you want to go even further and
want to simulate your real robot
or your robot, you might want to
build someday a gazebo for it
physically based simulator that's
integrated quite tightly with these
days.
And there are a couple of details
I leave out due to time.
There are different formats, but you can
convert them in the end.
And since
Ross is built the way I thought
told you about in the beginning of the
topics, the dynamic renaming and
some mechanisms for us
to not use the real
progress of time like what we call
wall time, usually in simulation work,
but your simulated time.
So if your computer, for example,
is too slow to run the simulation in real
time, so it is only it
can progress one with 30
percent of the speed, the real
world time progresses, all
of your notes will automatically adapt
to that. So if you say sleep ten
milliseconds through the rosapepe,
it will sleep ten seconds
or ten simulated seconds
all by itself.
You don't have to recompile anything to
run your notes against simulation or
on your robot.
And of course, you can do all kinds of
software engineering, test driven
development, continuous integration,
there are people who integrated this with
the conserva and so on.
You can do distributed development in
your hackerspace because perhaps not
everyone has the same robot available.
You can still develop
algorithms, test them on the simulator,
and then once you're back in the
hackerspace tested on a real one,
stay close to the button in that moment.
OK, let's assume we attached
a webcam and an Nazi video camera to
the robot we modeled earlier
and converted the format
into each other.
Now, I started the gazebo simulator
at this robot here.
And now there is the physics engine, the
background, so you can throw objects
around and if they hit each other, it
will happen more or less physically
plausible. So they're using
different game physics engines like
old and bullet in the background.
And so I could flip over this book
bookshelf if I hit it hard enough in the
simulation so you can really do some
kind of advanced manipulation.
And as I said, we just added a simulated
camera and you can get images
from the simulated camera in the same
manner you get it from the real camera,
which gives you opportunities to do what
I started to call a robot unit
testing. So you can use your.
Software one to one, to use
it in your real robot, run it against a
simulated environment Wolfen,
run it against the simulated robot with
simulated environment, and thereby ensure
that if you change the code, nothing
breaks. And nothing, not only on the
level of it still compiles, but on the
level of the robot still does the correct
thing. So, OK, here's the simulated
cloud again.
I have to skip at
debugging tools. I showed you the command
line tools. I showed you a tiny bit
of the plugins of the Arctic,
we and our three Bush
station.
Now we are at the outlook, so I talked
very, very briefly about
gazebo and move it simulation
and motion planning, there's a
whole new framework to do low level
control, for example, um,
control loops, PID
control and so on.
That is.
Already integrated Bifrost, So
if you use race control on your
microcontroller to get your motor to
stay in the correct position, if you're
Talita, it can automatically
interface with move it with the
visualizations and so on.
This is really nice.
So perhaps have a look at it
in the beginning and then start
implementing instead of the other way
around.
And of course, open cvn
the PC Alara to
talks for itself.
Then there's also the possibility of
using shared memory transparently within
the same house, instead of using
local loop back for
efficiency reasons, there's CPU
processing you can do slam simultaneous
localization mapping, which means you
put your robot here, it drives around
while it drives around.
It builds a map and localize itself
within the map it just built.
And this is a really huge
scientific research question for
the last 10 years.
And now we have open source software
that does this reliably for
indoor scenarios.
So basically you have your Roomba
stick, connect to it, configure the
navigation stack and you have slam,
which like many, many PhD students,
worked hard on
for the last decade.
OK, um, of course you can
integrate Beamer's for augmented
reality, like project something
onto the robot.
OK, now I'm really out of time.
Um, there's no Rob.
So if you want to do knowledge
processing, artificial intelligence
on ontologies,
um, all kinds of
world perception, world understanding, we
froster people from the tomb have
worked on that quite a while.
Have a look at these two pages.
What people are already using for,
um, ROS 2.0 is coming up next
year.
I hope this will be create.
The negative thing about it is
it will deprecate some things.
And as you know, if something gets
deprecated, the person who once
wrote it might not be there anymore to
fix it.
So there might be some breakage.
But they're basing Ross on a much
more solid middleware.
So it feels that everything will stay the
same from an AP point
of view, from a user point of view,
but as the lowest level,
they will be using Dede's data
distribution services that you can add
real time guarantees, channel resources
and all other kinds of things and
transparency between.
Network based communication of nodes
and shared memory based communication
of nodes will be completely
so, you don't know whether
you run your nodes on the same
house values shared memory.
If you run it on multiple hosts, it will
do network serialization.
So this will be great if some
of you will start working
with us and help make it great.
That would be even better.
So I thank you very much for your
attention.
I hope someone will
read the line at the very bottom.
If you use drugs for any of your hobby
projects or in your hackerspace
plan to use it now or just get
interested, find me at the conference,
talk to me.
And really interesting to hear what
other people are using for.
Just sent me an email if you don't catch
me. Thank you.
OK, thank you, Andrea.
So far, we still have maybe one or
two minutes left for a few quick
questions if anyone has a question
and just come to one
of the seismic microphones.
OK, so that doesn't seem to be the case
then. Thank you so much, Andreas.
Thank you for listening and attending.