Advances in Autonomous Technology

Daniela Rus is the Andrew (1956) and Erna Viterbi Professor of Electrical Engineering and Computer Science and Director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT. An expert in robotics and computer science, Daniela believes that an increasing reliance on technology will greatly contribute to our quality of life and safety. Presenting at Imagine Solutions on her lab’s achievements around autonomous driving using deep neural networks, Daniela offered video testimony of how, despite real remaining challenges, we are close to driverless cars and related advancements becoming the technology of now.

00.00

[Music] thank you good morning so as a kid I loved the science fiction books of Jules Verne and the movies of Jacques Cousteau in some years later after I learned how to code and how to make robots I met a student who also loved Jules Verne and Jacques Cousteau and we decided to make a robot together our own version of the 20,000 Leagues Under the Sea Observatory we call this robot arm or which stood for autonomous modular optical underwater robot but building this robot was really about love and more was lovable it moved nimbly it could take awesome pictures in water it could communicate them all of these enabled by very advanced capabilities now one very hot summer day off the coast of Singapore we decided to see just how fast and far a moor could go we

01.01

were feeling very confident that day and we decided to take down all the safety features over more to give it more juice and then off the robot went and we waited and waited and waited but I'm more never returned we lost the robot and we were devastated now the thing about science is that the first copy of an invention takes a long time to make but subsequent copies are much faster and it is also true that great failures teach us great lessons so equipped with this wisdom we went back to the lab and within 90 days we had three improved versions of this robot Alva Moore and we took it to our end this watched it do flips in the water we started wanting more and this is how we decided to create a different robot you can think of it as the sibling of our more the soft robot fish Sophie this is

02.00

a robot with a soft tail and the robot is very agile it can change the swimming direction in about 100 milliseconds which is comparable to what real fish do it's called the escape maneuver because it allows little fish to escape the big fish who want to eat them so we took the robot to the ocean and there it is the black spot on its face it's a fisheye lens camera and I'm right there next to it we swam with with it and we watched it glide beautifully and elegantly by beautiful corals and the robot saw many schools of fish with its camera and it even found Nemo so it took us three years to make her more two months to make three copies over more and today in the lab we can make Sophie in a few hours and this rapid change of pace may sound familiar to you because just two

03.01

decades ago computing was a task reserved for the experts few computers were large and expensive and difficult to use you really needed expertise but today the smartphone has democratized computation in fact computing is so normal you don't even notice how you depend on it and this raises an important question in a world so changed by computers that helped us with our computing work what might it look with robots helping us with physical work how much work would we be able to offload to machines to save lives and increase the quality of our lives for example I believe that autonomous driving will ensure that there are no road fatalities and we'll give all of us higher quality of life in our retirement and will give us all the ability to go anywhere anytime it is not a matter of if it is a

04.01

matter of when so imagine driving home from work in your autonomous car knowing your vehicle has the smarts to keep you safe and the intelligence to make that right fun you pull at the gross you pull over at the grocery store to pick up supplies for dinner and a robot connects with your house figures out what you need then later on passes a box with the items you need and later on when you get home you pass that box to your kitchen robot and you might even let the children help with the cooking because even though they make a mess the cleaning robots will clean it up now this may sound to you like one of those cartoons about the future that never comes to pass but that future is not that far off today robots have become our partners in domestic and industrial settings they mow our lawns they clean our pools they even milk our cows and in

05.02

the future they will do so much more for us and this is enabled by three interrelated technologies and that are advancing and these technologies are robotics and you can think of robotics as the technology that puts computation in motion gives machines the ability to move and artificial intelligence gives machines the ability to have human-like characteristics to hear like humans to see like humans to play games like humans to communicate like humans and machine learning cuts across robotics and artificial intelligence and aims to learn and make predictions from data now all fields that have data can take advantage and so for instance medicine is a great example machines today can look at more radiology scans in a day than a doctor will see in a lifetime so

06.02

in a recent study machines and doctors were presented with scans of lymph node cells and they had to label them cancer or not cancer the machine made 7.5% error as compared to the humans 3.5% error but working together the humans and the machines achieved 80% improvement at 0.5% error which is really huge now today these systems are deployed in the most advanced cancer cysts centers in the world but imagine a day when these tools are available to everyone to overwork doctors to doctors in remote areas who do not have time to stay on top of the latest and greatest clinical trials everybody will be able to take advantage of the greatest work knowledge in order to offer us all of us the most personalized and effective treatment now many of the things you hear in the media

07.00

about machine learning and advances are due to techniques related to machine learning and in particular to a technique called deep neural network and in deep learning we use data a lot of data usually millions of hand labelled examples to determine the weights in a very large graph structure like the one sketched in this picture so that when animate an unlabeled image like this one gets presented to the network the system says it's a beach so this process works by analyzing the data and given today's technologies with images we have the first step is to segment the image to figure out which pixels go with what object in the image and this can be done by machine so you can put you can pass pictures into the network and you get a lot a lot of segmented data and once you have this collection of segmented data

08.00

the object in the picture have to be identified in order for the system to learn and do you know how this is done well this is how it's done lots and lots of people sitting in in centers and and manually labeling what those pictures represent so I wanted to explain this to you because in addition to understanding how the technique works it's also important to understand what it means for a machine to recognize an image we say it's learning but in fact learning means that the machine associate a set of pixels with a different set of pixels some human labeled with a certain concept but in the process of saying this is a beach the machine has no idea what you do with a beach what do you do is it do you eat it do you do you drink it you play on it how does it feel none of that is associated with a beach

09.01

at the level of machine learning and so let me show you another another example take a look at these two pictures and let me and let me ask you if they look the same to you well they should because they look the same to me but in fact they are slightly different there is a little bit of imperceptible noise that is injected in these pictures and this noise is enough to trick a network that correctly recognizes the first picture as a dog to recognize the same picture as an ostrich so we are enabling a lot of new applications with these techniques but these techniques are not perfect and if you're going to use them or rely on them you have to understand when they work how they work what kind of error they yield so deep learning has enabled applications but there are significant limitations take a look at this picture so this is an 18 months old

10.01

that's watching the scene for the very first time ever and look at what he's done he's figured out what the adult wanted he goes and executes an action to help the adult and then he makes like contact as though saying did I understand you so we have no machines that have the capability this extraordinary capability of the human mind we are working on some of these grand challenges and we hope that in the future we will have some progress but today's AI systems are not able to do what is 18 months old is doing so there are significant challenges with today's techniques manual data labeling obtaining massive sets and biasing the algorithms but there are also huge opportunities like I have shown you in the in the medical case and the opportunities are around personalization customization but with machines and people working together because machines are better at some

11.01

things than we are and we are better at some things than machines are working together we can do so much better now working together machines and people can also become better transportation systems and in one of a recent research projects we developed an algorithm that matches to apply in demand we applied it to taxis and and in particular to the traffic in New York City and we have shown that we were able to remove about eleven thousand cars from the roads of New York City and still provide a very comparable level service of about a three minutes delay - when you get to the destination so these techniques can really help us increase the quality of our lives now if the vehicles become autonomous we have the added benefit that we expect to remove

12.00

road fatalities and we are very much working on this problem in fact this is the MIT C style autonomous car its body is the regular car extended with sensors later scanners GPS and cameras and this car operates by by driving in the world by looking at what the sensors give to understand what's happening around me around it by detecting where the obstacles are localizing and planning where to go next so this autonomous driving pipeline is actually the pipeline that's adopted by most commercial companies and in fact there are some challenges because every situation needs to be explicitly modeled for each type of situation whether you're driving at night or you're driving on roads with or without lane markings etc so autonomous systems really need the ability to handle a wide

13.02

range of scenarios and in response to these scenarios they just need to know what to do right now we hand code everything now in my research group at MIT we are adopting another method which is to learn from humans how to drive so what we are doing is we're using a deep neural network engine and we're looking at how examples of how humans steer the car in certain Road situations and from that we decide how to associate that particular image with a control system and let's look at what we can do this is my graduate student Alexander he's listening to country music which I took out for this talk but in this in this example Alexander is telling the car let's go the car has a topological map of the world so the cat

14.01

the map is very simple it's literally what you see in the corner and this is a first drive of the car on this road the car was trained in a different environment and just look at how how easily and smoothly this car glides on the road and figures out what to do I told you that I was involved in teaching my daughter's how to drive and their first drive I was not as smooth as have this car so the car uses three cameras that are it uses left center and right cameras and the images from these cameras are fused and when it comes upon intersection an intersection it's able to look at and special cues that it finds special features that it finds in these Sekulow in the images to figure out what to do so it's very exciting I think that although we don't have level 5 autonomy we do have robust autonomous

15.03

driving that could be deployed immediately in places like villages in places where we have a low complexity in the environment fairly low driving speed now here's another awesome thing we can do the same technology for autonomous driving can be mapped onto wearables where the laser scanners become belts and the cameras become necklaces and with this we can provide blind and vision impaired users to safely navigate the world so with the system a blind user can walk down the street and get a fabulous description of a fabulous window display or warning of a presence of an obstacle or that a friend is going by so we can use this technology to replace the walking stick and provide blind people and visually impaired people with the kind of flexibility and

16.00

ease of movement in the world that is simply not possible today so here's our blind user who has used the system and has found a way to get to a bench sit on it and feed the duck the system will tell the user when the Ducks are going by so we're making great technological advances but in order to for these advances to be broadly used people and machines have to be able to work together without knowing precisely what happens under the hood you can drive your cars without understanding every component of the engine so how can we get there one interesting question we're posing in my research group is can we get to a point where machines adapt to us rather than the other way around because today we are adapting to machines in that we have to learn how to program them so question is can we connect our ideas and our our desires directly to the machine and the answer

17.00

is in general no in general the kind of sensors we have to listen for brain activity are too impoverished to make sense of of our brain waves but there is one type of thought that of brainwave pattern that we can detect reliably and this is what our brain does when we see something that is wrong it is that you are wrong signal it's it does it's not trained it's the same no matter what language you speak in so by detecting that you are wrong signal we are able to connect people with machines directly in a very simple scenario but this is beginning of what might come so here's our user wearing an EEG cap and through this EEG cap we have 48 electrodes that measure brain activity and we have a robot that's tasks to store it paint can and wires for paint goes in one box wire goes in the other and when the robot

18.01

makes the wrong move the user thinks oh that's wrong that's that triggers the special ear related potential and that allows the robot to move over to the other box so here the robot made a mistake the user detected it and look at how the robot changed without any kind of actual programming intervention so there are some very exciting opportunities for AI machine learning and robotics for the future and I really believe that with these technologies we will reduce and eliminate car accidents we will better diagnose monitor and treat disease and we will keep your information private and safe we will transport people and goods faster and more efficiently and safer we will deliver education to everyone we ensure that people can connect to one another instantly globally no matter what language they speak in general we will allow humans and machines to work

19.01

together with machines doing routine tasks and humans focusing on the high level critical thinking strategic tasks the future is great thank you very much [Applause]