Hello, robot!
By Ahti Heinla, co-founder and CTO at Starship Technologies
I see robots every day. I see them sliding on the sidewalks at pedestrian speed, stopping to make sure it’s safe to cross the road. Sometimes I even catch them talking to pedestrians. This is a look at the fantasies of technologists – a wonderland of AI. But this is not a hallucination, not a dream, this is a reality that our team of dedicated visionaries has built over the last 5 years; we have moved the future into the present.
As recently as a few years ago, these robots needed little human support and were accompanied during their travels, similar to the form followed by autonomous car manufacturers who test their cars in public using “safety drivers”.
Starship became the first robotics team to start working regularly in public spaces about 18 months ago without the use of safety drivers; we leave our robots to explore the world on their own. We now operate our network of robots every day in several cities around the world, bringing people dinner, parcels and groceries.
Shared knowledge is acquired knowledge
It’s exciting to be first.
When I was a founding engineer at Skype, we were the first to make Voice over IP accessible in a practical way; we are now working to do the same with robots in public spaces. For four years, our engineering teams have been working behind closed doors on what is a significant breakthrough and an amazing experience.
I want to share with you some details of our technical journey. In the coming weeks and months, other members of Starship’s engineering team will also share aspects of their journey.
During this trip we worked with computer vision, road planning and finding obstacles-topics that are well researched in the field of academic robotics. In fact, Starship began as a research project, but soon became a functioning, practical delivery operation.
This means that in addition to fine-tuning Levenberg-Marquardt’s nonlinear optimization algorithm, we had to develop software to:
- Automatically calibrate most of our sensors – after all, we don’t want to spend hours calibrating them by hand; we have produced hundreds of robots and are currently preparing for a larger operation.
- Predict how much energy each trip will draw from the robot’s battery – so we can organize which robot to send based on the state of their battery.
- Predict how many minutes it takes a restaurant to prepare food – so the robot will appear just in time!
Most autonomous robots that exist in the world today are expensive, they are built as technology demonstrators or research vehicles and are not used for commercial operations. An autonomous sensor package alone can cost more than $ 10,000. It just won’t work in the delivery space, it’s not a luxury industry where you can charge a premium.
Autonomous driving test vehicles often have 3 kilowatts of computing power in the boot; impractical for a small, safe delivery robot. Therefore, part of our engineering journey is related to designing for a lower unit economy. Here are some topics we had to look at:
- Advanced image processing on a lower-end computing platform.
- Work around hardware problems in the software.
- Track how often robots need maintenance and why.
- Develop advanced route planning systems to ensure that we use our network of robots effectively.
It was quite a journey in visual design, involving hundreds of sketches, drawings and research before we produced the first plastic body of our robot.
In the early days, when we were still in stealth mode, we didn’t want to reveal what our robots looked like. Regular public testing required the creative use of a garbage bag glued to the robot’s body as a disguise!
Building practical robotics is a combination of science, systematic engineering and hacking. This combination of different disciplines is the main feature of Starship. Nothing is simple in robotics. All your knowledge of the situation is probabilistic; all sensors have modes of failure and malfunction and even a seemingly simple task such as causing the robot to stop at obstacles can become your own small research project.
Starship is a fast-growing start-up business and it is important not to become just a big research project. Engineers who are excited about Starship are often not pure scientists, they are not pure hackers, they are not pure engineers; they have several of these features and can use them as appropriate for the task at hand. We need complex technical solutions that can be implemented quickly and within the resource constraints of cheap hardware.
Ingenuity and resourcefulness are valuable skills.
One week is a lot of time in Starship
Earlier this week, our team will implement a new algorithm to detect curbs from point clouds and test it back for an entire database with full test cases overnight, they will test it live on our private test site by the end of the week. .
This will be on the street next Monday, with the team already reporting on its progress during our engineering meeting on Monday. Most Mondays, some of the engineering team reported 300% + profit on at least one of the indicators that had been achieved only a week before.
Data as a result and facilitator of scale
Performance and data have become a big part of Starship’s engineering.
You see, even when we were just starting out, we didn’t have data – we hadn’t driven much yet. Every day we modified our robot (yes, only the one then), took it out on the sidewalks and saw how it performed. We now have many that move autonomously every day – too many for engineers to observe directly.
Thanks to the data, we can now see how our robots perform, hundreds of them. We can organize weekly data diving workshops, where engineers share findings and monitor random deliveries to keep in touch with their work in action.
When we work to make our robots drive smoother, we analyze the data in the Acceleration Events table in our Data Warehouse; there are at least 1 billion rows in this table. Other tables include “crossing events”, our maps, every command that each robot has ever received from our servers, and apparently data collected from every delivery they perform.
Four years ago, we had none of that. Even when we were just starting out – and still not making commercial deliveries – I often had to convince people that robotic delivery really worked. It was hard for people to believe and they were quick to point out different reasons.
Do skepticism and fear always accompany new technologies?
A few years ago I landed at JFK Airport in New York with a robot in my luggage. The customs officer apparently asked, “What is this thing?” I explained that it was a sidewalk delivery robot, to which he replied, “Dude, this is New York! It will be stolen in minutes! ”
In fact, at the time, almost everyone thought these robots would be stolen – I’m sure they probably would be (mail delivery vans would be stolen, though rarely). To date, our robots have traveled more than 200,000 km (130,000 miles) and we have not yet seen this problem.
Of course, security features are available. The robot has sirens and 10 cameras, is constantly connected to the Internet and knows its exact location with an accuracy of 2 cm (thanks to the aforementioned Levenberg-Marquardt algorithm and 66,000 lines of automatically generated C ++ code that allows our robots to use it ).
People also believe that pedestrians may be afraid of sidewalk robots or not accept their presence. Will people call the police? Honestly, we weren’t sure about that either! However, after we put one of the robots on the sidewalk, a lot of surprise awaited us.
What happened next surprised us: people just ignored it. The majority of society paid no attention to the robots, even those who saw it for the first time, and people were certainly not afraid. Others will take out their phones and post on Instagram about how they saw the future.
And that’s what we wanted.
We want people to pay as much attention to our robots as they do to their dishwashers. This pattern of silent acceptance of robots, as if they have always been with us, is repeated in every city in the world where we have worked.
It’s getting better. Once people learn that these robots provide a useful service to the neighborhood, they develop an affinity for them. Kids even write letters thanks to robots, we have a “wall of thank you letters” to prove it!
Automating last mile delivery would never be easy, and we knew it would be a bold project. We also knew all along that there would be more than one fundamental obstacle that needed to be addressed – it turned out that there were hundreds of obstacles! But we have long realized that all these problems are solvable – they simply require ingenuity and perseverance.
Some startups start as a sprint run, putting together a minimally viable product for 3 months. For Starship, it’s more like a marathon – it takes a lot of effort, but the end result is huge for the world.
Last mile delivery is one of the world’s industries that has noticed slight technological disruptions from the acceptance of the car. The Starship team is working to change that, and with more than 20,000 deliveries under our belt, we’re on our way.
If you are interested in learning more, see our second post on the neural network engineering blog and how they power our robots here-https://medium.com/starshiptechnologies/how-neural-networks-power-robots-at-starship -3262cd317ec0