Automatic parcel induction and sorting for the last mile
In collaboration with the Bern University of Applied Science and the innovation specialist CSEM (Centre Suisse d’Electronique et de Micro-technique), Prolistic has developed an automatic induction system for the flexible parcel sorting system, eSorter. lncoming parcels are dumped and channeled to a conveyor. A 3D vision system and neural network detect individual parcels and their position. With this information, a robot picks each parcel individually and orients it correctly on an output conveyor.
The challenge
Stable 3D measurements during runtime
The growing e-commerce demand requires automatic parcel handling systems. Today, most automated parcel sorters use manual induction for the last mile. This method is slow, tedious and physically demanding.
The solution
Automatic parcel sorter
The system architecture is modular and allows many robot units based on the desired throughput. Modules communicate with gRPC/protobuf. CSEM's Visard builds the user interface and controls the whole process. The 3D-printed vacuum gripper has multiple gripping zones, combines suction cups and sealing foam, and includes a blow-off function. This allows manipulating parcels of varied size, weight and packaging material. The system combines vision, neuronal networks and robotics and shall:
- Handle parcels up to 10 kg weight and 400 x 300 x 200 mm3 volume
- Manage any packaging material
- Move at least 4000 parcels per hour
- ldentify parcel pose to ±5 mm and ±20°
- ldentify the address label and read it
The benefits
3D vision system and neural network
The eSorter system is equipped with a Nerian 3D stereo vision system. The 3D system consists out of the Karmin3 stereo vision camera, which is connected to SceneScan Pro, Nerian's stereo vision sensor. An additional LED-pattern projector adds texture to the measurement objects and improves the 3D results on almost any surface. The projector is triggered simultaneously through the SceneScan Pro system. The stereo vision system provides up to 135 fps or is capable to process up to 6 MP. A neuronal network segments and detects the parcels.
Learning is done with photo-realistic synthetic data, enabling access to all scene information (parcel dimensions, pose, materials) as well as fast and unlimited amount of training data. The network training must be supported by real data to close the simulation-reality-gap. The 3D pose estimation of the bounding box is calculated and sent to the robot parcel handler.
SceneScan Pro with Karmin3 stereo vision camera
3D depth camera and image processing
The Nerian SceneScan Pro is a 3D stereo vision sensor which connects either to Nerian's Karmin3 stereo camera or two Alvium USB3 cameras in order to get a full functional 3D stereo vision system. With the USB3 camera support, SceneScan Pro is a very flexible sensor which allows configuration and adaption to almost any application. It enables:
- extremely fast & detailed real-time 3D image processing with up to 135 fps
- precise depth information with up to 43 Million 3D points/sec
- direct transmission of fully computed 3D data via Ethernet, which solves the need for additional processing power on the host computer or GPU.
Highlights at a glance
Stable 3D measurements during runtime in harsh environments thanks to automatic camera recalibration
Ability to work indoors and outdoors without interference with sunlight or other sensors
Possibility to work with an additional LED-Pattern projector in order to improve the 3D results on challenging surfaces