E08 – At the checkout desk

General description. The robot is in front of the checkout desk of a shop or supermarket. On the desk are the items purchased by a customer, which are a set of different objects differing in size and shape as well as in physical properties (such as hardness/deformability, fragility, surface properties). For instance, possible items may comprise: a metal soup can (cylindrical, heavy, not deformable, not fragile, not slippery), a paper flour packet (box-shaped, heavy, deformable, not fragile, quite slippery), a plastic blister containing two light bulbs (irregular shape, light, deformable in some places, fragile, not slippery) and a pack of sponges (box-shaped, light, highly deformable, not fragile, not slippery).

Upon the desk, beside the purchased items, is a shopping basket of the type generally used in grocery shops or supermarket for small quantities of products: i.e., made of plastic or metal, open from the top side, with holes in the sides that can be used to perceive contents. The basket is not affixed to the desk in any way, so that if pushed by the robot it will move.

A good choice for the first editions of the episode is a basket made from metal wire: heavy (thus difficult to move) and with sides that are very transparent to perception. In later editions, the episode may be made more difficult by using a plastic basket, which is lighter and more obstructive perceptually. The task of the robot is to take the items from the desk and store them in the basket, without dropping or damaging them. The space in the basket is limited; moreover, the purchased items, when in the basket, should remain within the volume of the basket without protruding from its top: therefore the robot should take into account the shape of the items in order to plan and execute an efficient storage configuration. For first editions, this requirement may be made easy to comply with by making the item set small and/or choosing small items; later, difficulty may be raised by increasing the number and/or size of the items, as well as by including items which are difficult to manipulate.

Platforms allowed. Any mobile platform with a manipulator.

Setting. The actual checkout desk of one of the shops of the shopping mall, possibly moved out of its usual position to help visibility by the public. No human-robot interaction occurs during the episode.

Procedure. The episode has a fixed duration. It starts when the robot has been (manually) positioned in front of the checkout desk and it ends when a timeout is reached. During the course of the episode the robot can move the items (both those outside and those inside the basket) as many times as needed. When the timeout is reached the robot must stop and the items in the basket are evaluated by human referees.

DH interaction. Type: data consumption and generation. The list of goods to place in the basket is communicated by the MK:DataHub. It will communicate to the robot what to pick (and in which quantity) from the different goodies. Moreover, after having finalised the task, the robot has to update the quantity of each product in the MK:DataHub.

Main functionality(ies). The main functionalities tested are Object Perception and Manipulation (the latter including both planning and execution).

Auxiliary functionalities. Other required functionality is strategic planning to optimally store the items into the basket.


  • Correctly placing the items into the basket.
  • Not damaging any items.
  • No items protrude above the top of the side walls of the basket.
  • Correct updating of the MK:DataHub.

Penalising behaviours.

  • Damaging an item.
  • Letting an item fall to the floor.
  • Damaging the basket or the checkout desk.
  • Letting the basket fall to the floor.
  • Wrongly updating the MK:DataHub.