E02 – Shopping Guide

General description. A customer has gone to the mall for some shopping. Robotic shopping assistants operate in the mall to help customers during their shopping. One of their roles to is to accompany customers while wandering in the mall, properly reacting to unexpected situations. The robot task is therefore to safely follow a person through a public space: the robot will have to conduct proper behaviour according to the events, e.g., receiving instructions from the customer (“please wait for me for a moment”); dealing with obstacles and/or people standing in the way; following the customer through narrow passages; or detecting objects accidentally lost by the customer.

Platforms allowed. Any robot capable of Navigation, Perception, and Human-Robot Interaction capability can be used in this episode.

Setting. The episode takes place inside the shopping mall, both in the main wide corridors and inside a shop. A specific area inside which the robot will have to follow the customer should be specified. It should comprehend:

  • a wider part, maybe with a higher presence of people;
  • at least a narrow passage, simulating the entrance of a shop;
  • a narrower corridor, like the lane of a shop, where people can be encountered, as well as other objects to be avoided (trolleys, items fallen from the shelves, and so on).

Procedure. At the beginning of the episode, the robot may be required to memorise the customer that it will have to follow (as s/he can be not specified from the beginning). The customer may use some basic vocal interaction to tell the robot to behave in a certain way, e.g., asking to wait outside a shop.

DH interaction. Type: data consumption and generation. The MK:DataHub will provide location of the shops in the mall, as well as availability of products. Additionally, the MK:DataHub may provide crowd monitoring information. The robot should produce data about the crowdedness of some areas, detect potential hazards found on the way (slippery liquid poured on the floor) and inform about displaced items in the shop lanes.

Main functionality(ies). The main functionalities tested are Person tracking and following, Social Navigation in dynamic environments, Crowd detection and analysis and Semantic mapping.

Auxiliary functionalities. Other functionalities required are Spoken Language Understanding and Object Recognition.


  • Follow the customer inside a narrow passage
  • Navigate around group of people
  • Follow operator behind the group of persons
  • Understanding and reacting to the customer “wait” command (stop and restart when required)
  • Correctly reporting about crowdedness to the MK:Datahub.

Penalizing behaviors.

  • Touching the customer.
  • Misunderstanding or ignoring customer requests.
  • Obstructing the way of the customer.
  • Bumping into shopping mall walls and/or furniture.
  • Stopping following the customer.
  • Following another person.
  • Not producing the information required for the MK:Datahub.