Toyota Motor Europe NV/SA and its affiliates retain all intellectual property and proprietary rights in and to this software, related documentation and any modifications thereto. Any use, reproduction, disclosure or distribution of this software and related documentation without an express license agreement from Toyota Motor Europe NV/SA is strictly prohibited.

MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning

arXiv | website

Repository providing the source code for the paper

MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning
Mohammad Mohammadi* , Daniel Honerkamp* , Martin Büchner* , Matteo Cassinelli* , Tim Welschehold, Fabien Despinoy, Igor Gilitschenski and Abhinav Valada

Please cite the paper as follows:

@article{more25mohammadi,
title={MORE: Mobile Manipulation Rearrangement Through Grounded Language Reasoning},
author={Mohammadi, Mohammad and Honerkamp, Daniel and Büchner, Martin and Cassinelli, Matteo and Welschehold, Tim and Despinoy, Fabien and Gilitschenski, Igor and Valada, Abhinav},  
journal={IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year={2025}
}

Setup

Dockerized framework for robotic manipulation tasks with OmniGibson and BEHAVIOR-1K.

1. Build Docker image

Clone this repo locally and edit ./setup/Dockerfile and add your OpenAI API key.
Lastly, build the image by running the following command inside the repository:

docker build -t more:latest -f ./setup/Dockerfile .

2. Run Docker container

bash ./setup/run_docker.sh -h ./data

• When running the docker container for the first time, the BEHAVIOR-1K dataset will be downloaded at the specified location ./data. You can change if needed!

Run the code

Inside the container:

cd /app/more
python main.py agent=more +tasks='bringing_newspaper_in-0.yaml'

• You can run diffrent tasks just by changing the +tasks parameter. Task configurations can be found under config/tasks/. Those are the same referred in the paper.
• Set agent=momallm to run momallm baseline or agent=bumble to use BUMBLE. Their configuration can be found under the following folder configs/agent/.
• By deafult, gpt-4o is used for You can change this by adding the following parameter agent.llm_variant=<desired gpt version> to the bash command above or by manually changing it in the agent configuration file.

Experiments

Results can be replicated by executing all the task under config/task and compute the average of the metrics. Here below we provide the command for all MORE/BUMBLE configs tested:

# MORE
python main.py agent=more +tasks='<task_name>'

# MOMALLM
python main.py agent=momallm +tasks='<task_name>'

# MOMALLM + SPATIAL RELATIONS
python main.py agent=momallm agent.external_states=True +tasks='<task_name>'

# BUMBLE
python main.py agent=bumble +tasks='<task_name>'

# BUMBLE + FILTERING
python main.py agent=bumble agent.filter_irrelevant_objects=True +tasks='<task_name>'

Customize the code

Customize agents

• Agents can be costumized by edit their own .yaml file at configs/agent.
• If you want to develop a new agent, please refer for the code to the classes in more/agents/agent_generator.py and configs/agent/*.yaml for the configuration.

Customize task

• Task description can be edited manually by modifying the corresponding file under configs/tasks.
• Remember that each file is assigned to a specific task from BEHAVIOUR-1K! The underlying BDDL task definition can be found here: https://github.com/StanfordVL/bddl-100/tree/master/bddl/activity_definitions.
• If you want to define new task, fee free to check their documentation: https://behavior.stanford.edu/behavior_components/behavior_tasks.html#installing-bddl-for-customization-optional

Contacts

For any technical question, please contact Mohammad Mohammadi.
For any usage and collaboration inquiry, please contact Abhinav Valada.

Acknowledgement

We thank the authors and their work for the following (nice) repositories: BEHAVIOUR-1K, MoMa-LLM and BUMBLE.
Please refer to their license and citation for usage.

License

See the LICENSE file for details about the license under which this code is made available.