Building truly useful in-home service robots. “Habit” + “Robot”.

Update #1: April 20.

Mechanical design progress screenshots below.

Background and Motivation

Part 1: Habit-Learning Cognitive Architecture

We all have countless habits, big and small, that keep our lives efficient. I would bet that you couldn’t even list them all, since some are so deeply buried in your routine. It is essential that a robot picks up on these habits and conforms to them if they are to be as useful as possible.

Cognitive Architecture: perception, action, learning, adaptation, autonomy, memory, and reasoning

The idea is that although much of current research is based around task learning (e.g., through VLAs or foundation models), for a robot to be even more useful in the home, it would need to learn to adapt to the specific habits of its user.

I’m planning on training a model using data about previous tasks, including time, state of home when the task was requested, state of the external world like weather, and various calendar events (e.g., dinner guests), so that the robot can develop habits and predict which tasks will be requested at which time.

This could be pre-trained on more generic data and then refined by each user individually through reinforcement learning where the user gives feedback on any developed habits.

Various memory systems could be integrated, but I’m currently considering a multi-layer graph with each modality of data as a separate layer (tasks, states, and external events). Each layer can host inter and intra connections to inform the model.

The goal would be that within a couple weeks of living together with the user, the robot would be able to work seamlessly with the user and provide personalized help.

Part 2: Stair-Climbing Wheelbase

Wheelbase in-home service robot with a small footprint that can rapidly climb stairs.

Currently, much of the industry and research attention is focused on legged robots for inside the home.

However, wheelbase robots provide greater efficiency, stability, speed, and accuracy. There is just one problem: they cannot climb bumps and stairs.

Existing mechanisms either require a large footprint (unsuitable for the home), or are extremely slow (unsuitable for the home). I plan to fix this through this project.

Multi Joint Contact simulation slowed down

Previous Relevant Projects:

Readings

Interactive Continual Learning: https://arxiv.org/pdf/2403.03462

LLM Integration for home service robots:
https://arxiv.org/pdf/2305.05658

Few Shot Class Incremental Learning:
https://arxiv.org/pdf/2404.02117