Research

Research

My research explores Behavioral Steerability in humanoid behavior foundation models and Human-AI Co-creation for complex artifact generation.

arXiv Preprint arXiv:2610.10198 (2026)

Benchmarking Behavioral Steerability in Behavior Foundation Models

Minghe Gao*, Zhanxi Yan, Jiahui Liu, Wendong Bu, Xiaoting Chen, Qizhou Wang*, Yi Su, Siliang Tang, Jun Xiao, Yueting Zhuang, Tat-Seng Chua, Juncheng Li†
Joint Laboratory of Embodied Intelligence, Zhejiang University & Unitree Robotics

arXiv Paper Project Page Dataset (HuggingFace) Evaluator Benchmark Code

Figure 1. (a) Foundation models evolve from single-task generation toward steerable generalists. (b) Steerable behavior foundation models reliably execute multimodal user intentions.

Research Problem

Behavior Foundation Models (BFMs) translate human intentions into executable humanoid behaviors. However, current benchmarks focus primarily on unconstrained motion generation or single-task fidelity. RoboSteer introduces the concept of behavioral steerability—the ability of BFMs to faithfully generate behaviors that satisfy user-specified behavioral intentions—and establishes the first unified evaluation benchmark.

Figure 2. Overview of the three-level behavioral steerability hierarchy in RoboSteer: Conditional, Constraint, and Compositional Steering.

Three-Level Steering Hierarchy

• Level 1 — Conditional Steering (12 task types): Generates behaviors matching input conditions across text, video, audio rhythm, and pose trajectories, testing full conditioning reproduction, temporal completion, and limb completion.
• Level 2 — Constraint Steering (7 constraint types): Executes behaviors under explicit physical constraints: controlling movement speed, amplitude, direction, action order, repetition times, spatial trajectory, or restraining specific limbs.
• Level 3 — Compositional Steering (1 task type): Interleaved multi-source steering that sequentially coordinates diverse conditions within continuous humanoid behavior.

Figure 3. Distribution and statistics of multimodal conditions and constraint categories across the RoboSteer benchmark corpus.

Personal Contribution (Second Author)

• Benchmark Task Construction: Formalized task specifications and boundary conditions across all three hierarchy levels to systematically evaluate model steerability limits.
• Multimodal Benchmark Curation: Built and curated multimodal prompt pairings across natural language descriptions, reference videos, audio rhythms, and pose keyframes.
• Data Organization & Quality Assurance: Established dataset organization schemas and conducted verification protocols on kinematic limits and metadata integrity for the public HuggingFace release.

BibTeX Citation

@misc{gao2026benchmarking,
  title={Benchmarking Behavioral Steerability in Behavior Foundation Models},
  author={Minghe Gao and Zhanxi Yan and Jiahui Liu and Wendong Bu and Xiaoting Chen and Qizhou Wang and Yi Su and Siliang Tang and Jun Xiao and Yueting Zhuang and Tat-Seng Chua and Juncheng Li},
  year={2026},
  eprint={2610.10198},
  archivePrefix={arXiv},
  primaryClass={cs.RO},
  url={https://arxiv.org/abs/2610.10198}
}

---

ICIC 2026 Springer CCIS / LNCS

Setting the PACE: A Progressive App Co-creation Environment for Complex App Design

Shixian Ding, Xinyu Yu, Zhanxi Yan, Fengchang Liu, Chengwei Shi, Yanchang Dong, Liuqing Chen
College of Computer Science and Technology, Zhejiang University & ZJU-UIUC Institute

Springer DOI (10.1007/978-981-92-3397-7_47)

Figure 1. Co-creation across layers in solution space. Users progressively drive the design from structural definitions and skeleton assembly to visual specification and high-fidelity generation with localized prompt edits.

Research Problem

App design is driven by complex functional requirements and UX goals. While Generative AI offers assistance, current tools predominantly rely on a one-shot generation paradigm. For novice designers, this black-box approach obscures design logic, forces premature commitment, and provides limited control.

Figure 2. Conceptual comparison between the traditional "one-shot" generation approach and PACE's progressive co-creation approach grounded in cumulative exploration, reusable module generation, and cognitive layering.

Methodology & Architecture

PACE introduces a progressive co-creation workflow grounded in theories of complex artifacts:

• Progressive Workflow: Grounded in Problem Exploration, Structural Decoupling, and Cognitive Decomposition, replacing opaque one-shot prompts.
• Multi-Granularity Structured Representation: Unfolds the generation process progressively from functional dependency logic, to skeleton wireframes, to visual interface assets.
• Structural Decoupling: Allows novice designers to incrementally generate, refine, and assemble independent functional organisms without breaking global UI coherence.
• Empirical Validation: A comparative user study with 16 novice designers demonstrated that PACE significantly enhances user agency, creative support, and artifact quality.

Personal Contribution

Co-authored the paper; contributed to the formulation of the multi-stage representation pipeline (functional logic to wireframes), structural decoupling mechanisms, and empirical user evaluation analysis.

BibTeX Citation

@inproceedings{ding2026setting,
  title={Setting the PACE: A Progressive App Co-creation Environment for Complex App Design},
  author={Ding, Shixian and Yu, Xinyu and Yan, Zhanxi and Liu, Fengchang and Shi, Chengwei and Dong, Yanchang and Chen, Liuqing},
  booktitle={International Conference on Intelligent Computing (ICIC 2026)},
  series={Communications in Computer and Information Science},
  pages={568--579},
  year={2026},
  publisher={Springer},
  doi={10.1007/978-981-92-3397-7_47}
}