Utilizing AMD Schola and UnrealRoboticsLab with AMD ROCm™ Software to Train a Robotic Arm

10 min read. | 2443 total words.

A great reinforcement learning (RL) training environment excels along many axes. Unreal® Engine brings a powerful combination of capabilities, including physically based rendering, high-fidelity visual environments, and a mature toolset for building rich interactive scenes. These strengths make it an excellent fit for training tasks that involve complex lighting or rich vision-based observations.

AMD Schola is an open-source Unreal Engine plugin that brings standard Gymnasium-compatible RL training directly into the Unreal Editor, connecting to Python training frameworks over gRPC. Unreal Robotics Lab (URLab) is a plugin that embeds MuJoCo inside Unreal, letting you define robots in the MuJoCo XML format (MJCF) and drive them with fast, accurate contact physics. The bundled C++ source under source/ connects those two plugins in your game module, and together they form a pipeline that takes MuJoCo’s physics through Unreal’s rendering and tooling, with Schola exposing the result to a Python training loop on AMD hardware. This post walks through how that pipeline is structured and how to implement a reach task for a 6 degrees-of-freedom (DOF) robot arm from scratch.

Unreal® Engine 5.7 with the Schola version 2.1.1 and UnrealRoboticsLab commit 352a9ea plugins enabled in your project. Copy the bundled C++ glue from source/ into your game module (see Step 2); that is the only integration code required beyond those two plugins.

Unreal® Engine 5.7 with the Schola version 2.1.1 and UnrealRoboticsLab commit 352a9ea plugins enabled in your project. Copy the bundled C++ glue from source/ into your game module (see Step 2); that is the only integration code required beyond those two plugins.

Python 3.10-3.12 with the requirements of the Schola plugin installed.

Python 3.10-3.12 with the requirements of the Schola plugin installed.

Familiarity with MJCF robot descriptions. The MuJoCo documentation is a useful reference if you are new to the format.

Familiarity with MJCF robot descriptions. The MuJoCo documentation is a useful reference if you are new to the format.

A basic understanding of reinforcement learning concepts (observations, actions, reward, and episode termination).

A basic understanding of reinforcement learning concepts (observations, actions, reward, and episode termination).

Familiarity with Unreal Engine and how to navigate the editor.

Familiarity with Unreal Engine and how to navigate the editor.

RL loop over gRPC. Exposes the environment to Python as a standard gymnasium.Env

RL loop over gRPC. Exposes the environment to Python as a standard gymnasium.Env

URLab and Schola do not need to know about each other directly, and you do not install a third bridge plugin to connect them. The bundled AXArmEnvironment and AURLabScholaGymConnectorManager read the robot’s joints, sensors, and actuators from URLab, re-package them in the FDictSpace format that Schola expects, and start gRPC only after MuJoCo is ready.

Deeper configuration notes live in the Configuration Notes section.

Everything in the pipeline starts with an MJCF file. URLab imports this XML and instantiates the robot as a standard Unreal Engine actor. The reach task uses an xArm6 description with six hinge joints, six position actuators with ctrlrange bounds, and a single framepos sensor at the tool-center point (TCP).

The bundled MJCF also includes a gripper so the arm looks complete in the viewport. This reach task does not observe or control it as the environment exposes only the six arm joints, the TCP sensor, and six arm actuators. The gripper is still simulated with the model; we simply do not use it here.

The MJCF references stereolithography (STL) mesh files for visual geometry and collision. This post includes the MJCF only. Mesh files are not bundled. Download them from the xArm Robot Operating System (ROS) repository:

Arm links: download link_base.stl through link6.stl from xarm_description/meshes/xarm6/visual/

Arm links: download link_base.stl through link6.stl from xarm_description/meshes/xarm6/visual/

Place the XML and downloaded meshes so the relative paths in the <asset> block resolve. In your Unreal project, create a folder called MuJoCo/ under Content/ (for example, YourProject/Content/MuJoCo/). Create subfolders for the MJCF XML files (mjcf/) and meshes (meshes/) inside it. Since the bundled MJCF uses meshdir="../meshes", mesh files resolve relative to the XML folder in that adjacent Content/MuJoCo/meshes/ subdirectory:

Copy source/xarm6_reach.xml into Content/MuJoCo/mjcf/, download the meshes referenced in its <asset> block from xarm_description/meshes/ (starting with xarm6/visual/), and mirror that directory layout before importing. The MJCF is derived from UFACTORY’s xArm Unified Robot Description Format (URDF); see Acknowledgements for license terms.

With your MJCF and mesh assets ready, import them through URLab’s MuJoCo import action (for example, Import MuJoCo XML from the Content Browser or URLab menu, depending on your project setup). URLab compiles the model and creates an articulation Blueprint under your Content folder, typically an AMjArticulation-based asset you can drag into the level.

All screenshots in this post were captured in Unreal Engine 5.7.

Once import completes, URLab converts every <joint>, <actuator>, and <sensor> element into an Unreal Engine (UE) component (UMjJoint, UMjActuator, UMjSensor) attached to that articulation Blueprint. Open the Blueprint to inspect the component tree: you should see six arm actuators (joint1_act–joint6_act), the tcp_pos sensor, and any other elements your MJCF declares.

Drop an instance of the articulation into your level and you have a physically simulated robot you can inspect in the viewport before writing any RL code.

See Sensor Naming under Configuration Notes when you clone articulations for parallel training.

This post ships four C++ files under source/ that handle connecting the MuJoCo components on your xArm6 to Schola. Copy them into your game module:

Copy source/XArmEnvironment/XArmEnvironment.h and source/XArmEnvironment/XArmEnvironment.cpp into your game module’s Public/ and Private/ folders.

Copy source/XArmEnvironment/XArmEnvironment.h and source/XArmEnvironment/XArmEnvironment.cpp into your game module’s Public/ and Private/ folders.

Replace YOURPROJECT_API with your module’s API macro in all four files. You can find it on any existing class in your game module’s Public/ headers. It follows the pattern {MODULENAME}_API (for example, a module folder Source/MyGame/ uses MYGAME_API; Source/UnrealRoboticsLab/ uses UNREALROBOTICSLAB_API).

Replace YOURPROJECT_API with your module’s API macro in all four files. You can find it on any existing class in your game module’s Public/ headers. It follows the pattern {MODULENAME}_API (for example, a module folder Source/MyGame/ uses MYGAME_API; Source/UnrealRoboticsLab/ uses UNREALROBOTICSLAB_API).

Add URLab, Schola, and ScholaTraining to your game module’s Build.cs dependencies, then rebuild.

Add URLab, Schola, and ScholaTraining to your game module’s Build.cs dependencies, then rebuild.

AXArmEnvironment handles observation and action spaces, physics stepping, reset, reward shaping, and observation normalization. AURLabScholaGymConnectorManager defers Schola’s gym connector init until URLab finishes compiling MuJoCo. After rebuilding, place both actors in your level (see Step 3).

Schola requires every parallel environment instance to report identical observation and action space definitions. Rather than auto-discovering every joint and sensor on the articulation (which would silently expand the space if the MJCF changes), the bundled environment pins the reach-task I/O lists in code:

Gripper joints and actuators from the MJCF are omitted from these lists; only the six arm joints, the TCP sensor, six arm actuators, and the home keyframe participate in the Schola interface.

Unreal does not guarantee actor BeginPlay order. Schola’s stock gym connector manager calls Init() immediately, which can cache empty component maps if URLab has not finished compiling MuJoCo yet. The bundled connector manager polls until AAMjManager::IsInitialized() returns true:

AMjArticulation: your imported xArm6 instance (one per training environment; clone the actor for parallel envs).

AMjArticulation: your imported xArm6 instance (one per training environment; clone the actor for parallel envs).

AAMjManager: the URLab simulation manager that compiles and steps the shared MuJoCo world. Place one instance in the level.

AAMjManager: the URLab simulation manager that compiles and steps the shared MuJoCo world. Place one instance in the level.

AXArmEnvironment (or a Blueprint subclass): the reach task environment. Wire RobotArticulation and RobotManager to the actors above.

AXArmEnvironment (or a Blueprint subclass): the reach task environment. Wire RobotArticulation and RobotManager to the actors above.

AURLabScholaGymConnectorManager: drop-in replacement for Schola’s stock gym connector manager (from Step 2). It waits for URLab to compile before Schola initializes; without this actor Python may connect before the model is ready.

AURLabScholaGymConnectorManager: drop-in replacement for Schola’s stock gym connector manager (from Step 2). It waits for URLab to compile before Schola initializes; without this actor Python may connect before the model is ready.

A visual target marker: any static mesh actor for the goal indicator. The environment teleports it to the episode target on each reset.

A visual target marker: any static mesh actor for the goal indicator. The environment teleports it to the episode target on each reset.

Only the articulation and target marker have visible meshes. The URLab manager, Schola connector, and environment actor are present in the level but do not render in the view.

Use the Outliner when dragging references into the environment Details panel (RobotArticulation, RobotManager, TargetActor).

The provided MJCF, which includes onboard cameras, disable streaming before training. See Disable Onboard Camera Streaming (Recommended for Training) under Configuration Notes.

Select the environment actor and wire RobotArticulation, RobotManager, and TargetActor in the Details panel, then set TargetLocationMeters to position the reach target. The visual marker updates live in the editor as you edit target properties.

With the level wired up, the reach task is defined entirely in AXArmEnvironment: how observations are normalized for the policy, how reward and termination are computed each step, and how episodes reset with randomized targets. Most of these knobs are exposed as editor properties on the environment actor; the subsections below explain what each piece does and why the defaults are shaped the way they are. Expand a dropdown when you want to read the exact implementation from the bundled source.

ComputeReward_Implementation builds a scalar reward from TCP distance to the episode target, optional close-region shaping, success and out-of-bounds bonuses, and penalties on action rate and joint velocity. Distance thresholds are in meters; TCP positions arrive in Unreal world centimeters from URLab.

Success requires the TCP within SuccessThreshold and (optionally) arm joint velocities below SuccessMaxJointVelocity, so the episode does not terminate while the arm is still moving quickly through the goal. The action-rate penalty discourages the rapid full-range joint commands that are common in early training. The joint-velocity penalty encourages the arm to settle at the target rather than passing through it.

ComputeTermination_Implementation mirrors the reward geometry: success terminates the episode, out-of-bounds can terminate when enabled, and a step limit truncates long episodes.

MuJoCo sensor values span very different physical scales: joint positions in radians, velocities in rad/s, accelerations in rad/s², and TCP positions in centimeters. Before observations reach the policy, NormalizePolicyObservations divides each channel by a physically meaningful scale and clips to a symmetric range. Reward and termination use the raw UE-cm TCP readings from the snapshot, so you can tune normalization without changing reward geometry.

URLab reports TCP positions in Unreal world centimeters (MuJoCo meters × 100, Y-negated). Editor-facing target properties stay in env-local MuJoCo meters and convert once per episode via LocalMetersToUeCm:

Reward thresholds (SuccessThreshold, CloseRegionMeters, etc.) remain in meters for readability. Keeping the meter-to-UE-cm conversion in one place prevents silent sign-error bugs when the arm moves in the wrong direction.

At the start of each episode, Reset() clears the action-rate history, samples a new target location, resets the articulation to the home keyframe, and teleports the visual marker. SampleEpisodeTargetLocation() draws a uniform offset from the configured spawn center, symmetric in X and Y and upward-only in Z to keep the target above the table surface.

All target positions are stored in env-local MuJoCo coordinates (meters, local to the environment actor’s transform). You can clone the environment actor to multiple world positions for parallel training and each clone’s target will resolve correctly in world space at runtime without any additional configuration.

BuildInfo_Implementation returns an empty map by default; override it when you want to log diagnostics or curriculum metadata to Python.

With the environment implemented, there are two ways to launch training.

Editor workflow: open your level, confirm AURLabScholaGymConnectorManager and AXArmEnvironment are placed, then press Play In Editor (PIE). Wait for the log line indicating the gym connector initialized and gRPC is listening (port 8002 by default). Only then run the Schola CLI from a separate terminal as Python connects to the already-running Unreal process.

Python Managed workflow: package the level into an executable and let Schola launch it (preferred when you do not need the editor viewport). Using the project launch option will also note the location of the compiled executable, which can be used in following training runs.

While training can be run with or without the --headless parameter, we recommend including it to maximize throughput when rendering visual observations is not required (such as for this reach task).

Schola transmits the FDictSpace observation and action dicts to Python over gRPC on every step. Physics runs entirely inside Unreal at the configured MuJoCo timestep; Python sees only the high-level reset, step, and reward interface.

To increase training speed further, we can enable utilization of the GPU. Visit the installing PyTorch Python packages documentation on TheRock and navigate to the section for your GPU model.

Be sure to install this PyTorch inside the same virtual environment used for Schola.

Once PyTorch is installed in your Schola virtual environment, increase --num-simulators to launch multiple Unreal environment processes in parallel (Python Managed workflow). The GPU accelerates policy forward passes, observation batching, and PPO gradient updates on the Python side while the simulators gather experience concurrently.

Once training is complete, evaluate the trained policy against the live Unreal environment using Schola’s CLI eval entry point.

Omit the --headless parameter to visually inspect the trained policy’s behavior when launching from the executable. If you want to gather bulk performance statistics across a large number of evaluation episodes, you can re-enable --headless to maximize inference throughput.

This post covered the URLab + Schola pipeline for training a robot arm policy in Unreal Engine. Starting from an MJCF robot description, you imported the xArm6 into Unreal using URLab, connected it to Schola’s RL loop through the bundled C++ code under source/ (AXArmEnvironment plus AURLabScholaGymConnectorManager), and implemented a reach task covering reward shaping, observation normalization, and per-episode target randomization. On the Python side, Schola exposes the Unreal environment as a standard Gymnasium interface, making it immediately compatible with Stable Baselines 3 and exportable to the Open Neural Network Exchange (ONNX) format for inference.

The combination of MuJoCo’s accurate contact physics, Unreal’s rendering and editor tooling, and Schola’s gRPC-based training loop creates a pipeline that is practical to develop against: iterate on the reward in the editor, visualize the policy live in the scene, and deploy the trained ONNX model without framework changes. The xArm6 reach task in this post is a foundation for more complex manipulation scenarios that build on the same architecture.

We recommend designing your own tasks for the robot and building upon the reach task, such as enabling the gripper, using the camera, or defining a new objective for the robot to achieve. If you’re interested in Schola, or you’ve spotted a potential bug in the code, we welcome issues and pull requests at our GitHub repository.

URLab assigns each sensor an instance-prefixed name in MuJoCo (for example, xarm6_reach_C_0_tcp_pos). This prefix changes when you clone the articulation for parallel training environments. Always reference sensors by their UE component name (for example, tcp_pos) in your environment code. The bundled environment resolves these correctly across all instances.

If your MJCF includes RealSense-style onboard cameras (d435i_rgb, d435i_depth), URLab creates UMjCamera components that can stream 640×480 captures every frame. The camera is necessary for some environments, but it is computationally expensive and unneeded for the reach task. Turning off the per-camera Enable Zmq Broadcast alone is not enough: AAMjManager’s Network Manager forces Enable All Cameras on every camera at registration unless you turn it off.

On your AAMjManager actor, open Network Manager → Enable All Cameras and uncheck it. Also set bAutoCreateSimulateWidget = false on the manager if you do not need the MjSimulate dashboard during training (the widget re-enables camera streaming for its preview feeds).

If you still see camera meshes in the viewport, deactivate the d435i_rgb / d435i_depth components on the articulation Blueprint, or hide group-3 auxiliary geoms via the articulation’s Show Group 3 toggle.

This tutorial uses Unreal Robotics Lab (URLab) (Embley-Riches et al., ICRA 2026; arXiv:2504.14135). URLab is distributed under the Apache License 2.0.

The xArm6 MJCF bundled with this post (source/xarm6_reach.xml) was created from the xArm Unified Robot Description Format (URDF) in the xArm Robot Operating System (ROS) repository, specifically xarm_description/urdf/xarm. Mesh files are not included with this post; download them separately from xarm_description/meshes/ as referenced in the MJCF.

That upstream project is Copyright © 2018, UFACTORY Inc., and is distributed under the BSD-3-Clause license:

Unreal® is a trademark or registered trademark of Epic Games, Inc. in the United States of America and elsewhere.

The information contained herein is for informational purposes only and is subject to change without notice. While every precaution has been taken in the preparation of this document, it may contain technical inaccuracies, omissions and typographical errors, and AMD is under no obligation to update or otherwise correct this information. Advanced Micro Devices, Inc. makes no representations or warranties with respect to the accuracy or completeness of the contents of this document, and assumes no liability of any kind, including the implied warranties of noninfringement, merchantability or fitness for particular purposes, with respect to the operation or use of AMD hardware, software or other products described herein. No license, including implied or arising by estoppel, to any intellectual property rights is granted by this document. Terms and limitations applicable to the purchase or use of AMD products are set forth in a signed agreement between the parties or in AMD’s Standard Terms and Conditions of Sale. GD-18u.

© 2026 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo, ROCm, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective owners. Certain AMD technologies may require third-party enablement or activation. Supported features may vary by operating system. Please confirm with the system manufacturer for specific features. No technology or product can be completely secure.