Human Handwriting Datasets
Several datasets have been proposed to study human handwriting and drawing behaviour. Early work such as the IAM handwriting database and more recent datasets like EMNIST captured handwritten characters in standardised image-based formats for recognition tasks, whereas more recent efforts have incorporated temporal and kinematic information.
However, most existing datasets remain image-based, and those that include dynamic information primarily capture position and stroke timing, with limited consideration of contact force. While datasets such as OnHW include additional sensing modalities, they are typically collected in unconstrained settings where participants write freely on paper without guidance on character structure, scale, or stroke order, with sensors capturing data throughout the process. This results in high variability, making such datasets less suitable for direct use in robot learning pipelines or for producing consistent trajectories aligned with robotic control requirements. Consequently, their applicability for learning human-like motion in physical interaction tasks is limited.
In contrast, teleoperation of a simulated robot is used during data collection, incorporating realistic constraints such as a maximum end-effector speed and requiring participants to adjust their trajectories accordingly. In addition, predefined character templates, font size, and stroke order are guided through the interface. This yields structured and consistent demonstrations that are better suited for training and evaluating learning methods for generating robot trajectories.
Robot Learning from Demonstration with GMM+GMR
Learning from demonstration enables robots to acquire new skills by observing human demonstrations rather than relying on explicit programming. This approach lowers the barrier to robot programming, allowing non-expert users to intuitively teach robots new tasks.
More recent imitation learning approaches, including diffusion-based policies and transformer-based action sequence models, can model temporally extended action sequences and leverage larger demonstration datasets. However, these methods typically require more data and computational resources and are generally less interpretable than probabilistic trajectory models. For the constrained, low-data setting considered in this work, Gaussian Mixture Model and Gaussian Mixture Regression offers a practical balance of expressiveness, interpretability, computational efficiency, and ease of implementation.
In most existing GMM+GMR-based learning-from-demonstration approaches, the input space is primarily limited to spatial position and time, and trajectories are assumed to be continuous. This study extends both aspects by incorporating force and normalized time as additional dimensions, enabling a richer representation of human dynamics and greater flexibility for integrating further modalities, and by adapting the framework to non-continuous, multi-segment trajectories. Moreover, data are often collected either via teleoperation or manually guiding the robot’s end-effector, both of which introduce robot dynamics into the recorded trajectories, making it challenging to isolate purely human-generated motion.
Human-Like Robot Motion and User Evaluation
This work captures human motion dynamics from demonstrations across all 52 uppercase and lowercase alphabet characters, and presents a user-centred evaluation of how closely the learned handwriting trajectories align with human perception of motion dynamics.
Methodology
Two separate user studies were conducted. The first collected human writing trajectory data while participants teleoperated a simulated robot, capturing position, speed, and force. The second evaluated whether the robot trained on this data produces human-like motion compared to the original fonts. The characters chosen were the 26 Latin alphabet characters in both uppercase and lowercase versions, which are highly nonlinear and complex, with discontinuities that make them challenging to program manually.
Participants
Two separate participant groups were recruited for each study phase to ensure evaluators had no prior knowledge that the robot motion was trained from human demonstrations. In total, 43 participants took part across both studies, with 22 participants contributing to the demonstration data collection phase and 21 participants evaluating the human-likeness of the resulting robot motion.
For the first user study, the 22 participants comprised 14 male and 8 female participants, aged 22–57 years (M = 31.45, SD = 10.60). Of these, 21 were right-handed and 1 was left-handed. Regarding stylus experience, 11 reported using one occasionally, 6 regularly, 4 never, and 1 only once or twice.
For the second user study, the 21 participants comprised 14 male and 7 female participants, aged 21–53 years (M = 27.24, SD = 8.47).
Ethics
Both phases of the study were approved by the Research Ethics Committee of the University of the West of England (Reference: 13470965). A participant information sheet was provided to all participants prior to obtaining their consent to take part, and signed consent forms were collected before the experiments began.
Appropriate measures were taken to ensure participant confidentiality and data security throughout the study. Participants retained the right to withdraw at any point and to request the removal of their data up to seven days following their participation.
Human Input Dataset
An example trajectory for uppercase B, second repetition by participant two, is shown in two-dimensional and three-dimensional views. The x and y axes show position in millimetres, the z axis represents the timestamp in seconds, and the colour map displays contact force in Newtons, with blue indicating lower force and red indicating higher force.
Learning Algorithm
Robot motion learning from human demonstrations is implemented using GMM+GMR. Two task-specific adaptations are introduced to meet the requirements of handwriting trajectory learning.
First, the state vector is augmented beyond spatial coordinates to include contact force and normalised time as additional learned dimensions, enabling the robot to reproduce not only the geometry of handwriting but also its dynamics and contact behaviour.
Second, a segment detection step is applied prior to training. This partitions each demonstration into individual strokes and trains each segment independently, addressing the trajectory degradation that arises at discontinuities caused by pen lifts between strokes.
Each demonstrated trajectory is represented as a four-dimensional state vector:
The state vector contains the two-dimensional position, contact force, and normalized time at each timestep.
Discussion
The generated trajectories exhibit similar spatial coverage to the human demonstrations while showing more consistent durations and smoother force profiles, reflecting the regularising effect of the learning algorithm.
The human demonstration dataset reveals the natural variability inherent in human handwriting. Duration ranged from 0.302 to 14.674 s with a standard deviation of 1.543 s, and contact force varied from 0.100 to 5.380 N, reflecting the considerable individual differences across participants and repetitions.
Despite this variability, the dataset collectively captures the spatial and dynamic characteristics of each character, providing a sufficiently rich basis for probabilistic learning. The similarity in mean path lengths between human demonstrations, at 65.31 mm, and robot-generated trajectories, at 63.87 mm, confirms that the GMM+GMR approach successfully preserves the spatial structure.
This study makes three contributions to robot learning from demonstration:
- It provides a publicly available handwriting dataset capturing position, timing, and force data, offering a resource for benchmarking learning methods.
- It extends the GMM+GMR framework with contact force and normalised time, enabling it to capture the full dynamics of human demonstrations beyond spatial paths and to generate complex multi-segment trajectories with discontinuities.
- It presents a user evaluation of the generated trajectories using a simulated robot, providing empirical evidence of perceived human-likeness.
This work demonstrates a touchscreen-based interface for teaching a robot through physical demonstration without requiring programming expertise, lowering the barrier to accessible human–robot interaction. While validated on Latin alphabet characters, the pipeline is in principle applicable to any complex surface trajectory, suggesting broader deployment potential in tasks requiring human-like motion.
Limitations include the sample sizes of 22 and 21 participants and single laboratory settings for each study, which may limit generalisability. Free-space transitions between strokes were not captured, resulting in linear interpolation, and pen orientation was not recorded during teleoperation.
The absence of a pre-programmed trajectory control condition limits direct comparison. Demonstrations followed predefined guidelines that may not reflect individual writing strategies, potentially limiting generalisation. More expressive writing styles, such as cursive or calligraphic writing, where pen orientation and force modulation play a greater role, remain for future work.
Finally, the evaluation was conducted using a simulated robot rather than a physical system due to hardware access constraints, which may not fully capture real-world deployment conditions.
Conclusion
The results demonstrate that probabilistic learning from demonstration is effective for generating human-like trajectories, with the publicly available dataset and implementation providing an open-source benchmark resource.
Frequently Asked Questions
What data does the handwriting dataset capture? The dataset captures position, timing, and contact force data from human demonstrations.
How does GMM+GMR support handwriting trajectory learning? It models spatial position, contact force, and normalized time, while training individual strokes separately to handle discontinuities.
How many participants took part in the two studies? A total of 43 participants took part: 22 collected demonstration data and 21 evaluated robot motion.
What are the main limitations of the study? Limitations include small participant groups, single laboratory settings, missing pen orientation and free-space transitions, and evaluation with a simulated rather than physical robot.