Autonomous Object Manipulation Task with a VLM Using a 6- DOF Robotic Arm

Abstract

Despite significant advancements in robotic manipulation within structured environ ments, current systems face persistent challenges when deployed in dynamic, un structured settings. These challenges become particularly evident in scenarios whererobots must interpret ambiguous human commands, detect and localise objects undervarying conditions, and perform complex manipulation tasks without pre-programmed instructions. The system exhibits several key limitations that impact overall performance. Firstly, object detection can be inaccurate due to the limited diversity of the training dataset and the inherent resolution constraints of the visual sensor. Additionally, localization accuracy is often compromised, particularly in scenarios involving inconsistent object placement or partial occlusions within the workspace. The system also experiences computational delays and inefficiencies within the control algorithms, which affect its ability to respond in real time and reduce the precision of robotic motion. Furthermore, there exists a misalignment between the control performance metrics and the practical demands of real-world manipulation tasks, thereby limiting the effectiveness of system evaluation and optimisation...