Experimental Technology and Management

[Objective] In recent years, with the rapid development of large model technology, research on robotic arm manipulation based on large models has advanced and become a mainstream research direction. However, most existing educational platforms for robotic grasping focus on traditional algorithms and do not delve into integrating large-scale model-based technologies. This limits the students' understanding of cutting-edge intelligent grasp methods. To address this issue, an intelligent experimental teaching platform for robotic arm grasping was designed and developed based on domestic large models. The new platform integrates advanced technologies such as multimodal human–computer interaction, decision-making by large-language-model agents, open-vocabulary visual detection, and fine-tuning of large models. [Methods] The platform has a modular design comprising three core functional modules. First, a human–robot voice interaction channel was constructed using speech recognition and synthesis APIs to support Mandarin and English instructions and feedback. Second, the locally deployed DeepSeek, fine-tuned with Prompt-Tuning or QLoRA, was used to develop an agent system. This system parses user instructions, automatically decomposes them into executable task steps, and generates corresponding robotic-arm motion-control sequences. Third, the locally deployed Grounding DINO or the cloud-based multimodal Qwen model was used to recognize and locate target objects of any category, returning their position coordinates in the image workspace. In addition, a simulation environment was developed for the experimental platform. This environment was built using CoppeliaSim software, thereby significantly reducing experimental costs and facilitating offline learning for students. [Results] The results of physical and simulated grasping experiments on the designed platform revealed that the robotic arm grasping system performed complex grasping tasks through natural language interaction. The fine-tuned large-language model effectively parsed user instructions, automatically decomposed complex tasks into executable steps, and generated corresponding robotic-arm motion-control sequences in a rational manner. The deployed vision-language models detected various object types and provided accurate position coordinates for robotic-arm grasping. [Conclusions] The developed platform helps students to gain a deep understanding of the core knowledge needed for tasks such as robotic visual control and motion planning. In addition, it stimulates their research interests in artificial intelligence and robotics and cultivates their interdisciplinary innovation capabilities.