← Portfolio

Digital Twins

Analyzing time-series of robots in production facilities

Rafael Konlechner, June 2018
Tags:
At CDL-MINT, we test new ways of programming robots in industial contexts. To experiment with the integration of runtime-data with design models, I developed a 3D simulation of a robotic arm.

Robots can be remarkably powerful at completing simple and repeated tasks, but it is hard to develop and maintain the software for more complex scenarios1. With increasingly flexible production, involving a multitude of different machines and systems, traditional ways of developing robot controllers are no longer efficient. The CDL-MINT investigates, whether model-driven development and the integration of runtime data will help to make industrial robot programming easier and less error-prone. In short, we investigate the concept of Digital twins 2. My main task as an engineer is to provide a playing field for experimentation. One of these playing fields is the simulated pick-and-place unit that you can see in the video below.

The pick-and-place unit, looping through a simple procedure.

Virtual Internet of Things

The extraordinary thing about this simulation is the way it is controlled. The simulation communicates via Internet protocols and its components act just like regular IoT-devices. This enables us to run most of the logic on a server, rather than on local hardware controllers, leaving us with more computational resources and thus, more powerful programming tools.

The simulated scene is equipped with a number of movement sensors, as well as two virtual cameras, that monitor items when they pass the unit. In the clips below, you can watch items being detected on the conveyor and the QR codes, that are scanned when the items are placed on the workbench. All this data is available to procedure programmers via a message-based API.

Two virtual cameras check for items on the belt and scan the product QR code.

Thanks to this API, programming procedures for the pick-and-place unit is flexible and not bound to a special programming language. The figure below adds a small user interface for monitoring the unit and programming new procedures in a graphical editor.

Editor with rich information about the current state.

Time Series Data

For further analysis, we record all sensor data sent from the unit. This establishes a feedback-loop to the software controller and can help to optimize or adapt the controlling procedure. The real-time analysis of sensor data has itself become a large field of research.

All sensor measurements are recorded in a time-series database.

Physical Robots

In Spring of 2018, the playing field was joined by two physical robots. These robotic arms are controlled by custom Raspberry Pi computers, developed by the Practical Robotics Institute Austria.

Relevance

So far, the work related to 3D simulations and the physical robotic arms resulted in two publications 3, 4 and one thesis (in progress).


Notes and References

  1. Robot Conquers One of the Hardest Human Tasks: Assembling Ikea Furniture. Published in: The New York Times. (Link to Article)
  2. Wikipedia article about Digital twins (Link to Article)
  3. Wolny, S., Mazak, A., Konlechner, R., & Wimmer, M. (2017). Towards Continuous Behavior Mining. Data-driven Process Discovery and Analysis 2017., 2016, 149-150.
  4. Wolny, S., Mazak, A., Wimmer, M., Konlechner, R., & Kappel, G. (2018). Model-Driven Time-Series Analytics.Enterprise Modelling and Information Systems Architectures, 13, 252-261.