KAMERA HATA ENJEKSİYON ARACI İLE KAMERA TABANLI ROBOTİK DENETLEME SİSTEMİNİN DOĞRULANMASI VE ONAYLANMASI
Abstract
Günümüzde, gelişen görüntü işleme teknikleri ile birlikte kamera tabanlı robotik inceleme sistemleri oldukça popülerlik kazanmıştır. Bu tür sistemler gıdadan, askeriyeye birçok sektörde yoğun olarak kullanılmaktadır. Bu sistemler geliştirilirken gerekli olan doğrulama ve onaylama süreçleri oldukça uzun ve maliyetli olmaktadır. Bu çalışma, kamera tabanlı endüstriyel robotik sistemler üzerinde doğrulama ve onaylama faaliyetlerini gerçekleştirmek ve iyileştirmek amacıyla geliştirilmiştir. RGB ve TOF kameralara farklı türlerde (Open, Close, Dilation, Erosion, Gradient, Motionblur, Tuz&Biber, Gaussian ve Poisson) hata enjeksiyon yöntemleri kullanılmasını mümkün Kamera Hata Enjeksiyon Aracı (CamFITool) ile gerçekleştirilmiş testler ve sonuçlar açıklanmıştır. Yapılan çalışma, VALU3S projesi kapsamında, OTOKAR’ın ROKOS robotik sistemine, CamFITool ile gerçek ortamdan alınmış kamera görüntülerinden oluşan kitaplıklara, çeşitli konfigürasyonlarda hatalar enjekte edilip, bu enjeksiyonun sisteme etkilerinin incelenmesine odaklanmıştır. Bu kapsamda 49 farklı test komfigürasyonunda hata enjeksiyonu gerçekleştirilmiştir. Sonuç olarak, kamera tabanlı endüstriyel robotik sistemlerin daha güvenli ve stabil çalışmalarının sağlanması için, bu sistemlerin hataya dayanıklı olup olmadıklarını test eden açık kaynaklı bir hata enjeksiyon aracı olan CamFITool önerilmiştir.
Supporting Institution
TÜBİTAK
Project Number
120N803
Thanks
Bu çalışma, İnovasyon Mühendislik tarafından yürütülen 120N803 numaralı TÜBİTAK Projesi kapsamında desteklenmektedir. Ayrıca bu çalışma, 876852 sayılı hibe sözleşmesi kapsamında ECSEL Ortak Girişiminden (JU) finansman almıştır. JU, Avrupa Birliği'nin Horizon 2020 araştırma ve yenilik programındaki Avusturya, Çek Cumhuriyeti, Almanya, İrlanda, İtalya, Portekiz, İspanya, İsveç, Türkiye 'den destek almaktadır. Bu belgede ifade edilen görüşler yalnızca yazarların sorumluluğundadır ve Avrupa Komisyonu'nun görüşlerine veya konumuna tepki göstermeyebilir.
References
- Acton, S. T., & Mukherjee, D. P. (2000). Scale space classification using area morphology. IEEE Transactions on Image Processing, 9(4), 623-635. http://dx.doi.org/10.1109/83.841939.
- Barbu, T. (2013). Variational image denoising approach with diffusion porous media flow. In Abstract and Applied Analysis (Vol. 2013). Hindawi. https://doi.org/10.1155/2013/856876.
- Blanter, Y. M., & Büttiker, M, (2000). Shot noise in mesoscopic conductors. Physics reports, 336(1-2), 1-166. https://doi.org/10.1016/S0370-1573(99)00123-4.
- Erdogmus, A. K., & Karaca, M. (2021). Manipulation of Camera Sensor Data via Fault Injection for Anomaly Detection Studies in Verification and Validation Activities For AI. arXiv preprint arXiv:2108.13803. http://dx.doi.org/10.48550/arXiv.2108.13803.
- Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2), 303-338. https://doi.org/10.1007/s11263-009-0275-4.
- Fisher, M., Cardoso, R. C., Collins, E. C., Dadswell, C., Dennis, L. A., Dixon, C., ... & Webster, M. (2021). An overview of verification and validation challenges for inspection robots. Robotics, 10(2), 67. https://doi.org/10.3390/robotics10020067.
- Jankowski, M. (2006). Erosion, dilation and related operators. Department of Electrical EngineeringUniversity of Southern Maine Portland, Maine, USA.
- Jha, S., Banerjee, S. S., Cyriac, J., Kalbarczyk, Z. T., & Iyer, R. K. (2018). Avfi: Fault injection for autonomous vehicles. In 2018 48th annual ieee/ifip international conference on dependable systems and networks workshops (dsn-w) (pp. 55-56). IEEE.
- http://dx.doi.org/10.1109/DSN-W.2018.00027.
- Ji, H., & Liu, C. (2008). Motion blur identification from image gradients. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE. http://dx.doi.org/10.1109/CVPR.2008.4587537.
- Kendall, A., Grimes, M., & Cipolla, R. (2015). Posenet: A convolutional network for real-time 6-dof camera relocalization. In Proceedings of the IEEE international conference on computer vision (pp. 2938-2946).
- Larnier, S., Fehrenbach, J., & Masmoudi, M. (2012). The topological gradient method: From optimal design to image processing. Milan Journal of Mathematics, 80, 411-441. https://doi.org/10.1007/s00032-012-0196-5.
- Park, H., & Mu Lee, K. (2017). Joint estimation of camera pose, depth, deblurring, and super-resolution from a blurred image sequence. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4613-4621).
- Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., ... & Ng, A. Y. (2009). ROS: an open-source Robot Operating System. In ICRA workshop on open source software (Vol. 3, No. 3.2, p. 5).
- Rosin, P., & Collomosse, J. (Eds.). (2012). Image and video-based artistic stylisation (Vol. 42). Springer Science & Business Media.
- Schottky, W. (2018). On spontaneous current fluctuations in various electrical conductors. Journal of Micro/Nanolithography, MEMS, and MOEMS, 17(4), 041001. https://doi.org/10.1117/1.JMM.17.4.041001.
- Yayan, U. & Erdoğmuş, A. (2021). Endüstriyel Robot Hareket Planlama Algoritmaları Performans Karşılaştırması. Journal of Scientific, Technology and Engineering Research, 2 (2), 31-45. DOI: 10.53525/jster.979689.
- Yayan, U., & Erdoğmuş, A. K. (2022). Development of A Fault Injection Tool & Dataset for Verification of Camera Based Perception In Robotic Systems. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi, 30(3), 328-339. https://doi.org/10.31796/ogummf.1054761.
- CamFITool ROS Wiki Sayfası. (2021). Erişim adresi: http://wiki.ros.org/camfitool/.
- Sciforce, (2019). Anomaly Detection, A Key Task for AI and Machine Learning, Explained. Erişim adresi: https://www.kdnuggets.com/2019/10/anomaly-detection-explained.html
- OpenCV, (2021). Morphological Transformations. Erişim adresi: https://docs.opencv.org/4.5.3/d9/d61/tutorial_py_morphological_ops.html.
- CamFITool Github Sayfası, (2021). Erişim adresi: https://github.com/inomuh/Camera-Fault-Injection-Tool.
VERIFICATION AND VALIDATION OF CAMERA-BASED ROBOTIC INSPECTION SYSTEM WITH CAMERA FAULT INJECTION TOOL
Abstract
Nowadays, camera-based robotic inspection systems have gained popularity with the developing image processing techniques. Such systems are used extensively in many sectors from food to military. The verification and validation processes required during the development of these systems are quite long and costly. This study was developed to perform and improve verification and validation activities on camera-based industrial robotic systems. The tests and results are explained with the Camera Fault Injection Tool (CamFITool), which enables the use of different types of fault injection methods (Open, Close, Dilation, Erosion, Gradient, Motionblur, Salt & Pepper, Gaussian and Poisson) to RGB and TOF cameras. Within the scope of the VALU3S project, the study focussed on OTOKAR's ROKOS robotic system by injecting faults in various configurations into libraries consisting of camera images taken from the real environment with CamFITool and analysing the effects of this injection on the system. In this context, fault injection was performed in 49 different test configurations. As a result, CamFITool, an open-source fault injection tool that tests the fault tolerance of camera-based industrial robotic systems, is proposed to ensure safer and more stable operation of these systems.
Keywords
Robotics Verification&Validation Fault injection Robot operating system Camera-based perception
Project Number
120N803
References
- Acton, S. T., & Mukherjee, D. P. (2000). Scale space classification using area morphology. IEEE Transactions on Image Processing, 9(4), 623-635. http://dx.doi.org/10.1109/83.841939.
- Barbu, T. (2013). Variational image denoising approach with diffusion porous media flow. In Abstract and Applied Analysis (Vol. 2013). Hindawi. https://doi.org/10.1155/2013/856876.
- Blanter, Y. M., & Büttiker, M, (2000). Shot noise in mesoscopic conductors. Physics reports, 336(1-2), 1-166. https://doi.org/10.1016/S0370-1573(99)00123-4.
- Erdogmus, A. K., & Karaca, M. (2021). Manipulation of Camera Sensor Data via Fault Injection for Anomaly Detection Studies in Verification and Validation Activities For AI. arXiv preprint arXiv:2108.13803. http://dx.doi.org/10.48550/arXiv.2108.13803.
- Everingham, M., Van Gool, L., Williams, C. K., Winn, J., & Zisserman, A. (2010). The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2), 303-338. https://doi.org/10.1007/s11263-009-0275-4.
- Fisher, M., Cardoso, R. C., Collins, E. C., Dadswell, C., Dennis, L. A., Dixon, C., ... & Webster, M. (2021). An overview of verification and validation challenges for inspection robots. Robotics, 10(2), 67. https://doi.org/10.3390/robotics10020067.
- Jankowski, M. (2006). Erosion, dilation and related operators. Department of Electrical EngineeringUniversity of Southern Maine Portland, Maine, USA.
- Jha, S., Banerjee, S. S., Cyriac, J., Kalbarczyk, Z. T., & Iyer, R. K. (2018). Avfi: Fault injection for autonomous vehicles. In 2018 48th annual ieee/ifip international conference on dependable systems and networks workshops (dsn-w) (pp. 55-56). IEEE.
- http://dx.doi.org/10.1109/DSN-W.2018.00027.
- Ji, H., & Liu, C. (2008). Motion blur identification from image gradients. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (pp. 1-8). IEEE. http://dx.doi.org/10.1109/CVPR.2008.4587537.
- Kendall, A., Grimes, M., & Cipolla, R. (2015). Posenet: A convolutional network for real-time 6-dof camera relocalization. In Proceedings of the IEEE international conference on computer vision (pp. 2938-2946).
- Larnier, S., Fehrenbach, J., & Masmoudi, M. (2012). The topological gradient method: From optimal design to image processing. Milan Journal of Mathematics, 80, 411-441. https://doi.org/10.1007/s00032-012-0196-5.
- Park, H., & Mu Lee, K. (2017). Joint estimation of camera pose, depth, deblurring, and super-resolution from a blurred image sequence. In Proceedings of the IEEE International Conference on Computer Vision (pp. 4613-4621).
- Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., ... & Ng, A. Y. (2009). ROS: an open-source Robot Operating System. In ICRA workshop on open source software (Vol. 3, No. 3.2, p. 5).
- Rosin, P., & Collomosse, J. (Eds.). (2012). Image and video-based artistic stylisation (Vol. 42). Springer Science & Business Media.
- Schottky, W. (2018). On spontaneous current fluctuations in various electrical conductors. Journal of Micro/Nanolithography, MEMS, and MOEMS, 17(4), 041001. https://doi.org/10.1117/1.JMM.17.4.041001.
- Yayan, U. & Erdoğmuş, A. (2021). Endüstriyel Robot Hareket Planlama Algoritmaları Performans Karşılaştırması. Journal of Scientific, Technology and Engineering Research, 2 (2), 31-45. DOI: 10.53525/jster.979689.
- Yayan, U., & Erdoğmuş, A. K. (2022). Development of A Fault Injection Tool & Dataset for Verification of Camera Based Perception In Robotic Systems. Eskişehir Osmangazi Üniversitesi Mühendislik ve Mimarlık Fakültesi Dergisi, 30(3), 328-339. https://doi.org/10.31796/ogummf.1054761.
- CamFITool ROS Wiki Sayfası. (2021). Erişim adresi: http://wiki.ros.org/camfitool/.
- Sciforce, (2019). Anomaly Detection, A Key Task for AI and Machine Learning, Explained. Erişim adresi: https://www.kdnuggets.com/2019/10/anomaly-detection-explained.html
- OpenCV, (2021). Morphological Transformations. Erişim adresi: https://docs.opencv.org/4.5.3/d9/d61/tutorial_py_morphological_ops.html.
- CamFITool Github Sayfası, (2021). Erişim adresi: https://github.com/inomuh/Camera-Fault-Injection-Tool.
Details
| Primary Language | Turkish |
|---|---|
| Subjects | Computer Software |
| Journal Section | Research Articles |
| Authors | |
| Project Number | 120N803 |
| Early Pub Date | April 22, 2024 |
| Publication Date | April 22, 2024 |
| Acceptance Date | February 20, 2024 |
| Published in Issue | Year 2024 Volume: 32 Issue: 1 |
