Hacia la construcción de una base de datos abierta de la LSM

Keny Ordaz-Hernández; Diego Castillo-Gaytán; Arturo Segundo Rodríguez-Recio; Rodrigo Daniel Boone-Obregón; Luis Ángel Hernández-García; Gabriela Hilario-Acuapan

doi:10.29057/icbi.v11iEspecial2.10699

Keny Ordaz-Hernández Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional https://orcid.org/0000-0001-9528-9035
Diego Castillo-Gaytán Tecnológico de Monterrey campus Saltillo https://orcid.org/0009-0005-5051-1735
Arturo Segundo Rodríguez-Recio Tecnológico de Monterrey campus Saltillo https://orcid.org/0000-0002-7968-2600
Rodrigo Daniel Boone-Obregón Tecnológico de Monterrey campus Saltillo https://orcid.org/0009-0002-1129-0165
Luis Ángel Hernández-García Tecnológico de Monterrey campus Saltillo https://orcid.org/0009-0002-0153-3637
Gabriela Hilario-Acuapan Centro de Investigación y de Estudios Avanzados del Instituto Politécnico Nacional Unidad Saltillo https://orcid.org/0009-0009-8411-6214

DOI: https://doi.org/10.29057/icbi.v11iEspecial2.10699

Keywords: fingerspelling, mexican sign language, robotic hand, database, labelling

Abstract

This work reports the initial phase of the creation of an open visual database for the finger-spelling alphabet of the MSL. The design of the database is reported, and it consists of 29 static and dynamic signs. For each sign, the RGB color frames are captured along with the depth map, by means of RGB-D sensors. Also, in order to provide synthetic samples, a virtual robotic hand has been created to present the finger-spelling configurations. A reduced set of 7 signs has been selected to be presented and analyzed, in a robotics simulator.

Downloads

Download data is not yet available.

References

Aponte, P. y cvlabbonn (2015). Tools openni2. https://github.com/krontzo/recording_openni2_primesense.

Bradski, G. (1998). Real time face and object tracking as a component of a perceptual user interface. En Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV’98 (Cat. No.98EX201), pp. 214–219.

Buchholz, B., Armstrong, T. J., y Goldstein, S. A. (1992). Anthropometric data for describing the kinematics of the human hand. Ergonomics, 35(3):261–273. PMID: 1572336.

Bustos-Rubilar, M., Coloma, C.-J., Quezada, C., Caviedes, C., Morales, M.-F., Adrián, J., y País, J. (2021). La dactilología para el aprendizaje de la lectura en escolares oyentes: Un estudio exploratorio. Revista de Educación Inclusiva, 14(1):156–167.

Caballero-Morales, S.-O. y Trujillo-Romero, F. (2013). 3d modeling of the mexican sign language for a speech-to-sign language system. Computación y Sistemas, 17(4):593–608.

Canny, J. (1986). A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-8(6):679–698.

Carmona-Arroyo, G., Rios-Figueroa, H. V., y Avendaño-Garrido, M. L. (2021). Mexican sign-language static-alphabet recognition using 3d affine invariants. En Malarvel, M., Nayak, S., Pattnaik, P., y Panda, S., editores, Machine Vision Inspection Systems, Volume 2, capítulo 9, pp. 171–192. John Wiley & Sons, Ltd.

Coppelia Robotics (2020). CoppeliaSim v4.4.0 (september 22nd, 2022). https://www.coppeliarobotics.com.

Coppelia Robotics (2023). Zeromq remote api. https://www.coppeliarobotics.com/helpFiles/en/zmqRemoteApiOverview.htm.

Cruz-Aldrete, M. (2014). Hacia la construcción de un diccionario de lengua de señas mexicana. Revista de Investigación, 38:57–80.

Degen, C., Lenarz, T., y Willenborg, K. (2020). Acute profound sensorineural hearing loss after COVID-19 pneumonia. Mayo Clinic Proceedings, 95(8):1801–1803.

Escobedo Delgado, C. E., editor (2017). Diccionario de Lengua de Señas Mexicana de la Ciudad de México. INDEPEDI.

Galicia, R., Carranza, O., Jiménez, E. D., y Rivera, G. E. (2015). Mexican sign language recognition using movement sensor. En 2015 IEEE 24th International Symposium on Industrial Electronics (ISIE), pp. 573–578.

Garcia-Bautista, G., Trujillo-Romero, F., y Diaz-Gonzalez, G. (2016). Advances to the development of a basic Mexican sign-to-speech and text language translator. En Tescher, A. G., editor, Applications of Digital Image Processing XXXIX, volumen 9971, p. 99713E. International Society for Optics and Photonics, SPIE.

García Orza, J. (2002). Neuropsicología cognitiva de la lengua de signos: una piedra de toque para el estudio del lenguaje, la visión, las emociones faciales y el movimiento. Revista de psicología general y aplicada: Revista de la Federación Española de Asociaciones de Psicología, 55(1):89–104.

García-Bautista, G., Trujillo-Romero, F., y Caballero-Morales, S. O. (2017). Mexican sign language recognition using kinect and data time warping algorithm. En 2017 International Conference on Electronics, Communications and Computers (CONIELECOMP), pp. 1–5.

Giovanelli, E., Gianfreda, G., Gessa, E., Valzolgher, C., Lamano, L., Lucioli, T., Tomasuolo, E., Rinaldi, P., y Pavani, F. (2023). The effect of face masks on sign language comprehension: performance and metacognitive dimensions. Consciousness and Cognition, 109:103490.

Gómez-Terán, A., Sánchez, B. P., y Pinto, D. (2020). Postura esquelética de la mano para representar las configuraciones dactilológicas de la lengua de señas mexicana en un entorno virtual. Res. Comput. Sci., 149(8):227–240.

Jeong, J. y Choi, H. S. (2021). Sudden sensorineural hearing loss after covid-19 vaccination. International Journal of Infectious Diseases, 113:341–343.

Jimenez, J., Martin, A., Uc, V., y Espinosa, A. (2017). Mexican sign language alphanumerical gestures recognition using 3d haar-like features. IEEE Latin America Transactions, 15(10):2000–2005.

Kothadiya, D., Bhatt, C., Sapariya, K., Patel, K., Gil-González, A.-B., y Corchado, J. M. (2022). Deepsign: Sign language detection and recognition using deep learning. Electronics, 11(11).

Kwon, Y. y Choi, W. (2023). Yolo-label. https://github.com/krontzo/label_qt6.

Lau, W. K., Chalupny, J., Grote, K., y Huckauf, A. (2022). How sign language expertise can influence the effects of face masks on non-linguistic characteristics. Cognitive Research: Principles and Implications, 7(1):53.

Martínez-Gutiérrez, M. E., Rojano-Cáceres, J. R., Benítez-Guerrero, E., y Sánchez-Barrera, H. E. (2019). Data acquisition software for sign language recognition. Res. Comput. Sci., 148(3):205–211.

Martinez-Seis, B., Pichardo-Lagunas, O., Rodriguez-Aguilar, E., y Saucedo-Diaz, E.-R. (2019). Identification of static and dynamic signs of the mexican sign language alphabet for smartphones using deep learning and image processing. Res. Comput. Sci., 148(11):199–211.

Mejía-Peréz, K., Córdova-Esparza, D.-M., Terven, J., Herrera-Navarro, A.-M., García-Ramírez, T., y Ramírez-Pedraza, A. (2022). Automatic recognition of mexican sign language using a depth camera and recurrent neural networks. Applied Sciences, 12(11).

Nájera, L. O. R., Sánchez, M. L., Serna, J. G. G., Tapia, R. P., y Llanes, J. Y. A. (2016). Recognition of mexican sign language through the leap motion controller. En Proceedings of the International Conference on Scientific

Computing (CSC), pp. 147–151. The Steering Committee of The World Congress in Computer Science, Computer Engineering and Applied Computing (WorldComp).

Rao, G. A., Syamala, K., Kishore, P. V. V., y Sastry, A. S. C. S. (2018). Deep convolutional neural networks for sign language recognition. En 2018 Conference on Signal Processing And Communication Engineering Systems (SPACES), pp. 194–197.

Ricciardiello, F., Pisani, D., Viola, P., Cristiano, E., Scarpa, A., Giannone, A., Longo, G., Russo, G., Bocchetti, M., Coppola, C., Perrella, M., Oliva, F., y Chiarella, G. (2021). Sudden sensorineural hearing loss in mild covid-19: Case series and analysis of the literature. Audiology Research, 11(3):313–326.

Rios-Figueroa, H. V., Sánchez-García, A. J., Sosa-Jiménez, C. O., y Solís-González-Cosío, A. L. (2022). Use of spherical and cartesian features for learning and recognition of the static mexican sign language alphabet. Mathematics, 10(16).

Rohmer, E., Singh, S. P. N., y Freese, M. (2013). V-rep: A versatile and scalable robot simulation framework. En 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1321–1326.

Serafín de Fleischmann, M. E. y González Pérez, R. (2011). Manos con voz: diccionario de lengua de señas mexicana. Consejo Nacional para Prevenir la Discriminación.

Solís, F., Martínez, D., Espinosa, O., y Toxqui, C. (2016). Automatic Mexican sign language and digits recognition using normalized central moments. En Tescher, A. G., editor, Applications of Digital Image Processing XXXIX, volumen 9971, p. 997103. International Society for Optics and Photonics, SPIE.

Solís, F., Martínez, D., y Espinoza, O. (2016). Automatic mexican sign language recognition using normalized moments and artificial neural networks. Engineering, 8(10):733–740.

Solís, F., Toxqui, C., y Martínez, D. (2015). Mexican sign language recognition using jacobi-fourier moments. Engineering, 7(10):700–705.

Solís-V, J.-F., Toxqui-Quitl, C., Martínez-Martínez, D., y Margarita, H.-G. (2014). Mexican sign language recognition using normalized moments and artificial neural networks. En Optics and Photonics for Information Processing VIII, volumen 9216, pp. 316–320. SPIE.

Sosa-Jiménez, C. O., Ríos-Figueroa, H. V., Rechy-Ramírez, E. J., Marin-Hernandez, A., y González-Cosío, A. L. S. (2017). Real-time mexican sign language recognition. En 2017 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC), pp. 1–6.

Sriwijitalai, W. y Wiwanitkit, V. (2020). Hearing loss and covid-19: A note. American Journal of Otolaryngology, 41(3):102473.

Trujillo-Romero, F. y Caballero-Morales, S.-O. (2012). Towards the development of a mexican speech-to-sign-language translator for the deaf community. Acta Universitaria, 22:83–89.

Trujillo-Romero, F. y García-Bautista, G. (2023). Mexican Sign Language corpus: Towards an automatic translator. ACM Trans. Asian Low-Resour. Lang. Inf. Process. Just Accepted.

Trujillo-Romero, F. y García Bautista, G. (2021). Reconocimiento de palabras de la Lengua de Señas Mexicana utilizando información RGB-D. ReCIBE, Revista electrónica de Computación, Informática, Biomédica y Electrónica, 10(2):C2–23.

Xia, K., Lu,W., Fan, H., y Zhao, Q. (2022). A sign language recognition system applied to deaf-mute medical consultation. Sensors, 22(23).