🏆 2023 KNU K-Digital Platform Hackathon – Excellent Award(2nd!)
I participated in the 2023 KNU K-Digital Platform Hackathon and received the
Excellent Award for proposing and developing an AI-based sign language interpretation service titled “Hand in Hand.”
This project was highly evaluated for addressing a real-world social problem—the communication barrier between hearing-impaired and non-hearing individuals—through practical AI technology.
📌 Problem Statement
The Daegu–Gyeongbuk region has a relatively high proportion of hearing-impaired individuals, yet technological support for everyday communication remains limited.
Hearing-impaired people can fully participate in society if communication barriers are removed; however, the availability of sign language interpreters and assistive systems is insufficient in practice.
Our team aimed to address this gap by proposing an AI-driven, bidirectional communication service that supports both hearing-impaired and non-hearing users.
💡 Proposed Solution
The proposed system supports communication in both directions:
1. Hearing-impaired → Non-hearing
- Input: Sign language video
- Extraction of key motion points (keypoints) from hand and facial movements
- Interpretation of sign language sequences into natural language text using deep learning models
2. Non-hearing → Hearing-impaired
- Input: Spoken language
- Conversion of speech into text using Speech-to-Text (STT) technology
- Real-time textual delivery to hearing-impaired users
This design enables seamless, two-way communication within a single integrated service.
🛠️ Technical Overview (High-Level)
- Sign language video analysis based on keypoint extraction
- Deep learning–based temporal sequence modeling
- Speech-to-Text (STT) for spoken language conversion
- A web-based prototype capable of real-time demonstration
Detailed model architectures and training strategies are documented in the official Hackathon presentation materials.
🎬 Results and Achievements
- Successfully demonstrated real-time sign language interpretation and speech transcription
- Integrated both communication directions into a single service workflow
- Recognized for technical feasibility, social impact, and practical applicability
- Awarded the Excellent Award at the 2023 KNU K-Digital Platform Hackathon
🌱 Expected Impact
- Improved communication efficiency in public institutions and customer service environments
- Expanded social and employment opportunities for hearing-impaired individuals
- Demonstration of AI as a tool for social problem solving, beyond purely technical performance
Award
- 2023 KNU K-Digital Platform Hackathon
- Excellent Award
