MKD Releases KeuralMoE-14B, a Proprietary 14B-Parameter Bilingual MoE Language Model Trained from Scratch

MKD Co., Ltd. has unveiled Keural-MoE-14B, its in-house large language model (LLM). As MKD's first foundation model, this release is a major milestone toward securing proprietary AI technology and building a sovereign AI environment.

As the importance of AI technological sovereignty grows worldwide, nations and enterprises are increasingly seeking to secure their own AI capabilities rather than depending on external AI models. In line with this trend, MKD is developing its own AI models and platform to build an independent AI ecosystem.

Keural-MoE-14B is a Mixture-of-Experts (MoE) based language model with 14 billion parameters that supports both Korean and English. From the model architecture to the tokenizer, data pipeline, and training process, the core technologies were designed and developed in-house — making it a true foundation model rather than a fine-tuning of an external model.

In particular, Keural-MoE-14B is built on the KeuralMoECausalLM architecture designed by MKD. The model adopts an MoE structure utilizing 32 Transformer layers and 32 experts, activating only 4 experts per token to achieve an efficient computation structure. MKD also developed its own SentencePiece Unigram tokenizer with a vocabulary of 131,072, optimized for multilingual environments including Korean and English.

The training process was likewise built in-house, spanning data collection, preprocessing, tokenization, pre-training, and alignment. Using data from diverse domains such as web documents, Wikipedia, technical documents, scientific materials, and code, MKD constructed a training dataset of approximately 69.5 billion tokens and trained the model on this foundation.

Keural-MoE-14B is currently a research-and-development-stage model. Rather than competing head-to-head with the latest global commercial models, it focuses on securing proprietary AI technology and accumulating experience in foundation-model development. It is especially significant that MKD carried out the entire process — from model architecture design to building the training infrastructure — on its own, even within a constrained research environment.

As the only company in Busan researching and developing its own foundation model, MKD plans to continuously pursue improvements in inference performance, long-context support, and multimodal technology, beginning with this release.

Keural-MoE-14B is set to serve as a core model for MKD's AI platform, Keural. Building on it, MKD plans to establish a sovereign AI environment that domestic and global companies and institutions can trust and use.

huggingface, GitHub