Moreh

Moreh develops full-stack inference software that optimizes large language model (LLM) inference performance across heterogeneous AI accelerators, including AMD GPUs, Tenstorrent chips, and NVIDIA GPUs. The company's MoAI Inference Framework provides end-to-end inference stack optimization with intelligent workload distribution, auto-scaling, SLO-driven optimization, and KV cache management across mixed-vendor clusters. Key products include Moreh vLLM for AMD (a drop-in vLLM replacement achieving up to 2x higher throughput on AMD GPUs), Moreh vLLM for Tenstorrent, MoAI Performance Gateway for cross-vendor GPU workload routing, and MoAI Fabric for software-defined cross-vendor GPU memory fabric enabling KV cache transfer. Moreh enables cross-vendor disaggregated inference, unifying GPUs across vendors, architectures, and generations into a single inference cluster to maximize efficiency and reduce infrastructure costs. Benchmarks include 1.68x throughput vs ROCm vLLM for DeepSeek R1, 20,000+ tokens/sec per node on MI300X clusters, and 1.7x improvement with NVIDIA+AMD prefill-decode disaggregation. Founded in 2020 and headquartered in Seoul, South Korea, Moreh has raised approximately $30 million at the Series B stage from investors including KT, AMD, Smile Gate Investment, and Forest Partners.

AI Infrastructure > Enabling Technologies > Software > Modelling & Simulation