Reported incidence of the zoonotic malaria Plasmodium knowlesi has markedly increased across Southeast Asia and threatens malaria elimination. Nonzoonotic transmission of P. knowlesi has been experimentally demonstrated, but it remains unknown whether nonzoonotic transmission is contributing to increases in P. knowlesi cases. Here, we adapt model-based inference methods to estimate R_C, individual case reproductive numbers, for P. knowlesi, P. falciparum and P. vivax human cases in Malaysia from 2012–2020 (n = 32,635). Best fitting models for P. knowlesi showed subcritical transmission (R_C < 1) consistent with a large reservoir of unobserved infection sources, indicating P. knowlesi remains a primarily zoonotic infection. In contrast, sustained transmission (R_C > 1) was estimated historically for P. falciparum and P. vivax, with declines in R_C estimates observed over time consistent with local elimination. Together, this suggests sustained nonzoonotic P. knowlesi transmission is highly unlikely and that new approaches are urgently needed to control spillover risks.