Objective This study aimed to estimate the impact of high physical work demands on expected labor market affiliation (ELMA) among men and women of different ages in the general working population. Methods After participating in the Danish Work Environment and Health study (2012, 2014, and/or 2016), 46 169 employees were followed for two years in national registers. Using multi-state modeling, taking all day-to-day transition probabilities of labor market affiliation into account (work, unemployment, sickness absence, temporary out, and permanently out), and performing multilevel adjustment, we estimated the prospective association between physical work demands (ergonomic index including 7 factors) and ELMA. Results During 104 896 person-years of follow-up, we identified of 439 045 transitions. Using low physical work demands as reference, higher physical work demands were associated with fewer days of active work (2–35 days) during 730 days of follow-up, and more days of sickness absence (4–26 days) and unemployment (ranging 1-9 days) among men and women of aged 40–49 and 50–64 years. Among men and women aged 18–39 years, high physical work demands only had minor and inconsistent impact on ELMA. Conclusions Analyzing multiple and highly detailed patterns of transition probabilities concerning labor market affiliation, we showed that reducing physical work demands is likely to increase the active working time and pre-vent high societal cost of sickness absence and unemployment, especially among middle-aged and older workers.