In this study, we estimate the weather-related increases in ambient air pollution (CO, NO2, SO2, O3 and PM2.5) in Brazil between 2003 and 2018. The impact of long-term weather changes on each air pollutant was defined as “weather penalty”. Overall, ambient air pollution levels in Brazil during the period 2003–2018 have decreased in most of the Brazilian regions. We estimated significant trends in meteorological variables, indicating an increase in temperature, relative humidity, and wind speed in all Brazilian regions over the 16-study period. Our findings suggest that PM2.5 was the pollutant most impacted by weather changes. For the 16-year period of analysis, we estimated a weather penalty ranging from 1.58 μg/m−3 (CI 95%:1.25; 1.91) to 0.41 μg/m−3 (CI 95%:0.28; 0.53) among the different Brazilian regions. If weather parameters had remained constant, PM2.5 would have decreased by 1.10 μg/m3 (95%CI: 0.74; 1.46) in the South and by 2.25 μg/m3 (95%CI: 2.72; 1.79) in the Midwest. Over the 16-year study period, the weather impact on PM2.5 in Brazil was associated with over 6500 excess deaths.