In a competing risks setting, the cumulative incidence of an event of interest describes the absolute risk for this event as a function of time. For regression analysis, one can either choose to model all competing events by separate cause-specific hazard models or directly model the association between covariates and the cumulative incidence of one of the events. With a suitable link function, direct regression models allow for a straightforward interpretation of covariate effects on the cumulative incidence. In practice, where data can be right-censored, these regression models are implemented using a pseudo-value approach. For a grid of time points, the possibly unobserved binary event status is replaced by a jackknife pseudo-value based on the Aalen-Johansen method. We combine a stagewise regression technique with the pseudo-value approach to provide variable selection while allowing for time-varying effects. This is implemented by coupling variable selection between the grid times, but determining estimates separately. The effect estimates are regularized to also allow for model fitting with a low to moderate number of observations. This technique is illustrated in an application using clinical cancer registry data from hepatocellular carcinoma patients. The results are contrasted with traditional hazard-based modeling. In addition to a more straightforward interpretation, when using the proposed technique, the identification of time-varying effect patterns on the cumulative incidence is seen to be feasible with a moderate number of observations.