The Canadian 48-year-old doesn't feel like his $14B business portfolio has been badly impacted by the substantial backlash ...