He had been Robert's right-hand man since 2003 and a big part of the actor's recovery after his addiction struggles. Robert paid tribute to his friend on social media saying he was "a brother, my right hand man, an uncle to our kids and beloved by all who experienced his singular character and wit".
A host of actors including Mark Ruffalo and Chris Evans have also offered their condolences to his family.
Jimmy was just 52 years old.
Photo Credit: Instagram/robertdowneyjunior