You can assume the gun was in his hand when the off-duty cop first arrived on the scene. That would make perfect sense considering he was coming to the aid of a fellow officer. However, since he was stopped by the first on-scene officer who quickly identified him as an off-duty officer and, thus, assessed that he wasn't a threat AND that his assistance wasn't required, I think it's reasonable to assume that the off-duty officer put his gun away (i.e., in his waist-band) once he was effectively given the "ALL CLEAR".
I've re-read the OP article just to re-acquaint myself with the basic facts as outlined leading up to the off-duty officer being shot. Here's a recap from the article. Pay very close attention to what transpired in the second bullet point:
So, if the officers who were carrying out the arrest could take the time to see someone who is armed approaching them, give clear instructions to halt his approach, recognize that the man is not a threat and de-escalate the situation by giving further clear instructions, and thus, PERMISSION for the off-duty cop to approach by S-L-O-W-L-Y walking towards them, why the hell couldn't this other officer who arrived late to the scene not do the exact same thing?
Even if the off-duty officer did still have his gun in his hand, it's clear from the article that the arresting officers already had things well under control. Why couldn't the officer who arrived late to the scene do basic police work and at the very least simply ask his fellow officers, "Do you require assistance?" or at worse instruct the armed man to "HALT! Get on the ground (and put down your weapon)," like his fellow officers did before he arrived on the scene?
Why?
It's time you stopped defending this cop's improper actions and admit you got this one wrong.