He was(not really because he would die at the hands of Hulk, but because how destructive Hulk is that it threatens all of humanity)... but he didn't back down and kept fighting and keeping up with him the whole time, even throwing in the towel to be Hulk's stress toy and endured it until Hulk tired out and went back to being Bruce Banner. Supes could've killed him there... but he didn't and decided to help Bruce.
Superman might have thought he could’ve killed the Hulk by killing Banner, but he would have been in for a rude awakening. The fight would have been re-ignited right there.