Loretta Young had worked with many of these people in other hit films.
Though you say she was annoyed with Cary's on-set antics during the making of THE BISHOP'S WIFE, they had previously costarred together in the precode drama BORN TO BE BAD. She did several films with David Niven (five of them between 1938 and 1947). And she worked with Elsa Lanchester in another feel-good drama from around this time, COME TO THE STABLE. Incidentally, Koster was also the director of COME TO THE STABLE.
This was made during the period in which Koster was in-between studios. He had finished a lengthy run at Universal, mostly directing Deanna Durbin musicals. He would head to MGM for awhile to work with musical producer Joe Pasternak, but then eventually settle in at Fox. And you did mention some of the Fox classics he helmed in the 50s.
I think one of the reasons Cary was upset about not being able to perform the ice-skating scene without a double is because he was very athletic. Also, he had begun his career doing trapeze work in a traveling show back in his native Britain. So he liked showing off his skills this way. My guess is Cary figured he could learn the moves on ice, and likely he could have, but probably Goldwyn objected to this for insurance reasons. He did not need an irreplaceable leading man getting injured and halting production. As you indicated earlier in your review, Goldwyn had already spent a lot of money when he switched directors and recast some of the main roles. And he could not afford any more cost overruns.
Plus I think it's fair to say that Goldwyn and Cary were embroiled in a power struggle. This is the only time Cary worked for Goldwyn.