IPAdapterAdvanced Error(s) in loading state_dict for ImageProjModel: size mismatch for proj.weight: copying a param with shape torch.Size([3072, 1024]) from checkpoint, the shape in current model is torch.Size([3072, 1280]).
Your comparison between IC-light and IP-Adapter is really a good idea. I have the feeling that you have more control of the final result with IP-Adapter in selecting a base image. With IC-light you always have a quite heavy color shift. Is the mask still playing a role if you are using IP-Adapter?