Many discussions/arguments on this effect. Some say the ":" means the summoner's enmity lowers when they have a summon. Some say it's the summons enmity that's lowered. I think it's the summons hate that is lowered. I'm not saying it is this is just my opinion. I'm sure if it was the smn's hate that was decreased while they have a summon out it would say
"With Summon: Enmity -etc."
or something like that. And there's 2 other piece's of the AF which say "Enmity -etc" which are clearly for the smn. If it was for the smn's hate that would make enmity -8 in total (2 pieces with "Enmity-2", 2 with "Avatar:Enmity-2") and that would be really good and knowing SE they'll want to keep a balance and not make anything too great. Just my thoughts don't take my word i'm not a smn.