If you feel it would be awkward and uncomfortable for any of you, then you could visit your brother at the Chapel of Rest and say your goodbyes to him there. Plus it is private, personal and a time you will always remember. Whereas the funeral you might feel intimidated and out of sorts being there. I had people at my dad's funeral who I'd rather not be there but I know that my dad wouldn't have objected so I let it go and just did my own thing, paying my respects in my own way, as I was his next of kin and had organised it. Do what you feel comfortable with. You could find a childhood picture of him, when things were better between you and place it on a windowsill and put some flowers and a candle next to it and sit and think as you light and burn it down a bit on the day instead of attending. You have to be ok with your conscience, so do it just the way you want. Thinking of you, take care.