I have run into a hand full of powder kegs in pvp without losing even against a larger ship, and have had a fair few times that spotting them has come in pretty handy.
That said im not for or against them being removed since they are pretty easily dodged when you get used to looking out for birds and flotsams even in a fight, and moments where you can jump in the sea to grab one and weaponize it are so far and few between. I can definitely understand not wanting them, just not really found too much problem from them other than when im goofing around with a friend, but when that happens its always a laugh for me.