Situational awareness is key to everything. Staying alert and knowing what's happening where allows you to stay one step ahead of your opponents. Avoiding boarders is often the single most important factor in winning. Every time they send someone to board your ship, they make themselves more vulnerable and less able to repair their own ship. Don't let them near your ladders. If they do grab a ladder, make sure your blunderbuss is ready to blast them before they can get on deck. I always love to see a couple people jump off a ship to board at once. Simply leave them in the water and go on the offensive at once. Nine times out of ten you'll sink them.
Knowing your ship and being able to fluidly move between tasks allows you to stay on top of everything and succeed in battle. This is a skill that mainly comes from experience. This includes all ship-related tasks like repairing, bailing, managing the helm and sails, firing the cannons, etc. Speed and fluidity comes from practice. Accuracy on the cannons is also an art that improves with experience.
Keep your supplies full, both on your ship and on your person. Whenever you go below deck, restock your supplies and reload your gun. Same thing when on an opponent's ship.
Learn when to press the attack and when to back off. This is an art form in itself, but often decides who wins and who loses. The best PvP people don't just run up and start hacking. They get in a shot here, a shot there, and wear the opponent down until they have a clear advantage. The temptation is to go in for the quick kill, before they can eat fruit or get away, but sometimes that's a mistake. If the other person manages to recover long enough to pull out a blunderbuss, you could be in trouble. Take your time, keep your guard up, and make your shots count. The same applies to ship battles.
Never stop moving. Standing in one place usually guarantees a quick death. Don't just stand there facing your opponent swinging a cutlass. With the cutlass, you want to try to circle the enemy so you can get past his defenses while making yourself a moving target. In gun battles, always keep moving left and right, and try to be unpredictable in your movements. Jumping around like a nut looks crazy but it works.
Limit your opponent's mobility. This applies to both ship battles and personal combat, and goes hand in hand with keeping your own mobility open. In naval combat, try to force the enemy ship into tight, unwieldy situations. Lure them through the rocks, then turn and give them your broadside. If you're on a sloop vs. a brigantine or galleon, use your superior maneuverability to your advantage. Circle the enemy while staying out of their range of fire as much as possible. Use cursed cannonballs to keep them immobile. Anchor them if possible. In hand-to-hand combat, funnel your opponents into tight spaces while leaving yourself room to move. If they have to come through a doorway or up a ladder, you have a free shot. A blunderbuss or sword lunge against a trapped opponent can end the fight.
Use the environment to your advantage. There are lots of hiding places on galleons. You can fire and duck behind some barrels. You can jump off the quarter deck onto the railing below and zip through the captain's quarters. You can funnel your opponents through a small space and cut them down with your cutlass. If you have an eye of reach and they have a blunderbuss, try to force the battle to the water, where you have the advantage. If it's reversed and you have the blunderbuss, go for tighter quarters.
Use the right tools for the job. As mentioned above, blunderbusses are best for close quarters fighting, like aboard a ship (especially a sloop). The Eye of Reach gives you the advantage at long range. Pistols are great anywhere, but not as powerful. I personally think you should always have a sword at your side, but some pirates prefer two guns. Whatever the case, always try to have the right tool for the job at hand.
Breathe. It might seem obvious, but if you're like me, PvP can be pretty stressful at times. I hate conflict as a rule. It took me a long time to get to where I can relax and not get flustered during combat. My hands used to shake so badly I could barely handle the controls. Taking a deep breath, exhaling slowly, and getting centered really helps. It's just a game, after all. I spent many years practicing martial arts in real life. I was always fine in the training hall, but I'd get really flustered at tournaments. I had to apply the same techniques of breathing, relaxation, etc. to calm my nerves and find my groove. It worked and I usually did fine, winning most of my matches. This is no different. Any competition of this nature is a mental game above all. Self-mastery is path to victory.
Play chess. You don't have to literally play chess, but thinking of combat in Sea of Thieves more like a chess match than a brawl will enable you to go for the long win. Chess takes a bit of patience. The idea is to control the board and give yourself every advantage possible, until you have a clear road to victory. In Sea of Thieves, that means playing it smart and taking your time. Force your opponent to play the game by your rules, not theirs. Never make it easy for them and never give away a positional advantage. When you're being chased, you have a positional advantage. You have lots of options, lots of ways to attack. You can lead them into a situation that's more favorable for you, like through some rocks or into the fog. You can harass them with your muskets, keg them, board them, etc. You can fake them out by leading them one way, then suddenly pulling an anchor turn and giving them your guns (after getting the anchor up of course).
Use deception. There are countless ways to trick your opponent. What you choose will depend upon the circumstances and what you hope to accomplish. Savvy opponents will know most of the tricks and be watching for them, but it's often worth a try. Sending your ship sailing into the wind (with sails set to stupid) gives you time to try some shenanigans. You can drop yourself off at a fort and use the cannons there to blast your opponents. If you are trying to escape with loot and happen to have a rowboat, you can load your booty into the rowboat, pass behind a large rock or island, and detach the rowboat while your ship sails merrily away. This works best if you can quickly hide the rowboat behind an outcropping or something. Then when the ships are clear, you row row row your boat to the nearest outpost. If you're more of the combative type, the simple effectiveness of the gun powder play can never be overstated. Fast maneuvers like anchor-turns or harpoon pulls executed out of sight can really throw off an opponent and change the course of a battle (just be sure it's to your advantage). Sneaky pirates can disrupt the rhythm of a battle by suddenly hiding (sleeping) and waiting for the right moment to pounce. Ships that carry gunpowder barrels in the crow's nest are vulnerable to all kinds of explosive plays. This isn't even scratching the surface, really. The limit is your imagination.
Practice, practice, practice. Honing one's skills is always a good way to improve at PvP. Take out a ship and practice things like jumping sword lunges, blocking, dodging, shooting, maneuvering about the ship, etc. Try it on different ships, not just your preferred type. Fighting on a galleon is quite different from fighting on a brigantine or a sloop. Take time to work on your techniques outside of combat. It really helps.
This is longer than I expected it to be, but hopefully there are some useful tips buried in there somewhere.