I somehow doubt that Shuuji and Irie can move as effortlessly on sand as they normally can on a court.
I'm also unsure if Mu, KnA, or Phantom would work nearly as well with a volleyball.
While Irie should be able to stand up to Shuuji and Sanada, the manga shows that Niou's a bit outmatched by them. So, overall, even though Mu isn't as deadly on a volleyball court, given that it's a smaller court, I'm still going to favor Shuuji/Sanada, but only because Niou is the weakest player here. Shuuji and Sanada are good enough to focus their attacks on/at Niou, and Irie can't defend everything on the court, even if he has no regular blindspots (according to KnS). If Sanada and Akutsu, two players with better footspeed, and you're slower on sand than on a court, even though the beach volleyball court is smaller, can't touch Mu, I doubt Niou can.
All of that said, it really comes down to the HSers. It's hard to say how much Shuuji or Irie would actually care, because it's merely beach volleyball, not tennis. Even when they play tennis, they both screw around. Their titles are the Star Performer and the Pierrot, respectively, which is reflected in their play on the court. However, Shuuji seems to refuse to lose, so even if they both clowned around at the beginning, I think that it would become serious by the end. I'll favor Shuuji or Shuuji/Sanada in a matchup against Irie, even though Mu probably isn't as effective with a volleyball. A lot of techniques probably only work with a racket.
Given that this is beach volleyball, it could really go either way. If Shuuji sloughs the whole thing off because it's not tennis, well, then Shuuji/Sanada would lose.