It's certainly a denouncement (at least implicitly) of their policy of non-interference.
Fun fact: Ulfric and Delphine seem to know each other. During Season Unending, he asks her who Esbern is in a way that implies they are acquainted. Given that Delphine is the only other character to really call out the Greybeards for refusing to use their power to help, I just think that it's interesting. (Obviously, they couldn't have met until he had already left High Hrothgar, but they certainly seem to have similar ideas about the responsibility that comes with power)
Though that is literally just a gesture. I'm surprised this is even brought up with lines like this in the game.
Ulfric: "How'd I do?"
Galmar: "Eh, not so bad. Nice touch about the High King."
Ulfric: "Thank you, I thought so, too."
Galmar: "It's a foregone conclusion, you know."
Ulfric: "Oh, I know."
~~~
We'll wait for the Moot to name me High King. It'll be better for all that way. But, that doesn't mean I won't start acting like it.
Dude's not humble, he wants to be High King, and plans on it.
Right? Guy's got it goin' on XD. Now I can't help thinking about the marriage pizza from Stargate.
Seriously, though, I took it as more of a "with us or against us" statement/question. I could be wrong, of course. Ulfric turning the down the challenge would definitely work against him, though, and seems like it'd be highly publicized by Balgruff, if not the Empire.
But unless Durnehviir was summoned by Ulfric to learn the Soul Tear shout, it's very unlikely Ulfric could have killed him by shouting. And the only shout Ulfric uses in battle is Unrelenting Force(I witnessed this when killing Tullius). Seeing as he cut his training with the Greybeards short due to the Great War, I doubt he would've learned any other shouts, but it may be possible. Now, even taking that into account, the most he could do is use all three words of one shout, and that would be it, after that, swordplay would be the main deciding factor. On the subject of legality and morality, Ulfric sought to show the Jarls of Skyrim that Toryg was unfit to rule. So he issued a challenge, Toryg could have denied the challenge, but the Jarls would then convene the moot to decide on a new king anyway. And it's not like Toryg was oblivious to the fact that Ulfric could harness the power of the Thu'um. He used shouts in the Markarth incident after all. Now consider the fact that mages could have challenged the High-King as well. Would he have stood a chance then? More than likely not, the duel with Ulfric is no different, aside from the fact that he could only shout once before having to resort to his sword, whereas a mage could cast a multitude of spells to annihilate Toryg, even turning him to ash. So with all this taken into affect, Ulfric's challenge legitimizes itself.
That's a mighty big probably you've got in there. Still, that's a serious shortcoming of the Civil War narrative for me; no one talks about Torygg's rule and what it was like. A few simple lines of dialogue, even in passing, would have gone miles towards better justifying Ulfric's actions.