Journalists unlucky enough to have spent a portion of our careers covering Labor factional politics are familiar with the dilemma David Marr faced when writing his excellent primer on Bill Shorten’s rise.

The dilemma is that, though fascinating to insiders, the grindings of Labor’s factional machine – at once impenetrable, distasteful and apparently crucial – are to outside observers dull to the point of stupor. But without understanding and accounting for the networks of influence and patronage that bind the union bosses, the branches (more accurately, the branch-stackers), the ethnic warlords and the parliamentarians, there is no explaining the Labor Party and how it identifies and promotes talent.

As Marr elegantly points out, Shorten’s rise through this system was both ruthless and relentless. In a series of ever more complex deals in 2005, he bent the Victorian party and all its parts to his will so as to feed his capacious ambition. It was no small undertaking.

As I remember his pre-selection – and I covered it closely for the Australian – he was at once the party’s most adept and most insecure manipulator. Almost everything he asked for, he got, but he still wanted more. Shorten wanted to make doubly, triply sure that he would gain pre-selection for his chosen safe seat, and then win it. The demands and deals he made in service of this ambition led to a cascade of other consequences that put several people out of their jobs and consumed the party for months, at both state and federal level. His ambition was not the only one being served by the massive ructions that shook the party that year, but somewhere in every local battle was Shorten, looking out for interests that invariably pointed back to himself. It didn’t seem so at the time, but in retrospect this was a huge, virtually unprecedented disruption to the Victorian Labor Party. 

Thanks to Tony Abbott and Dyson Heydon’s royal commission, we now know that factional numbers were not the only component of Shorten’s campaign. He was also raising funds by the tens of thousands of dollars or more, often from the employers he was simultaneously negotiating deals with. At least some of that money was directly used for his election campaign – particularly in the case of a campaign director funded by Unibuilt.

Shorten’s career has been defined by his collection and use of numbers, whether it be to secure his own seat, to turf first Kevin Rudd, then Julia Gillard, out of the Lodge, or to secure the party’s leadership from Anthony Albanese by sewing up the votes at the central level in the 2013 leadership contest after losing the popular ballot.

Otto von Bismarck famously compared sausage-making and law-making, saying that in neither case did we want to see the method of manufacture. It’s unlikely he had in mind the neighbourhood snag-fest that goes along with voting in Australia, but his analogy holds true here as well: it does not pay to look too closely at the constituent parts of what we’re biting into on election day – either literally or figuratively. 

Bill Shorten is the very embodiment of the Bismarckian conundrum. He’s a presentable leader emerging from an unpalatable process. He’s steeped in factional manipulation, but still carries something of an air of innocence about him. His number-crunching, the deals he’s made with the worst elements of his party’s machine, have at times represented the height of cynicism. But as Labor’s disabilities spokesperson, he was genuinely moved by the plight of the people he met, and was capable of turning that emotion into action, helping to craft the National Disability Insurance Scheme that will secure them a better deal.

Now Shorten has manoeuvred his way to being one step away from the prime ministership. A very big step nevertheless remains. Malcolm Turnbull presents a more significant challenge than Tony Abbott did – it’s a pity that the timing of Marr’s essay denied us his views on that head-to-head contest.

Shorten, meanwhile, has yet to attain mass political popularity. Perhaps that’s another result of his conundrum: nobody sees him as being particularly tough. It’s ironic that, for all the acuity and ruthlessness of Shorten’s rise through Labor, the perception that he’s not tough enough might prove to be his downfall. As Marr puts it: “He still wears the face his mother gave him, the face of a boy who wants to be liked. It’s a charming mask that hides too much for his own good. This man would be more respected if, like Hawke, Keating and Howard, he let us see the bastard that’s in there. Instead, the rough edges are politely hidden.”

One example of this perception issue is that in the narrative surrounding Tony Abbott’s fall, Shorten is not widely credited with the scalp – as Abbott was when he took down Turnbull, then Rudd (twice) and Gillard. Abbott himself, driven by his many flaws, is his own fourth victim, they say.

What all this means for Australian electoral politics it’s still a little too soon to say. Malcolm Turnbull’s rise has turned the spotlight back on Shorten, and not in a flattering way. The Age illustrator John Spooner depicted the Labor leader as a sitting duck, a large target painted on his downy chest, a gormless look about his beak. But Turnbull is just at the start of his honeymoon. That will inevitably sour. And as many a factional enemy has discovered over the years, it does not pay to underestimate Bill Shorten. His ability to do the numbers and his drive to power are profound. Marr has found a way, without once stupefying us, to remind us of that, and to suggest that we should not write off William Richard Shorten just yet.

Michael Bachelard is investigations editor at the Age and was Indonesia correspondent for the three years to January 2015. He is an award-winning reporter and the author of two books, the most recent being Behind the Exclusive Brethren.