Lord Kinnock, Labour's leader from 1983 to 1992, said he was "thrilled to bits".
"We had our midnight conversation, not the first in our lives it has to be said, and usually that was because he came home late.
"It was a delight."
"Now as secretary of state he'll be in a position position to directly influence UK affairs in a way that benefits our country," he added.
Stephen Kinnock pledged to champion Wales within government.
He said he would work across governments to deliver for Welsh communities and build "a strong Wales within a strong United Kingdom".
Jo Stevens had faced criticism from some Welsh Labour politicians for not supporting more powers for the Welsh government, such as for policing, with one former minister describing her as "anti-devolution".
But she said Wales saw its biggest budget settlement for public services in the history of devolution while she was in office.
Stevens added: "Wales has seen the biggest budget settlement for our public services in the history of devolution, our steel industry saved and transformed for the future, the largest public investment in Wales in history through new nuclear at Wylfa, a generational commitment of £14bn for our Welsh railways, the best local growth fund settlement of any of the UK nations and so much more.
"I wish the new prime minister and my successor the very best in their new roles and know that they will do a great job, alongside our new Welsh Labour leader Ken Skates, in delivering for the people of Wales."
Stevens said former prime minister Sir Keir Starmer "leaves with the country in a much better place than he found it".
