Pawlonia tomentosa, perhaps.

looks like some kind of poplar

From your description, sounds like a female Paper Mulberry (Broussonetia papyrifera). Leaves are variable but the juvenile form resemble those in your photo.

Here is a link that might be useful: Paper Mulberry (Broussonetia papyrifera

I'm with floral_UK, Paulownia species.

I can only see one photo.

What about the flowers,
if they are small, brown... that would not fit for a Paulownia

The rest does though

Further down is a pic of a shoot/ cane

Here is a link that might be useful: Young Paulownia t

I missed the reference to fuzzy brown clusters. That doesn't fit Paulownia flowers but it does fit Paulownia flower buds. If the OP coppices the plant every year, which s/he says s/he does, possibly it never gets a chance fully to develop flowers?

Here is a link that might be useful: {{gwi:179785}}

The buds look familiar. Perhaps this year I will just let it grow and see what happens. I have bnever seen flowers on it. What about the speckled, hollow branches? Anyone ever seen that?

Looks like Paulownia to me. An interesting but invasive tree, named after Princess Anna Paulovna of the old imperial Russian Court.

Was apparent it was a Paulownia just from the leves and now there is a confirming stem also. If you direct a shower of water at the soft, hairy leaves they will appear as though emitting detergent.

They do this when fully developed, anyway.