If your speedo cable is a direct drive to the back of the speedo from the gearbox, it might be the torsion effect. As you speed up the cable twists until fully tensioned, then as you brake it unwinds.
Happened to my speedo, took out the cable , cleaned & lubricated it - now OK.