0x80070002 when adding web part using Powershell

I was adding a webpart using this powershell:
$webpart = New-Object Microsoft.SharePoint.WebPartPages.SilverlightWebPart
$wpm = $web.GetLimitedWebPartManager($page.Uri.OriginalString, "Shared")
$wpm.AddWebPart($webpart, 'Header', 0)

... and got this error:

Exception calling "AddWebPart" with "3" argument(s): "<nativehr>0x80070002</nativehr><nativestack></nativestack>"
At line:1 char:16
+ $wpm.AddWebPart <<<< ($webpart, 'Header', 0)
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException


It turns out that my $page was the wrong one which was not checked out nor editable at the time.