Hacking SPRoleDefinition SPRoleType in the database

If you are a Site Collection Administrator, SharePoint allows you to go to a Site Collection, Site Actions->Site Permissions->Permission Levels, then edit a permission level and delete it.

Some predefined Permission Levels (SPRoleDefinition) have a Type (SPRoleType) defined. This is not visible via the UI. However you can get this via Powershell as follows:

$web = Get-SPWeb your-web-url
$web.RoleDefinitions | Format-Table Id, Name, Type, Hidden -AutoSize

This gives output like the following:

        Id Name                       Type Hidden
        -- ----                       ---- ------
1073741829 Full Control      Administrator  False
1073741828 Design              WebDesigner  False
1073741827 Contribute          Contributor  False
1073741826 Read                     Reader  False
1073741825 Limited Access            Guest   True

Now, if you delete one of these such as the Contribute and then decide to recreate it, the type is now "None". There is no way to make the Type "Contributor".

This is all fine, but if you have some code that calls Microsoft.SharePoint.SPRoleDefinitionCollection.GetByType then your code is now broken.

To "resolve" (using the term very loosely as I do not know if there are any implications), I was able to restore this by hacking the database directly.

Essentially, I looked into the Roles table in the WSS_Content database, filtered on the WebId and RoleId and set the Type column directly in the database. Is it a good idea? Who knows? It seemed to work for me, but I do not know what I may have broken as a result of this.

Automatically capturing matching Failed Request Trace Logs

I was trying to troubleshoot a particularly challenging user identity issue with SharePoint and thought about using Failed Request Tracing which I will write about some other time.

While doing so, I wanted a way to grab the relevant fr*.xml files. I thought that it would streamline things if I could watch for these log files as they are created, run some filter against them and, if matched, copy the file to another folder (hopefully before the log rolls over).

Then I came across this post regarding "check folder for new files": https://social.technet.microsoft.com/Forums/scriptcenter/en-US/c75c7bbd-4e32-428a-b3dc-815d5c42fd36/powershell-check-folder-for-new-files
I adapted it to my purpose:

$destination = 'c:\Inetpub\Logs\FailedReqLogFiles\Temp'
$folder = 'C:\Inetpub\logs\FailedReqLogFiles\W3SVC2'

$filter = 'fr*.xml'
$fsw = New-Object IO.FileSystemWatcher $folder, $filter -Property @{
  IncludeSubdirectories = $false
  NotifyFilter = [IO.NotifyFilters]'FileName, LastWrite'
}

$onCreated = Register-ObjectEvent $fsw Created -SourceIdentifier FileCreated -Action {
  $path = $Event.SourceEventArgs.FullPath
  $name = $Event.SourceEventArgs.Name
  $changeType = $Event.SourceEventArgs.ChangeType
  $timeStamp = $Event.TimeGenerated
  $frlogfile = Get-Content $path
  $xml = [xml]$frlogfile
  $url = $xml.failedRequest.url
  Write-Host "The file $($name) was $($changeType) at $($timeStamp) with URL $($url)"
  if (***my matching conditions***) {
    Copy-Item $path -Destination $destination -Force -Verbose
  }
}

I definitely need to read up more on Get-Event, Get-EventSubscriber, Register-*Event cmdlets as well as the IO.FileSystemWatcher class. I think I could do some pretty cool stuff with them.

Amazon AWS value cannot be null, parameter name awsAccessKeyId

Today, none of my AWS Powershell scripts would work any more. They were all returning
... : Value cannot be null.
Parameter name: awsAccessKeyId

Coincidentally, my Visual Studio would keep on crashing upon startup. I eventually dug into the crash logs and realized that it had to do with the Amazon plugin as well.

Initially I thought it had to do with expired credentials or even the Windows Patch Tuesday.

However, I ended up reinstalling AWS Powershell (and with a much newer version) and the problem went away.

Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information

I have been using the following Powershell to browse around some assemblies
$assembly = [Reflection.Assembly]::Load('myassembly')
$assembly.GetTypes()

Sometimes, I get the following exception:
Exception calling "GetTypes" with "0" argument(s): "Unable to load one or more of the requested types. Retrieve the LoaderExceptions property for more information."
At line:1 char:19
+ $assembly.GetTypes <<<< ()
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : DotNetMethodException


To get at the actual error, I use the following:
$x = $Error[0]
$x.Exception.GetBaseException().LoaderExceptions

Then all I had to do was to load the assemblies that were listed and my initial Load() now works.

The query cannot be completed because the number of lists in the query exceeded the allowable limit.

I have recently come across an error where our code is calling GetSiteData with a query and was returning this error:

The query cannot be completed because the number of lists in the query exceeded the allowable limit. For better results, limit the scope of the query to the current site or list or use a custom column index to help reduce the number of lists.

I then built the following powershell to mimic our code in order test it:

$web = Get-SPWeb some-url

$query = New-Object Microsoft.SharePoint.SPSiteDataQuery

$query.Webs = '<Webs Scope="Recursive">'

$query.Query = "<Where><And><Eq><FieldRef Name='_ModerationStatus' /><Value Type='Number'>0</Value></Eq><BeginsWith><FieldRef ID='{03e45e84-1992-4d42-9116-26f756012634}' /><Value Type='Text'>0x0110</Value></BeginsWith></And></Where><OrderBy><FieldRef Name='PublishedDate' Ascending='FALSE' /></OrderBy>"

$query.ViewFields = '<FieldRef Name="_ModerationStatus"/><FieldRef Name="Title"/><FieldRef Name="ID" /><FieldRef Name="Permalink"/><FieldRef Name="PublishedDate"/><FieldRef Name="Body"/><FieldRef Name="NumComments"/><FieldRef Name="Author"/><FieldRef Name="ContentType"/><FieldRef Name="ContentTypeId"/>'

$query.Lists = "<Lists BaseType='0'/>"

$query.RowLimit = [uint32]::MaxValue

$web.GetSiteData($query)

From what I have read about the call to GetSiteData, when SharePoint performs the query, it takes into consideration the lists that match the criteria passed in the query. It then returns a DataTable with a limit of RowLimit rows. However, since we have not set a MaxListLimit the query simply fails at 1000 lists (instead of giving you the first 1000). There is a parameter we can change in the code $query.Lists = "<Lists BaseType='0' MaxListLimit='0'>" which will allow the query to consider over 1000 lists, however, I am not sure about the performance impact.

References:
SPWeb.GetSiteData method
SPSiteDataQuery.Lists property - see description of MaxListLimit

SharePoint picture library not generating thumbnail and websize images

I recently came across a problem in a customer's site where the thumbnail and websize version of an image was not being generated. Typically when you upload an image to a picture library, a thumbnail and a websize version of that image are generated by SharePoint and saved into the /_t/ and /_w/ subfolders. For example, if upload an image to /Shared Pictures/image.jpg, then SharePoint also automatically generates /Shared Pictures/_t/image_jpg.jpg and /Shared Pictures/_w/image_jpg.jpg

For some reason, this was not happening. When you view the picture library with the default view, all the pictures were showing a blank image (since they use the thumbnail version).

Reaching out to Microsoft Support, it turns out that for these libraries, the parent SPWeb had the ParserEnabled property set to false.

The MSDN description of the SPWeb.ParserEnabled Property is quite useless.

Fortunately, I found a blog discussing this: http://neels-indites.blogspot.ca/2012/06/sharepoint-spweb-property-parserenabled.html. According to this blog, having ParserEnabled=false will cause the following:

• Cannot search in document library using Office document properties
• When you upload an image file, the thumbnail and websize images will not be generated
• When you save a list template, you will not see it in the list template gallery.

Lists.asmx GetListCollections and GetListItems example

I recently had to get all items of a list as well as all lists in a SPWeb. This involved calling Lists.asmx. For the service description of Lists.asmx just go to /_vti_bin/lists.asmx

Get all lists for some SPWeb

$webUrl = 'some-web-url'

$credential = Get-Credential -Message "Enter your login for $($webUrl)"

$uri = [uri]"$($webUrl)/_vti_bin/lists.asmx"

$contentType = 'text/xml'
$bodyString = @"
<?xml version="1.0" encoding="utf-8"?>
<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
<soap12:Body>
<GetListCollection xmlns="http://schemas.microsoft.com/sharepoint/soap/" />
</soap12:Body>
</soap12:Envelope>
"@

$result = Invoke-WebRequest -Credential $credential -Method Post -Uri $uri -Body $bodyString -ContentType $contentType
$xml = [xml]$result.Content
$xml.Envelope.Body.GetListCollectionResponse.GetListCollectionResult.Lists.List

Get all list items for some list

$webUrl = 'some-web-url'
$listName = 'some-list-name'

$credential = Get-Credential -Message "Enter your login for $($webUrl)"

$uri = [uri]"$($webUrl)/_vti_bin/lists.asmx"

$contentType = 'text/xml'
$bodyString = @"
<?xml version="1.0" encoding="utf-8"?>
<soap12:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">
<soap12:Body>
<GetListItems xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<listName>$($listName)</listName>
<viewName></viewName>
<rowLimit>5000</rowLimit>
<query>
<Query xmlns="">
<OrderBy> <FieldRef Name="ContentType" /> </OrderBy>
</Query>
</query>
<viewFields><ViewFields xmlns="" /></viewFields>
<queryOptions>
<QueryOptions xmlns="http://schemas.microsoft.com/sharepoint/soap/">
<IncludeMandatoryColumns>FALSE</IncludeMandatoryColumns>
<IncludeAttachmentUrls>TRUE</IncludeAttachmentUrls>
<ViewAttributes Scope='RecursiveAll' IncludeRootFolder='True' />
<DateInUtc>TRUE</DateInUtc>
</QueryOptions>
</queryOptions>
</GetListItems>
</soap12:Body>
</soap12:Envelope>
"@

$result = Invoke-WebRequest -Credential $credential -Method Post -Uri $uri -Body $bodyString -ContentType $contentType
$xml = [xml]$result.Content
$xml.Envelope.Body.GetListItemsResponse.GetListItemsResult.listitems.data.ItemCount
$xml.Envelope.Body.GetListItemsResponse.GetListItemsResult.listitems.data.row | Format-Table ows_LinkFilename, ows_Author, ows_Created, ows_Modified -AutoSize

I could probably use New-WebServiceProxy and skip all the SOAP envelope stuff. However, this better mimics the code that I was trying to troubleshoot.