Tim Wheeler's Blog

Azure Cloud Storage OutOfRange Error

2011-01-06T17:33:00.001-08:00

I'm having an interesting time learning Azure and have found various issues, including Visual Studio 2010 hanging when the project is bound to Visual SourceSafe, random hang on project build and the AspProviders not working with development storage.

Here's what I found with the AspProviders issue:

The AspProviders from the Azure Samples errored on start up with an OutOfRange error . The providers didn't seem to work with developer storage. The providers didn't allow access to the dev storage addresses, likely due to http instead of https. To fix this I created a helper class that would get either the developer storage or the https address.

I had problems with TableStorageRoleProvider, TableStorageSessionStateProvider, TableStorageProfileProvider and TableStorageMembershipProvider.

The error I received was:

Microsoft.WindowsAzure.StorageClient.StorageClientException: Exception of type 'Microsoft.WindowsAzure.StorageClient.StorageClientException' was thrown. ---> System.Data.Services.Client.DataServiceQueryException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code>OutOfRangeInput</code><message xml:lang="en-US">One of the request inputs is out of range.</message></error>

The client class I created:

Code Snippet
  /// <summary>
    /// Class to create <see cref="CloudTableClient"/> or <see cref="CloudBloblClient"/> or <see cref="CloudQueueClient"/>
    /// Works with developer storage at address "http://127.0.0.1" or https address.
    /// </summary>
    public static class StorageClient
    {
        
        public static CloudTableClient GetCloudTableClient(string endpoint, string accountName, string sharedKey)
        {
            CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
            return account.CreateCloudTableClient();
        }
        public static CloudTableClient GetCloudTableClient(string endpoint, StorageCredentialsAccountAndKey credentials)
        {
            CloudStorageAccount account = GetAccount(endpoint, credentials);
            return account.CreateCloudTableClient();
        }
 
        public static CloudBlobClient GetCloudBlobClient(string endpoint, string accountName, string sharedKey)
        {
            CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
            return account.CreateCloudBlobClient();
        }
        public static CloudBlobClient GetCloudBlobClient(string endpoint, StorageCredentialsAccountAndKey credentials)
        {
            CloudStorageAccount account = GetAccount(endpoint, credentials);
            return account.CreateCloudBlobClient();
        }
 
        public static CloudQueueClient GetCloudQueueClient(string endpoint, string accountName, string sharedKey)
        {
            CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
            return account.CreateCloudQueueClient();
        }
        public static CloudQueueClient GetCloudQueueClient(string endpoint, StorageCredentialsAccountAndKey credentials)
        {
            CloudStorageAccount account = GetAccount(endpoint, credentials);
            return account.CreateCloudQueueClient();
        }
        
        private static bool IsDevelopmentStorage(string endpoint)
        {
            const string devEndpoint = "http://127.0.0.1";
            return (endpoint != null && endpoint.Length >= devEndpoint.Length &&
                    devEndpoint.ToLower().Substring(0, devEndpoint.Length) == devEndpoint);
        }
        private static CloudStorageAccount GetAccount(string endpoint, StorageCredentialsAccountAndKey credentials)
        {
            CloudStorageAccount account = null;
            if (IsDevelopmentStorage(endpoint))
            {
                account = CloudStorageAccount.DevelopmentStorageAccount;
            }
            else
            {
                account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
            }
            return account;
        }
        private static CloudStorageAccount GetAccount(string endpoint, string accountName, string sharedKey)
        {
            CloudStorageAccount account = null;
            if (IsDevelopmentStorage(endpoint))
            {
                account = CloudStorageAccount.DevelopmentStorageAccount;
            }
            else
            {
                var credentials = new StorageCredentialsAccountAndKey(accountName, sharedKey);
                account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
            }
            return account;
        }
    }
 

I then changed the providers where they create the cloud client.

In the TableStorageRoleProvider I update the following line:

_tableStorage = new CloudTableClient(baseUri, info);

To:

_tableStorage = StorageClient.GetCloudTableClient(baseUri, accountName, sharedKey);

The other providers are similar.

That’s it!

The Heading “Libraries” automatically added through content import

2010-11-24T04:34:00.001-08:00

Problem

When exporting and importing a single Document Library using the Import-SPWeb command, the Heading "Libraries" is automatically added to the site navigation if it is not there. Exporting then re importing to the same site or another site always adds the heading. Seems to be no option to stop this behaviour.

Solution

As I don’t want the heading Libraries to be added I guess I can either delete them manually or build a PowerShell script that moves through all the SPWebs and deletes them. Not a great solution and this really should be controlled via parameters on Import-SPWeb. Hey Microsoft, can you sort that for me please? Thanks.

Content Migration and Taxonomy Fields

2010-11-17T04:06:00.001-08:00

I have been using Import-SPWeb and Export-SPWeb to migrate content from different environments and found an issue if the sites use Taxonomy Fields.

The problem in a nutshell

The content is exported from a site using the PowerShell command Export-SPWeb or with code. The site contains Taxonomy fields included in content types. The taxonomy fields are wired to a Managed Metadata Service Term Store. The taxonomy fields have id's to both a TaxonomyHiddenList at the root web level and to the Term stores themselves. These ID's are migrated along with the list. However in the next environment you try to import the content to the Managed Metadata Service has a different ID for the Term Store, and as such the taxonomy fields fail to work properly.

Note: I'm not sure if this is an issue if you use the default Managed Metadata Service, I am using a new one created from a PowerShell script.

Visible Issues

If the site was accessed as an anonymous user, some pages would error with:

The error message might as well just say “Something bad’s happened, but you have to figure out what. Good luck!” (Well, I could turn off custom errors but that would be too easy and harder to make fun of!)

So the Slightly more helpful are the trace logs which showed:

"Hidden list not found, creating new one"

"Creating taxonomy hidden list"

"Unknown SPRequest error occurred. More information: 0x80070005"

"System.ArgumentException: Value does not fall within the expected range."

Strangely, the first page would access correctly under anonymous and subsequent requests to other pages would fail. I have a theory about why but wild speculation is probably best left unsaid.

Other Symptoms

The publishing root web contains a list called "TaxonomyHiddenList". This list is used by the taxonomy fields to store terms in use for the site. After using Import-SPWeb many extra entries where added to this list with an empty title.

If you use SharePoint Manager (very useful BTW), you will see a bunch of entries with “(no title)”. As far as I can see this shouldn’t happen, normally when the taxonomy fields create these entries they include the label as the title.

The above screen shot of SharePoint Manager shows the bad entries. Notice as well that there are lots, 64 in this example. If this was working properly you should see the following:

In this screen shot the entries have a valid title. These values originally come from the terms in the term store. These entries where created by a powershell script and added to the term store, they are some of the ones required for a dublin core and AGLS metadata implementation. (Which is probably about as interesting as xml namespaces or writing documentation but I felt I had to explain them anyway)

The cause

If you look at the definition of the Microsoft.SharePoint.Taxonomy.TaxonomyField you will see a property called SspId. This property is the Guid of the Term Store that its connected to. After importing, this value points to the old term store. It also has a side effect of creating bad entries in the /Lists/TaxonomyHiddenList list and is likely the cause of the entries in the trace logs. The data in the field also points to the bad entry within that list rather than the entry it should point to. There is a good write up on how the entries are created here so I won't go into that part too much.

The fix

The fix for this is much simpler than tracking it down I assure you. After many frustrating hours and some stern words aimed directly at SharePoint I managed to coble together a script that suitably cleans my site collections after content migration.

The following is a PowerShell script that will clean up your publishing site. Use at your own risk. Backup your data, etc, etc! Wish you the best of luck! Like a Microsoft error message

Note: This script will utilise the field definitions in the root web. If you are migrating an entire web then you should fix your content types prior to running this script.

Code Snippet
 #Script: CleanTaxonomyFields.ps1
#Author: Tim Wheeler (http://tjwheeler.blogspot.com/)
#Created: 17/11/2010
#Notes:
#When content in imported into a new environment the Term Store ID is kept on the
#taxonomy fields.  This causes a problem as the new term store has a different id.
#These functions clean up the taxonomy list which has bad entries following an import
#and modify all the items and resets the correct values.
param 
    (    $siteCollectionUrl = (Read-Host "Please enter a site collection url") )
    
#region SharePoint Snappin Setup
$snapin="Microsoft.SharePoint.PowerShell"
if (get-pssnapin $snapin -ea "silentlycontinue") {
    write-host -f Green "PSsnapin $snapin is loaded"
}
else {
    if (get-pssnapin $snapin -registered -ea "silentlycontinue") {
        write-host -f Green "PSsnapin $snapin is registered"
        Add-PSSnapin $snapin
        write-host -f Green "PSsnapin $snapin is loaded"
    }
    else {
        write-host -f Red "PSSnapin $snapin not found"
    }
}
#endregion
    
#region Data Clean Up
 
function CleanTaxonomyList($site)
{
    Write-Host "Cleaning /Lists/TaxonomyHiddenList"
    $web = $site.RootWeb
      $taxlist =$web.GetList("Lists/TaxonomyHiddenList")
    for($count = $taxlist.Items.Count - 1; $count -ge 0; $count--)
    {
         $item = $taxlist.Items[$count]
        if([string]::IsNullOrEmpty($item.Title))
        {
            $item.Delete()
            if($?)
            {
                #Write-Host "Deleted Item with bad title"
            }
            else
            {
                Write-Host -ForegroundColor Red "Failed to deleted Item with bad title"
            }
        }
        
    }
}
#This function will reset the term store id if it is wrong, and will fix the WssId.
#If the correct term cannot be located, the default one will be used.
function ResetTaxonomyDefaults($site, [Microsoft.SharePoint.Publishing.PublishingWeb] $web)
{
    $txs = New-Object "Microsoft.SharePoint.Taxonomy.TaxonomySession" -ArgumentList $site
   $pages = $web.GetPublishingPages()
   foreach($page in $pages)
   {
        Write-Host -ForegroundColor Cyan "Checking publishing page " $page.Title
       if($page.ListItem.File.CheckOutStatus -ne "None")
        {
            $page.CheckIn("Checked in by data clean process");
        }
        $page.CheckOut();
       foreach ($field in $page.ListItem.Fields)
       {
           if($field.GetType().Name -eq "TaxonomyField")
           {
               $taxField = [Microsoft.SharePoint.Taxonomy.TaxonomyField] $field
                
                Write-Host "Found field to update:" $taxField.Title
                $currentValue = $page.ListItem.Properties[$taxField.InternalName]
                Write-Host "Current Value is" $currentValue
                $templateField = $page.ListItem.ParentList.ParentWeb.Site.RootWeb.Fields[$field.Id]
                $defaultValue = $templateField.DefaultValue
                $termStore = $txs.TermStores[$templateField.SspId]
                $termSet = $termStore.GetTermSet($templateField.TermSetId)
                if($taxField.SspId -ne $templateField.SspId)
                {
                    Write-Host "TaxField SspId is not correct, updating"
                    $taxField.SspId = $templateField.SspId
                    $taxField.Update()
                }
                if($taxField.TermSetId -ne $templateField.TermSetId)
                {
                    Write-Host "TaxField TermSetId is not correct, updating"
                    $taxField.TermSetId = $templateField.TermSetId
                    $taxField.Update()
                }
                $fieldValue = $templateField.GetFieldValue($currentValue)
                  if(($fieldValue.GetType().Name -eq "TaxonomyFieldValue") -and($fieldValue -eq $null -or [string]::IsNullOrEmpty($fieldValue.TermGuid)))
                {
                    $fieldValue = $templateField.GetFieldValue($defaultValue)
                }
                  if(($fieldValue.GetType().Name -ne "TaxonomyFieldValue") -and($fieldValue[0] -eq $null -or [string]::IsNullOrEmpty($fieldValue[0].TermGuid)))
                {
                    $fieldValue = $templateField.GetFieldValue($defaultValue)
                }
                if($fieldValue.GetType().Name -eq "TaxonomyFieldValue")
                {
                    try
                    {
                        $term = $termSet.GetTerm($fieldValue.TermGuid)
                    }
                    catch
                    {
                          Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
                    }
                }
                else
                {
                    try
                    {
                        $term = $termSet.GetTerm($fieldValue[0].TermGuid)
                    }
                    catch
                    {
                          Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
                    }
                }
                $taxField.SetFieldValue($page.ListItem, $term)
           }
       }
        $page.ListItem.Update()
       $page.CheckIn("Data clean process")
   }
 
}
function CleanSiteCollection ($siteColUrl)
{
    [Microsoft.SharePoint.SPSite] $site = get-spsite -Limit ALL | where-object {$_.Url -ieq $siteColUrl}
    if($site -eq $null)
    {
        Write-Host -ForegroundColor Red "Unable to find site collection"
        throw "Unable to find site collection"
    }
    CleanTaxonomyList $site
   $site | Get-SPWeb -limit all | ForEach-Object {
            #Check to see if site is a publishing site
            if ([Microsoft.SharePoint.Publishing.PublishingWeb]::IsPublishingWeb($_)) 
            {
                Write-Host "Cleaning pages in `"$($_.Title)`" site."
                #Get the Publishing Web and pages within it
                $publishingWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($_)
                ResetTaxonomyDefaults $site $publishingWeb    
            }
            $_.Dispose()
        }
    $site.Dispose()
}
#endregion
CleanSiteCollection $siteCollectionUrl
 

Moving Web Service Code into a GAC'd Assembly

2009-11-01T19:55:00.001-08:00

Don't know about you, but I can never remember the syntax of a fully qualified class. Here is a quick example of taking a normal web service and splitting up the code so that it can be referenced from the GAC. Often we will need to do this when the service is hosted in SharePoint.

Open the markup of a .asmx file. Right click -> View Markup

Change it from:

@ WebService Language="C#" CodeBehind="Service1.asmx.cs" Class="WebService1.Service1" %>

To:

@ WebService Language="C#" Class="MyNamespace.MyClassName, MyAssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=XYZ" %>

*There are many ways to get the public key token, easiest is probably to put the dll in the GAC then browse to %systemroot%\assembly

What this does

It allows you to put your .asmx file into a web site, such as the ISAPI folder in SharePoint, and when .Net handles the request the assembly resolver will get it out of the GAC.

Disadvantages

Once loaded from the GAC the assembly is cached, so changes won't show up until the app pools are recycled or you run IISRESET.

Also, you will need to put the DLL into the GAC each time you compile. This can be done less painfully by using a post build event or using WSPBuilder's "Copy to gac" function.

SharePoint Helper Code Library (SPHelper)

2009-10-21T19:21:00.001-07:00

I have added my SharePoint Helper library to my new SharePoint Code Zone.

This is a collection of utility classes in C#. A lot of the code is mine, but I also want to credit Edvar Pereira (Edge), Michael Van Den Berg, Andrew Cox from Readify, Chris Johnson from Microsoft (I think Chris might have written some of the MSDN based code) and possibly other smart people I have worked with. If anyone out there see's their code let me know and I'll credit.

PowerShell Execution Policy

2009-10-21T15:47:00.001-07:00

So I decided it was high time I looked into PowerShell and see what all the fuss was about. Grabbed a script I needed from CodePlex. Installed PowerShell. Found the syntax to run the script: .\myScript.ps1

Great, so I execute it. And nothing. It just shows a new line with a flashing cursor. Did it run? Did it error? Nothing. Hmm.. So I work out I can add a line to tell me the script is running:

Write-Host "Begin"

Cool, so I try again. Nothing, nada, zip. No error, no msg. My script definitely isn't running or I should have seen begin.

One of my colleagues, (Thanks Pete), points out that it could be the execution policy. I think sure, sounds plausible, but shouldn't there be an error message telling me that the policy won't allow scripts? You would think so but in this case, NO!

So, I ran:

get-executionpolicy

it returned: Restricted.

"Restricted" is the default policy on any powershell install (Or maybe its OS related, I am on Win 2003).

So I ran:

set-executionpolicy unrestricted

and voila, I could now run my script.

But I'm mad I tell you... MAD!!!!

For more info about this see: http://207.46.16.252/en-us/magazine/2007.09.powershell.aspx

Keywords not showing in Office 2003 integration

2009-10-20T18:18:00.001-07:00

Noticed today that the Keywords field from the content type doesn't appear in the Word integration to SharePoint 2007 SP2. Didn't investigate why as we are giving up on improving the Office 03 integration. Waiting for the enterprise to catch up to Office 07 or maybe skip and go straight to 2010....

SharePoint 2007, Rich Text fields and Office 2003

2009-10-19T21:45:00.001-07:00

I had an interesting issue to deal with today.

Issue (with SharePoint 2007 Service Pack 2):

When using Word 2003 and saving a new document to a document library in SharePoint, a validation issue exists where it ignores input of data in a required field and continually prompts the user to enter data into that field.

This occurs when the field is using Rich Text, is required and the user is saving from Word 2003 (Likely other Office 2003 products have the same issue but I have not tested this).

Steps to reproduce

Create a document in Word 2003, add some content.

Attempt to close the document, and save to a document library with a required Rich Text field. Enter data into the Rich Text field.

Click OK and a validation error occurs stating "You must specify a non-blank value for [FIELD]"

Cause:

Word accesses the SharePoint server and displays a small window allowing the user to enter data for the metadata fields. It references some html and javascript on the SharePoint server which handles saving and validation. The javascript file (BFORM.js) doesn't save the state of the Rich Text control, and subsequent validation calls fail.

Resolution Options:

1. Upgrade to Office 2007

2. Don’t use Rich Text fields in the metadata schemas (Content Types)

3. Manually fix the Javascript file (Not best practice and my invalidate your warranty - but hey, what good is that anyway)

4. Raise the issue to Microsoft Support (and wait 2 months)

Javascript Fix Option

If you choose to fix the javascript here is what you need to do:

Locate the file at:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\LAYOUTS\1033

*Note: The folder 1033 may be different depending on your language.

Edit this file and do the following:

1. Add a global variable "var custom_ActiveForm;" some where towards the top of the file.

2. Search for the function "function OWSForm" and

add the following line to the top of that function:

custom_ActiveFormName = stName;

//Customisation to fix issue with Word 2003

3. Search for the function "function RTE_SaveSelection" and add the following to the end of this function:

try
    {
        document.forms[custom_ActiveFormName][strBaseElementID].innerText = docEditor.activeElement.innerHTML;
    } catch (e) { }

What this does:

It makes the Html Editor save the data into the appropriate form property.

*Please note: I generally don't recommend this option as I cannot be sure it doesn't impact any other areas of SharePoint. Use at your own risk.