Sorry, the operation failed because another operation was updating WOPI bindings at the same time

Configuring SharePoint 2013 for Office Web Apps. While trying to configure the bindings, I would get the following error: "Sorry, the operation failed because another operation was updating WOPI bindings at the same time" This happened any time I tried to modify the bindings, such as Remove-SPWOPIBinding -all:$true. Initially I tried rebuilding the OWA server based on this page: https://blogs.technet.microsoft.com/office_web_apps_server_2013_support_blog/2013/12/20/office-web-apps-2013-rebuild-your-farm-in-a-few-easy-steps/ Unfortunately that didn't work. The resolution was to clear the Config cache across the SharePoint farm: http://alen.malhasoglu.com/2012/01/03/updatedconcurrencyexception-on-sharepoint/ Thanks to Alen Malhasoglu for that one :)

Using ELMAH with Couchbase NetClient 2.1

Originally I tried the nuget package for Elmah integration with Couchbase. However it was using an earlier version of the NetClient and I was having some trouble getting it going. After grabbing the code from git hub (https://github.com/couchbaselabs/elmah-couchbase)
 I managed to update it for the new SDK syntax.
I also needed a custom json converter for the NameValueCollection type due to a deserialization error I received.

Check out the converter here: NameValueCollection Json Converter


My web.config

   <sectionGroup name="couchbaseClients">
      <section name="couchbase"
               type="Couchbase.Configuration.Client.Providers.CouchbaseClientSection, Couchbase.NetClient"/>
    </sectionGroup>
  </configSections>
  <couchbaseClients>
    <couchbase useSsl="false">
      <servers>
        <add uri="http://localhost:8091/pools"></add>
      </servers>
      <buckets>
        <add name="default" useSsl="false">
          <connectionPool name="custom" maxSize="10" minSize="5"></connectionPool>
        </add>
      </buckets>
    </couchbase>
  </couchbaseClients>
  <startup>
    <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1" />
  </startup>

...
 <elmah>
    <errorLog type="MyNameSpace.CouchbaseErrorLog, MyAssembly" couchbaseConfigSection="couchbaseClients/couchbase"  />
  </elmah>

CouchBaseErrorLog.cs

public class CouchbaseErrorLog : ErrorLog
    {
        private static ClientConfiguration _client;
        private static Cluster _cluster;
        private readonly string _elmahBucketName = "default";

        /// <summary>
        ///     Initialize new instance of CouchbaseClient using either default config
        ///     or specified configuration name
        /// </summary>
        public CouchbaseErrorLog(IDictionary config)
        {
            if (config == null)
                throw new ArgumentNullException("config");

            if (_client == null)
            {
                if (config.Contains("couchbaseConfigSection"))
                {
                    _client = CreateCouchbaseClient(config["couchbaseConfigSection"] as string);
                }
                else
                {
                    _client = CreateCouchbaseClient();
                }

                if (config.Contains("applicationName"))
                {
                    ApplicationName = config["applicationName"] as string;
                }
                _cluster = new Cluster(_client);
                SetNameValueConverter();
                    //This fixes issue with NameValueCollection deserialization and the serverVaribles property.
            }
        }

        /// <summary>
        ///     Name displayed in ELMAH viewer
        /// </summary>
        public override string Name
        {
            get { return "Couchbase Server Error Log"; }
        }

        public static ClientConfiguration CreateCouchbaseClient(string couchbaseConfigSection = null)
        {
            if (couchbaseConfigSection != null)
            {
                var config = ConfigurationManager.GetSection(couchbaseConfigSection) as CouchbaseClientSection;
                if (config == null)
                    throw new ArgumentException("Couchbase config section " + couchbaseConfigSection + " not found.");

                return new ClientConfiguration(config);
            }

            return new ClientConfiguration();
        }

        private static void SetNameValueConverter()
        {
            var settings = new JsonSerializerSettings
            {
                ContractResolver = new CamelCasePropertyNamesContractResolver(),
                TypeNameHandling = TypeNameHandling.All,
                Formatting = Formatting.Indented
            };
            settings.Converters.Add(new NameValueCollectionConverter());
            JsonConvert.DefaultSettings = () => settings;
        }

        /// <summary>
        ///     Get error log entry by id
        /// </summary>
        public override ErrorLogEntry GetError(string id)
        {
            using (var bucket = _cluster.OpenBucket(_client.BucketConfigs[_elmahBucketName].BucketName))
            {
                return GetError(bucket, id);
            }
        }

        private ErrorLogEntry GetError(IBucket bucket, string id)
        {
            if (string.IsNullOrEmpty(id))
            {
                throw new ArgumentNullException("id");
            }
            try
            {
                Guid.Parse(id);
            }
            catch (FormatException e)
            {
                throw new ArgumentException(e.Message, "id", e);
            }
            var document = bucket.Get<string>(id);
            if (document.Success)
            {
                var error = JsonConvert.DeserializeObject<Error>(document.Value);
                return new ErrorLogEntry(this, id, error);
            }
            return null;
        }

        /// <summary>
        ///     Get list of errors for view in Elmah web viewer
        /// </summary>
        public override int GetErrors(int pageIndex, int pageSize, IList errorEntryList)
        {
            if (pageIndex < 0)
                throw new ArgumentOutOfRangeException("pageIndex", pageIndex, null);

            if (pageSize < 0)
                throw new ArgumentOutOfRangeException("pageSize", pageSize, null);

            var skip = pageSize*pageIndex;

            //this is NOT the most efficient way to page in Couchbase/CouchDB, but is necessary because
            //there is no way to keep state of the startkey between requests
            //see http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-views-writing-querying-pagination.html
            //for more information
            using (var bucket = _cluster.OpenBucket(_client.BucketConfigs[_elmahBucketName].BucketName))
            {
                var query = bucket.CreateQuery("errors", "by_date", false).Desc().Skip(skip).Limit(pageSize);

                var view = bucket.Query<dynamic>(query);

                foreach (var item in view.Rows)
                {
                    var errorLogEntry = GetError(bucket, item.Id);
                    if (errorLogEntry != null)
                    {
                        errorEntryList.Add(errorLogEntry);
                    }
                }

                return view.Rows.Count();
            }
        }

        /// <summary>
        ///     Log an error
        /// </summary>
        public override string Log(Error error)
        {
            if (error == null)
                throw new ArgumentNullException("error");

            var id = Guid.NewGuid().ToString();
            using (var bucket = _cluster.OpenBucket(_client.BucketConfigs[_elmahBucketName].BucketName))
            {
                // var errorJson = JsonConvert.SerializeObject(error);
                var errorJson = JsonConvert.SerializeObject(error);
                var result = bucket.Insert(id, errorJson);
#if DEBUG
                if (!result.Success && Debugger.IsAttached)
                {
                    Debugger.Break();
                }
#endif
            }
            return id;
        }
    }

View
One last thing, I need to modify the couchbase view.
//view code for Elmah viewer. Must be named "by_date" and located in design document named "errors" without the quotes.
function (doc) {
if (doc.exception) {
emit(doc.time, null);
}
}

Create Security Groups and Assign Permissions in SharePoint 2013

Here is a fun little helper for creating security groups with a permission assignment.
I use the word fun because I have been working with SharePoint too long and my brain now works in reverse.

/// <summary>

    /// Security Helper

    /// </summary>

    public class SecurityHelper

    {

        public static bool GroupExists(SPGroupCollection groups, string name)

        {

            if (String.IsNullOrEmpty(name) || (name.Length > 255) ||(groups == null) || (groups.Count == 0))

            {

                return false;

            }

            return groups.Cast<SPGroup>().FirstOrDefault(t => t.Name == name) != null;

        }

        /// <summary>

        /// Creates a group if it does not exist.  It will also assign a permission level to the group.

        /// </summary>

        /// <param name="web"></param>

        /// <param name="groupName"></param>

        /// <param name="permissionLevel"></param>

        /// <param name="description"></param>

        /// <param name="owner"></param>

        /// <returns>The Group</returns>

        public static SPMember CreateGroup(SPWeb web, string groupName, string permissionLevel, string description, SPMember owner = null)

        {

            string uniqueGroupName = String.Format("{0} - {1}", web.Name, groupName);

            if (!GroupExists(web.SiteGroups, uniqueGroupName))

            {

                var role = web.RoleDefinitions.Cast<SPRoleDefinition>().FirstOrDefault(t => t.Name == permissionLevel);

                if (role == null)

                {

                    throw new KeyNotFoundException(String.Format("The Security Permission level {0} does not exist at web {1}.", permissionLevel, web.Url));

                }

                if (owner == null)

                {

                    owner = web.CurrentUser;

                }

                web.SiteGroups.Add(uniqueGroupName, owner, null, description);

                SPGroup group = web.SiteGroups.GetByName(uniqueGroupName);

                var assignment = new SPRoleAssignment(@group);

                assignment.RoleDefinitionBindings.Add(role);

                web.RoleAssignments.Add(assignment);

            }

            return web.SiteGroups.GetByName(uniqueGroupName);

        }

    }

Breaking permissions with SharePoint 2013 & CSOM

Lately I've been doing a lot of work with the Client Side Object Model for SharePoint 2013.  While I have been able to do most things we used to, in the dark days of server side development. However one thing didn't seem to work right, Web.BreakRoleInheritance(bool, bool).
Along the lines of:

web.BreakRoleInheritance(true, true);
clientContext.ExecuteQuery();

Passing the first parameter states the call should break permissions at the web level and copy existing RoleAssignments.  However, I could not get this to work.  Every time the assignments where gone.  Regardless of the values I passed.

The fallback option was to devolve, create a feature with server side code, which looks pretty much identical.  I then call the feature activation automatically through my CSOM service.

public override void FeatureActivated(SPFeatureReceiverProperties properties)
        {
            try
            {
                var web = (SPWeb) properties.Feature.Parent;
                SPSecurity.RunWithElevatedPrivileges(() =>
                {
                    using (var site = new SPSite(web.Site.ID))
                    {
                        using (SPWeb elevatedWeb = site.OpenWeb(web.ID))
                        {
                            elevatedWeb.BreakRoleInheritance(true, true);
                        }
                    }
                   
                });
            }
            catch (Exception ex)
            {
                LogService.LogException(LogCategory.FeatureReceiver, ex);
                throw;
            }
        }

And that worked as expected, role assignments had been copied through from the parent.

Search service lost SQL permission

Summary

While working with Result Sources in a site collection, I got that wonderful "Something went wrong" error. The ULS showed a bunch of SQL execute permission errors on the search database.  The solution was to update the SQL permissions for the search service.  It was not obvious why the permissions where lost in the first place.

Symptoms

ULS Error events such as:

System.Data.SqlClient.SqlException (0x80131904): The EXECUTE permission was denied on the object 'proc_MSS_GetLease', database 'SP2013_DEV__SA_Search'

…and other SQL permission errors.

Fix

Grant SPSearchDBAdmin role to the search service account on the Search database.  (SQL Server Permissions)

Thoughts

Just before this occurred I had updated permissions of the Search Service Application in Central Admin.  I had added my user account.  Can't see how this would have caused it however.

Content classification approach for SharePoint 2013 / Office365

I had a request to implement a classification system for SharePoint 2013 sites.  This needed to work with Office 365 and On Premise so farm solutions are a no go.  My company gave permission to make this public so here goes.

The process to use this consists of:

1. Update property bag values on your site collection's root web
2. Upload the Classify.js to the site collection
3. Add a reference to the JS in the master page

For all the detailed information and downloadable scripts visit my Wiki.

TimeStamps with Entity Framework

So I've been building an MVC 5, EF6 solution with Code First Migrations.
Got what I think is a great structure of abstraction thanks to Le Long's Post.

One thing that wasn't obvious (to me at least), was how to handle concurrency with EF and the Scaffolded Views.
When editing an object, the TimeStamp was never returned, and of course EF would throw a "DbUpdateConcurrencyException".

Here is a solution that worked for me:

1. Create a base entity class with a TimeStamp on it
2. Add the attributes: [Timestamp, ScaffoldColumn(false)]
3. Edit the MVC View and add a hidden field for the timestamp

Base Class and Attributes
I defined a base class for all entities requiring concurrency.
public abstract class PersistedEntity : EntityBase
{
        [Timestamp, ScaffoldColumn(false)]
        public byte[] TimeStamp { get; set; }
}

MVC View
I found that the view did not by itself return the TimeStamp with the model when posting data back to the server.  So I added this after the @Html.BeginForm():
@Html.HiddenFor(model => model.TimeStamp)

It was then returned back to the server, and the entity could be saved correctly.

Knockout-Kendo Grid pager not updating

While using Knockout and KendoUI I came across a paging issue.  The pager wouldn't update but the grid rows did.  This only occurred with Internet Explorer and I was able to reproduce it with IE 9 and 11.
In Chrome the pager updated fine.

In the end, the fix was to detect when the data had changed then refresh the pager.

In my javascript I used:

ui.tasksUpdated = function () {
            var element = $("#grid");
            var grid = element.data('kendoGrid');
            grid.pager.setDataSource(grid.dataSource);
            grid.pager.refresh();
            grid.refresh();
}

Strictly speaking I don't think the 2 refresh methods are needed as it seems that the setDataSource method does it all.

SP2013 Error - [Forced due to logging gap, Original Level: Verbose] GetUriScheme (...)

I found an issue with a clients SharePoint 2013 today and thought I'd share.

Symptoms

• High CPU on w3wp process
• Pages (including system pages) would not load and eventually error after a few minutes
• No obvious error message in logs

What I did find in the ULS logs was:

[Forced due to logging gap, Original Level: Verbose] GetUriScheme(/web/Pages/Apage.aspx)

Cause

Something in that page was keeping SharePoint busy.  I don't know what it was as it had been created by someone else.  It felt like an infinite loop error, but who knows....

Fix

Delete the file.  (Had to check in the file first).

Made some simple scripts to help.

• Dir-SPFile (Show files in a list)
• CheckIn-SPFile (Checks in the specified file)
• Del-SPFile (Deletes the file.  No confirmation required)

Dir-SPFile

#Script: Dir-SPFile
#Usage: .\Dir-SPFile.ps1 http://my2013site Pages
param (
$webUrl = $(throw "Please specify parameter webUrl"),
$listName = $(throw "Please specify parameter listName")
)
$web = Get-SPWeb $webUrl -Limit all
if($web -ne $null)
{
$web.Lists[$listName].Items | ft Id, Title, Url
}
else
{
throw 'Unable to locate site ' + $webUrl
}

CheckIn-SPFile

#Script: CheckIn-SPFile
#Usage: .\CheckIn-SPFile.ps1 http://my2013site /Pages/SomePage.aspx
param (
$webUrl = $(throw "Please specify parameter webUrl"),
$fileUrl = $(throw "Please specify parameter fileUrl")
)


$web = Get-SPWeb $webUrl -Limit all
if($web -ne $null)
{
$file = $web.GetFile($webUrl + $fileUrl)
if($file -eq $null -or !$file.Exists)
{
throw 'Unable to locate file ' + $webUrl + $fileUrl
}
if($file.CheckedOutBy -eq $null)
{
throw 'file ' + $webUrl + $fileUrl + " is already checked in"
}
$file.CheckIn('Forced Checkin');
"Complete - $webUrl $fileUrl Checked In"
}
else
{
throw 'Unable to locate site ' + $webUrl
}

Del-SPFile

#Script: Del-SPFile

#Usage: .\Del-SPFile.ps1 http://my2013site /Pages/SomePage.aspx

param (

$webUrl = $(throw "Please specify parameter webUrl"),

$fileUrl = $(throw "Please specify parameter fileUrl")

)

$web = Get-SPWeb $webUrl -Limit all

if($web -ne $null)

{

$file = $web.GetFile($webUrl + $fileUrl)

if($file -eq $null -or !$file.Exists)

{

throw 'Unable to locate file ' + $webUrl + $fileUrl

}

$file.Delete();

"Complete - $webUrl $fileUrl Deleted"

}

else

{

throw 'Unable to locate site ' + $webUrl

}

SharePoint Designer 2013 can't connect

Recently I had an issue trying to use SharePoint Designer 2013.
After trying to connect to my SharePoint 2013 environment it kept displaying the very vague error message:

Request Error
         Request Error The server encountered an error processing the request. See server logs for more details.

Using fiddler I was able to see that the call to  http://server/_vti_bin/client.svc/ProcessQuery was failing with a 400 (Bad Request) error.

The only errors that stood out in the ULS logs (amoung the other countless errors that just seem to be normal) was:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Data.OData, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

And

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Data.Edm, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

Interestingly enough I had installed a later version of these nu-get packages on another (non SharePoint) project.  Since this is my general DEV environment I may have played a part in this error.

I did see some other posts talking about enabling anonymous access, but that wasn't the problem in my case.

At any rate, the fix was to gac the correct versions of those dll's.

1. Created a new blank project in Visual Studio 2012/13
2. In the package manager console install the nu-get packages:
3. Install-Package Microsoft.Data.OData -Version 5.0.0.50403 http://www.nuget.org/packages/Microsoft.Data.OData/5.0.0.50403
4. Install-Package Microsoft.Data.Edm -Version 5.0.0.50403

Then browse to your source folder for the new package and locate the \packages\Microsoft.Data.OData.5.0.0.50403\lib\net40\Microsoft.Data.OData.dll

and

\packages\Microsoft.Data.OData.5.0.0.50403\lib\net40\Microsoft.Data.Edm.dll

You can now easily gac these using gacutil or simple drag into C:\Windows\Assembly

One iisreset later and all was well in SPD land...

Azure Cloud Storage OutOfRange Error

I'm having an interesting time learning Azure and have found various issues, including Visual Studio 2010 hanging when the project is bound to Visual SourceSafe, random hang on project build and the AspProviders not working with development storage.

Here's what I found with the AspProviders issue:

The AspProviders from the Azure Samples errored on start up with an OutOfRange error . The providers didn't seem to work with developer storage. The providers didn't allow access to the dev storage addresses, likely due to http instead of https. To fix this I created a helper class that would get either the developer storage or the https address.

I had problems with TableStorageRoleProvider, TableStorageSessionStateProvider, TableStorageProfileProvider and TableStorageMembershipProvider.

The error I received was:

Microsoft.WindowsAzure.StorageClient.StorageClientException: Exception of type 'Microsoft.WindowsAzure.StorageClient.StorageClientException' was thrown. ---> System.Data.Services.Client.DataServiceQueryException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code>OutOfRangeInput</code><message xml:lang="en-US">One of the request inputs is out of range.</message></error>

The client class I created:

Code Snippet

1. /// <summary>
2.     /// Class to create <see cref="CloudTableClient"/> or <see cref="CloudBloblClient"/> or <see cref="CloudQueueClient"/>
3.     /// Works with developer storage at address "http://127.0.0.1" or https address.
4.     /// </summary>
5.     public static class StorageClient
6.     {
7.         
8.         public static CloudTableClient GetCloudTableClient(string endpoint, string accountName, string sharedKey)
9.         {
10.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
11.             return account.CreateCloudTableClient();
12.         }
13.         public static CloudTableClient GetCloudTableClient(string endpoint, StorageCredentialsAccountAndKey credentials)
14.         {
15.             CloudStorageAccount account = GetAccount(endpoint, credentials);
16.             return account.CreateCloudTableClient();
17.         }
18.  
19.         public static CloudBlobClient GetCloudBlobClient(string endpoint, string accountName, string sharedKey)
20.         {
21.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
22.             return account.CreateCloudBlobClient();
23.         }
24.         public static CloudBlobClient GetCloudBlobClient(string endpoint, StorageCredentialsAccountAndKey credentials)
25.         {
26.             CloudStorageAccount account = GetAccount(endpoint, credentials);
27.             return account.CreateCloudBlobClient();
28.         }
29.  
30.         public static CloudQueueClient GetCloudQueueClient(string endpoint, string accountName, string sharedKey)
31.         {
32.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
33.             return account.CreateCloudQueueClient();
34.         }
35.         public static CloudQueueClient GetCloudQueueClient(string endpoint, StorageCredentialsAccountAndKey credentials)
36.         {
37.             CloudStorageAccount account = GetAccount(endpoint, credentials);
38.             return account.CreateCloudQueueClient();
39.         }
40.         
41.         private static bool IsDevelopmentStorage(string endpoint)
42.         {
43.             const string devEndpoint = "http://127.0.0.1";
44.             return (endpoint != null && endpoint.Length >= devEndpoint.Length &&
45.                     devEndpoint.ToLower().Substring(0, devEndpoint.Length) == devEndpoint);
46.         }
47.         private static CloudStorageAccount GetAccount(string endpoint, StorageCredentialsAccountAndKey credentials)
48.         {
49.             CloudStorageAccount account = null;
50.             if (IsDevelopmentStorage(endpoint))
51.             {
52.                 account = CloudStorageAccount.DevelopmentStorageAccount;
53.             }
54.             else
55.             {
56.                 account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
57.             }
58.             return account;
59.         }
60.         private static CloudStorageAccount GetAccount(string endpoint, string accountName, string sharedKey)
61.         {
62.             CloudStorageAccount account = null;
63.             if (IsDevelopmentStorage(endpoint))
64.             {
65.                 account = CloudStorageAccount.DevelopmentStorageAccount;
66.             }
67.             else
68.             {
69.                 var credentials = new StorageCredentialsAccountAndKey(accountName, sharedKey);
70.                 account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
71.             }
72.             return account;
73.         }
74.     }

I then changed the providers where they create the cloud client.

In the TableStorageRoleProvider I update the following line:

_tableStorage = new CloudTableClient(baseUri, info);

To:

_tableStorage = StorageClient.GetCloudTableClient(baseUri, accountName, sharedKey);

The other providers are similar.

That’s it!

The Heading “Libraries” automatically added through content import

Problem

When exporting and importing a single Document Library using the Import-SPWeb command, the Heading "Libraries" is automatically added to the site navigation if it is not there. Exporting then re importing to the same site or another site always adds the heading. Seems to be no option to stop this behaviour.

Solution

As I don’t want the heading Libraries to be added I guess I can either delete them manually or build a PowerShell script that moves through all the SPWebs and deletes them.  Not a great solution and this really should be controlled via parameters on Import-SPWeb.  Hey Microsoft, can you sort that for me please?  Thanks.

Content Migration and Taxonomy Fields

 

I have been using Import-SPWeb and Export-SPWeb to migrate content from different environments and found an issue if the sites use Taxonomy Fields.

The problem in a nutshell

The content is exported from a site using the PowerShell command Export-SPWeb or with code. The site contains Taxonomy fields included in content types. The taxonomy fields are wired to a Managed Metadata Service Term Store. The taxonomy fields have id's to both a TaxonomyHiddenList at the root web level and to the Term stores themselves. These ID's are migrated along with the list. However in the next environment you try to import the content to the Managed Metadata Service has a different ID for the Term Store, and as such the taxonomy fields fail to work properly.

Note: I'm not sure if this is an issue if you use the default Managed Metadata Service, I am using a new one created from a PowerShell script.

Visible Issues

If the site was accessed as an anonymous user, some pages would error with:

 

The error message might as well just say “Something bad’s happened, but you have to figure out what. Good luck!”  (Well, I could turn off custom errors but that would be too easy and harder to make fun of!)

So the Slightly more helpful are the trace logs which showed:

"Hidden list not found, creating new one"

"Creating taxonomy hidden list"

"Unknown SPRequest error occurred. More information: 0x80070005"

"System.ArgumentException: Value does not fall within the expected range."

Strangely, the first page would access correctly under anonymous and subsequent requests to other pages would fail.  I have a theory about why but wild speculation is probably best left unsaid.

Other Symptoms

The publishing root web contains a list called "TaxonomyHiddenList". This list is used by the taxonomy fields to store terms in use for the site. After using Import-SPWeb many extra entries where added to this list with an empty title.

If you use SharePoint Manager (very useful BTW), you will see a bunch of entries with “(no title)”.  As far as I can see this shouldn’t happen, normally when the taxonomy fields create these entries they include the label as the title. 

The above screen shot of SharePoint Manager shows the bad entries.  Notice as well that there are lots, 64 in this example.  If this was working properly you should see the following:

In this screen shot the entries have a valid title.  These values originally come from the terms in the term store.  These entries where created by a powershell script and added to the term store, they are some of the ones required for a dublin core and AGLS metadata implementation.  (Which is probably about as interesting as xml namespaces or writing documentation but I felt I had to explain them anyway)

The cause

If you look at the definition of the Microsoft.SharePoint.Taxonomy.TaxonomyField you will see a property called SspId. This property is the Guid of the Term Store that its connected to. After importing, this value points to the old term store. It also has a side effect of creating bad entries in the /Lists/TaxonomyHiddenList list and is likely the cause of the entries in the trace logs. The data in the field also points to the bad entry within that list rather than the entry it should point to. There is a good write up on how the entries are created here so I won't go into that part too much.

The fix

The fix for this is much simpler than tracking it down I assure you. After many frustrating hours and some stern words aimed directly at SharePoint I managed to coble together a script that suitably cleans my site collections after content migration.

The following is a PowerShell script that will clean up your publishing site.  Use at your own risk. Backup your data, etc, etc! Wish you the best of luck! Like a Microsoft error message

Note: This script will utilise the field definitions in the root web.  If you are migrating an entire web then you should fix your content types prior to running this script.

Code Snippet

1. #Script: CleanTaxonomyFields.ps1
2. #Author: Tim Wheeler (http://tjwheeler.blogspot.com/)
3. #Created: 17/11/2010
4. #Notes:
5. #When content in imported into a new environment the Term Store ID is kept on the
6. #taxonomy fields.  This causes a problem as the new term store has a different id.
7. #These functions clean up the taxonomy list which has bad entries following an import
8. #and modify all the items and resets the correct values.
9. param
10.     (    $siteCollectionUrl = (Read-Host "Please enter a site collection url") )
11.     
12. #region SharePoint Snappin Setup
13. $snapin="Microsoft.SharePoint.PowerShell"
14. if (get-pssnapin $snapin -ea "silentlycontinue") {
15.     write-host -f Green "PSsnapin $snapin is loaded"
16. }
17. else {
18.     if (get-pssnapin $snapin -registered -ea "silentlycontinue") {
19.         write-host -f Green "PSsnapin $snapin is registered"
20.         Add-PSSnapin $snapin
21.         write-host -f Green "PSsnapin $snapin is loaded"
22.     }
23.     else {
24.         write-host -f Red "PSSnapin $snapin not found"
25.     }
26. }
27. #endregion
28.     
29. #region Data Clean Up
30.  
31. function CleanTaxonomyList($site)
32. {
33.     Write-Host "Cleaning /Lists/TaxonomyHiddenList"
34.     $web = $site.RootWeb
35.       $taxlist =$web.GetList("Lists/TaxonomyHiddenList")
36.     for($count = $taxlist.Items.Count - 1; $count -ge 0; $count--)
37.     {
38.          $item = $taxlist.Items[$count]
39.         if([string]::IsNullOrEmpty($item.Title))
40.         {
41.             $item.Delete()
42.             if($?)
43.             {
44.                 #Write-Host "Deleted Item with bad title"
45.             }
46.             else
47.             {
48.                 Write-Host -ForegroundColor Red "Failed to deleted Item with bad title"
49.             }
50.         }
51.         
52.     }
53. }
54. #This function will reset the term store id if it is wrong, and will fix the WssId.
55. #If the correct term cannot be located, the default one will be used.
56. function ResetTaxonomyDefaults($site, [Microsoft.SharePoint.Publishing.PublishingWeb] $web)
57. {
58.     $txs = New-Object "Microsoft.SharePoint.Taxonomy.TaxonomySession" -ArgumentList $site
59.    $pages = $web.GetPublishingPages()
60.    foreach($page in $pages)
61.    {
62.         Write-Host -ForegroundColor Cyan "Checking publishing page " $page.Title
63.        if($page.ListItem.File.CheckOutStatus -ne "None")
64.         {
65.             $page.CheckIn("Checked in by data clean process");
66.         }
67.         $page.CheckOut();
68.        foreach ($field in $page.ListItem.Fields)
69.        {
70.            if($field.GetType().Name -eq "TaxonomyField")
71.            {
72.                $taxField = [Microsoft.SharePoint.Taxonomy.TaxonomyField] $field
73.                 
74.                 Write-Host "Found field to update:" $taxField.Title
75.                 $currentValue = $page.ListItem.Properties[$taxField.InternalName]
76.                 Write-Host "Current Value is" $currentValue
77.                 $templateField = $page.ListItem.ParentList.ParentWeb.Site.RootWeb.Fields[$field.Id]
78.                 $defaultValue = $templateField.DefaultValue
79.                 $termStore = $txs.TermStores[$templateField.SspId]
80.                 $termSet = $termStore.GetTermSet($templateField.TermSetId)
81.                 if($taxField.SspId -ne $templateField.SspId)
82.                 {
83.                     Write-Host "TaxField SspId is not correct, updating"
84.                     $taxField.SspId = $templateField.SspId
85.                     $taxField.Update()
86.                 }
87.                 if($taxField.TermSetId -ne $templateField.TermSetId)
88.                 {
89.                     Write-Host "TaxField TermSetId is not correct, updating"
90.                     $taxField.TermSetId = $templateField.TermSetId
91.                     $taxField.Update()
92.                 }
93.                 $fieldValue = $templateField.GetFieldValue($currentValue)
94.                   if(($fieldValue.GetType().Name -eq "TaxonomyFieldValue") -and($fieldValue -eq $null -or [string]::IsNullOrEmpty($fieldValue.TermGuid)))
95.                 {
96.                     $fieldValue = $templateField.GetFieldValue($defaultValue)
97.                 }
98.                   if(($fieldValue.GetType().Name -ne "TaxonomyFieldValue") -and($fieldValue[0] -eq $null -or [string]::IsNullOrEmpty($fieldValue[0].TermGuid)))
99.                 {
100.                     $fieldValue = $templateField.GetFieldValue($defaultValue)
101.                 }
102.                 if($fieldValue.GetType().Name -eq "TaxonomyFieldValue")
103.                 {
104.                     try
105.                     {
106.                         $term = $termSet.GetTerm($fieldValue.TermGuid)
107.                     }
108.                     catch
109.                     {
110.                           Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
111.                     }
112.                 }
113.                 else
114.                 {
115.                     try
116.                     {
117.                         $term = $termSet.GetTerm($fieldValue[0].TermGuid)
118.                     }
119.                     catch
120.                     {
121.                           Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
122.                     }
123.                 }
124.                 $taxField.SetFieldValue($page.ListItem, $term)
125.            }
126.        }
127.         $page.ListItem.Update()
128.        $page.CheckIn("Data clean process")
129.    }
130.  
131. }
132. function CleanSiteCollection ($siteColUrl)
133. {
134.     [Microsoft.SharePoint.SPSite] $site = get-spsite -Limit ALL | where-object {$_.Url -ieq $siteColUrl}
135.     if($site -eq $null)
136.     {
137.         Write-Host -ForegroundColor Red "Unable to find site collection"
138.         throw "Unable to find site collection"
139.     }
140.     CleanTaxonomyList $site
141.    $site | Get-SPWeb -limit all | ForEach-Object {
142.             #Check to see if site is a publishing site
143.             if ([Microsoft.SharePoint.Publishing.PublishingWeb]::IsPublishingWeb($_))
144.             {
145.                 Write-Host "Cleaning pages in `"$($_.Title)`" site."
146.                 #Get the Publishing Web and pages within it
147.                 $publishingWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($_)
148.                 ResetTaxonomyDefaults $site $publishingWeb    
149.             }
150.             $_.Dispose()
151.         }
152.     $site.Dispose()
153. }
154. #endregion
155. CleanSiteCollection $siteCollectionUrl

Moving Web Service Code into a GAC'd Assembly

Don't know about you, but I can never remember the syntax of a fully qualified class.  Here is a quick example of taking a normal web service and splitting up the code so that it can be referenced from the GAC.  Often we will need to do this when the service is hosted in SharePoint. 

Open the markup of a .asmx file.  Right click -> View Markup

Change it from:

<%

@ WebService Language="C#" CodeBehind="Service1.asmx.cs" Class="WebService1.Service1" %>

To:

<%

@ WebService Language="C#" Class="MyNamespace.MyClassName, MyAssemblyName, Version=1.0.0.0, Culture=neutral, PublicKeyToken=XYZ" %>

*There are many ways to get the public key token, easiest is probably to put the dll in the GAC then browse to %systemroot%\assembly

What this does

It allows you to put your .asmx file into a web site, such as the ISAPI folder in SharePoint, and when .Net handles the request the assembly resolver will get it out of the GAC. 

Disadvantages

Once loaded from the GAC the assembly is cached, so changes won't show up until the app pools are recycled or you run IISRESET.

Also, you will need to put the DLL into the GAC each time you compile.  This can be done less painfully by using a post build event or using WSPBuilder's "Copy to gac" function.

SharePoint Helper Code Library (SPHelper)

I have added my SharePoint Helper library to my new SharePoint Code Zone.

This is a collection of utility classes in C#.  A lot of the code is mine, but I also want to credit Edvar Pereira (Edge), Michael Van Den Berg,  Andrew Cox from Readify, Chris Johnson from Microsoft (I think Chris might have written some of the MSDN based code) and possibly other smart people I have worked with.  If anyone out there see's their code let me know and I'll credit.

PowerShell Execution Policy

So I decided it was high time I looked into PowerShell and see what all the fuss was about.  Grabbed a script I needed from CodePlex.  Installed PowerShell.  Found the syntax to run the script:  .\myScript.ps1

Great, so I execute it.  And nothing.  It just shows a new line with a flashing cursor.  Did it run? Did it error?  Nothing.  Hmm.. So I work out I can add a line to tell me the script is running:

Write-Host "Begin"

Cool, so I try again.  Nothing, nada, zip.  No error, no msg.  My script definitely isn't running or I should have seen begin.

One of my colleagues, (Thanks Pete), points out that it could be the execution policy.  I think sure, sounds plausible, but shouldn't there be an error message telling me that the policy won't allow scripts?  You would think so but in this case, NO!

So, I ran:

get-executionpolicy

it returned: Restricted.

"Restricted" is the default policy on any powershell install (Or maybe its OS related, I am on Win 2003).

So I ran:

set-executionpolicy unrestricted

and voila, I could now run my script.

But I'm mad I tell you... MAD!!!!

For more info about this see: http://207.46.16.252/en-us/magazine/2007.09.powershell.aspx

Keywords not showing in Office 2003 integration

Noticed today that the Keywords field from the content type doesn't appear in the Word integration to SharePoint 2007 SP2.  Didn't investigate why as we are giving up on improving the Office 03 integration.  Waiting for the enterprise to catch up to Office 07 or maybe skip and go straight to 2010....

SharePoint 2007, Rich Text fields and Office 2003

I had an interesting issue to deal with today. 

Issue (with SharePoint 2007 Service Pack 2):

When using Word 2003 and saving a new document to a document library in SharePoint, a validation issue exists where it ignores input of data in a required field and continually prompts the user to enter data into that field.

This occurs when the field is using Rich Text, is required and the user is saving from Word 2003 (Likely other Office 2003 products have the same issue but I have not tested this).

Steps to reproduce

Create a document in Word 2003, add some content.

Attempt to close the document, and save to a document library with a required Rich Text field.  Enter data into the Rich Text field.

Click OK and a validation error occurs stating "You must specify a non-blank value for [FIELD]"

Cause:

Word accesses the SharePoint server and displays a small window allowing the user to enter data for the metadata fields.  It references some html and javascript on the SharePoint server which handles saving and validation.  The javascript file (BFORM.js) doesn't save the state of the Rich Text control, and subsequent validation calls fail.

Resolution Options:

1.       Upgrade to Office 2007

2.       Don’t use Rich Text fields in the metadata schemas (Content Types)

3.       Manually fix the Javascript file (Not best practice and my invalidate your warranty - but hey, what good is that anyway)

4.       Raise the issue to Microsoft Support (and wait 2 months)

Javascript Fix Option

If you choose to fix the javascript here is what you need to do:

Locate the file at:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\12\TEMPLATE\LAYOUTS\1033

*Note: The folder 1033 may be different depending on your language.

Edit this file and do the following:

1.  Add a global variable "var custom_ActiveForm;" some where towards the top of the file.

2.  Search for the function "function OWSForm" and

add the following line to the top of that function:

custom_ActiveFormName = stName;

//Customisation to fix issue with Word 2003

3. Search for the function "function RTE_SaveSelection" and add the following to the end of this function:

try
    {
        document.forms[custom_ActiveFormName][strBaseElementID].innerText = docEditor.activeElement.innerHTML;
    } catch (e) { }

 

What this does:

It makes the Html Editor save the data into the appropriate form property. 

*Please note:  I generally don't recommend this option as I cannot be sure it doesn't impact any other areas of SharePoint.  Use at your own risk.