Please note: This blog is no longer active. My new blog is located at http://blog.timwheeler.io

Sunday, March 9, 2014

TimeStamps with Entity Framework

So I've been building an MVC 5, EF6 solution with Code First Migrations.
Got what I think is a great structure of abstraction thanks to Le Long's Post.

One thing that wasn't obvious (to me at least), was how to handle concurrency with EF and the Scaffolded Views.
When editing an object, the TimeStamp was never returned, and of course EF would throw a "DbUpdateConcurrencyException".

Here is a solution that worked for me:

  1. Create a base entity class with a TimeStamp on it
  2. Add the attributes: [Timestamp, ScaffoldColumn(false)]
  3. Edit the MVC View and add a hidden field for the timestamp


Base Class and Attributes
I defined a base class for all entities requiring concurrency.
public abstract class PersistedEntity : EntityBase
{
        [Timestamp, ScaffoldColumn(false)]
        public byte[] TimeStamp { get; set; }
}

MVC View
I found that the view did not by itself return the TimeStamp with the model when posting data back to the server.  So I added this after the @Html.BeginForm():
@Html.HiddenFor(model => model.TimeStamp)

It was then returned back to the server, and the entity could be saved correctly.





Sunday, February 16, 2014

Knockout-Kendo Grid pager not updating

While using Knockout and KendoUI I came across a paging issue.  The pager wouldn't update but the grid rows did.  This only occurred with Internet Explorer and I was able to reproduce it with IE 9 and 11.
In Chrome the pager updated fine.

In the end, the fix was to detect when the data had changed then refresh the pager.

In my javascript I used:

ui.tasksUpdated = function () {
            var element = $("#grid");
            var grid = element.data('kendoGrid');
            grid.pager.setDataSource(grid.dataSource);
            grid.pager.refresh();
            grid.refresh();
}

Strictly speaking I don't think the 2 refresh methods are needed as it seems that the setDataSource method does it all.



Saturday, February 15, 2014

SP2013 Error - [Forced due to logging gap, Original Level: Verbose] GetUriScheme (...)

I found an issue with a clients SharePoint 2013 today and thought I'd share.

Symptoms

  • High CPU on w3wp process
  • Pages (including system pages) would not load and eventually error after a few minutes
  • No obvious error message in logs
What I did find in the ULS logs was:
[Forced due to logging gap, Original Level: Verbose] GetUriScheme(/web/Pages/Apage.aspx)

Cause
Something in that page was keeping SharePoint busy.  I don't know what it was as it had been created by someone else.  It felt like an infinite loop error, but who knows....

Fix
Delete the file.  (Had to check in the file first).

Made some simple scripts to help.

  • Dir-SPFile (Show files in a list)
  • CheckIn-SPFile (Checks in the specified file)
  • Del-SPFile (Deletes the file.  No confirmation required)
Dir-SPFile


#Script: Dir-SPFile
#Usage: .\Dir-SPFile.ps1 http://my2013site Pages
param (
$webUrl = $(throw "Please specify parameter webUrl"),
$listName = $(throw "Please specify parameter listName")
)
$web = Get-SPWeb $webUrl -Limit all
if($web -ne $null)
{
$web.Lists[$listName].Items | ft Id, Title, Url
}
else
{
throw 'Unable to locate site ' + $webUrl
}


CheckIn-SPFile
#Script: CheckIn-SPFile
#Usage: .\CheckIn-SPFile.ps1 http://my2013site /Pages/SomePage.aspx
param (
$webUrl = $(throw "Please specify parameter webUrl"),
$fileUrl = $(throw "Please specify parameter fileUrl")
)


$web = Get-SPWeb $webUrl -Limit all
if($web -ne $null)
{
$file = $web.GetFile($webUrl + $fileUrl)
if($file -eq $null -or !$file.Exists)
{
throw 'Unable to locate file ' + $webUrl + $fileUrl
}
if($file.CheckedOutBy -eq $null)
{
throw 'file ' + $webUrl + $fileUrl + " is already checked in"
}
$file.CheckIn('Forced Checkin');
"Complete - $webUrl $fileUrl Checked In"
}
else
{
throw 'Unable to locate site ' + $webUrl
}

Del-SPFile
#Script: Del-SPFile
#Usage: .\Del-SPFile.ps1 http://my2013site /Pages/SomePage.aspx
param (
$webUrl = $(throw "Please specify parameter webUrl"),
$fileUrl = $(throw "Please specify parameter fileUrl")
)

$web = Get-SPWeb $webUrl -Limit all
if($web -ne $null)
{
$file = $web.GetFile($webUrl + $fileUrl)
if($file -eq $null -or !$file.Exists)
{
throw 'Unable to locate file ' + $webUrl + $fileUrl
}
$file.Delete();
"Complete - $webUrl $fileUrl Deleted"
}
else
{
throw 'Unable to locate site ' + $webUrl
}


Wednesday, November 6, 2013

SharePoint Designer 2013 can't connect

Recently I had an issue trying to use SharePoint Designer 2013.
After trying to connect to my SharePoint 2013 environment it kept displaying the very vague error message:

Request Error
         Request Error The server encountered an error processing the request. See server logs for more details.

Using fiddler I was able to see that the call to  http://server/_vti_bin/client.svc/ProcessQuery was failing with a 400 (Bad Request) error.

The only errors that stood out in the ULS logs (amoung the other countless errors that just seem to be normal) was:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Data.OData, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

And

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.Data.Edm, Version=5.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' or one of its dependencies. The system cannot find the file specified.

Interestingly enough I had installed a later version of these nu-get packages on another (non SharePoint) project.  Since this is my general DEV environment I may have played a part in this error.

I did see some other posts talking about enabling anonymous access, but that wasn't the problem in my case.

At any rate, the fix was to gac the correct versions of those dll's.

  1. Created a new blank project in Visual Studio 2012/13
  2. In the package manager console install the nu-get packages:
  3. Install-Package Microsoft.Data.OData -Version 5.0.0.50403 http://www.nuget.org/packages/Microsoft.Data.OData/5.0.0.50403
  4. Install-Package Microsoft.Data.Edm -Version 5.0.0.50403
Then browse to your source folder for the new package and locate the \packages\Microsoft.Data.OData.5.0.0.50403\lib\net40\Microsoft.Data.OData.dll
and
\packages\Microsoft.Data.OData.5.0.0.50403\lib\net40\Microsoft.Data.Edm.dll

You can now easily gac these using gacutil or simple drag into C:\Windows\Assembly
One iisreset later and all was well in SPD land...






Thursday, January 6, 2011

Azure Cloud Storage OutOfRange Error

I'm having an interesting time learning Azure and have found various issues, including Visual Studio 2010 hanging when the project is bound to Visual SourceSafe, random hang on project build and the AspProviders not working with development storage.

Here's what I found with the AspProviders issue:

The AspProviders from the Azure Samples errored on start up with an OutOfRange error . The providers didn't seem to work with developer storage. The providers didn't allow access to the dev storage addresses, likely due to http instead of https. To fix this I created a helper class that would get either the developer storage or the https address.

I had problems with TableStorageRoleProvider, TableStorageSessionStateProvider, TableStorageProfileProvider and TableStorageMembershipProvider.

The error I received was:

Microsoft.WindowsAzure.StorageClient.StorageClientException: Exception of type 'Microsoft.WindowsAzure.StorageClient.StorageClientException' was thrown. ---> System.Data.Services.Client.DataServiceQueryException: An error occurred while processing this request. ---> System.Data.Services.Client.DataServiceClientException: <?xml version="1.0" encoding="utf-8" standalone="yes"?><error xmlns="http://schemas.microsoft.com/ado/2007/08/dataservices/metadata"><code>OutOfRangeInput</code><message xml:lang="en-US">One of the request inputs is out of range.</message></error>

The client class I created:

Code Snippet
  1. /// <summary>
  2.     /// Class to create <see cref="CloudTableClient"/> or <see cref="CloudBloblClient"/> or <see cref="CloudQueueClient"/>
  3.     /// Works with developer storage at address "http://127.0.0.1" or https address.
  4.     /// </summary>
  5.     public static class StorageClient
  6.     {
  7.         
  8.         public static CloudTableClient GetCloudTableClient(string endpoint, string accountName, string sharedKey)
  9.         {
  10.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
  11.             return account.CreateCloudTableClient();
  12.         }
  13.         public static CloudTableClient GetCloudTableClient(string endpoint, StorageCredentialsAccountAndKey credentials)
  14.         {
  15.             CloudStorageAccount account = GetAccount(endpoint, credentials);
  16.             return account.CreateCloudTableClient();
  17.         }
  18.  
  19.         public static CloudBlobClient GetCloudBlobClient(string endpoint, string accountName, string sharedKey)
  20.         {
  21.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
  22.             return account.CreateCloudBlobClient();
  23.         }
  24.         public static CloudBlobClient GetCloudBlobClient(string endpoint, StorageCredentialsAccountAndKey credentials)
  25.         {
  26.             CloudStorageAccount account = GetAccount(endpoint, credentials);
  27.             return account.CreateCloudBlobClient();
  28.         }
  29.  
  30.         public static CloudQueueClient GetCloudQueueClient(string endpoint, string accountName, string sharedKey)
  31.         {
  32.             CloudStorageAccount account = GetAccount(endpoint, accountName, sharedKey);
  33.             return account.CreateCloudQueueClient();
  34.         }
  35.         public static CloudQueueClient GetCloudQueueClient(string endpoint, StorageCredentialsAccountAndKey credentials)
  36.         {
  37.             CloudStorageAccount account = GetAccount(endpoint, credentials);
  38.             return account.CreateCloudQueueClient();
  39.         }
  40.         
  41.         private static bool IsDevelopmentStorage(string endpoint)
  42.         {
  43.             const string devEndpoint = "http://127.0.0.1";
  44.             return (endpoint != null && endpoint.Length >= devEndpoint.Length &&
  45.                     devEndpoint.ToLower().Substring(0, devEndpoint.Length) == devEndpoint);
  46.         }
  47.         private static CloudStorageAccount GetAccount(string endpoint, StorageCredentialsAccountAndKey credentials)
  48.         {
  49.             CloudStorageAccount account = null;
  50.             if (IsDevelopmentStorage(endpoint))
  51.             {
  52.                 account = CloudStorageAccount.DevelopmentStorageAccount;
  53.             }
  54.             else
  55.             {
  56.                 account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
  57.             }
  58.             return account;
  59.         }
  60.         private static CloudStorageAccount GetAccount(string endpoint, string accountName, string sharedKey)
  61.         {
  62.             CloudStorageAccount account = null;
  63.             if (IsDevelopmentStorage(endpoint))
  64.             {
  65.                 account = CloudStorageAccount.DevelopmentStorageAccount;
  66.             }
  67.             else
  68.             {
  69.                 var credentials = new StorageCredentialsAccountAndKey(accountName, sharedKey);
  70.                 account = new CloudStorageAccount(credentials, null, null, new Uri(endpoint));
  71.             }
  72.             return account;
  73.         }
  74.     }

I then changed the providers where they create the cloud client.

In the TableStorageRoleProvider I update the following line:

_tableStorage = new CloudTableClient(baseUri, info);

To:

_tableStorage = StorageClient.GetCloudTableClient(baseUri, accountName, sharedKey);

The other providers are similar.

That’s it!

Wednesday, November 24, 2010

The Heading “Libraries” automatically added through content import

Problem

When exporting and importing a single Document Library using the Import-SPWeb command, the Heading "Libraries" is automatically added to the site navigation if it is not there. Exporting then re importing to the same site or another site always adds the heading. Seems to be no option to stop this behaviour.

clip_image001

Solution

As I don’t want the heading Libraries to be added I guess I can either delete them manually or build a PowerShell script that moves through all the SPWebs and deletes them.  Not a great solution and this really should be controlled via parameters on Import-SPWeb.  Hey Microsoft, can you sort that for me please?  Thanks.

Wednesday, November 17, 2010

Content Migration and Taxonomy Fields

 

I have been using Import-SPWeb and Export-SPWeb to migrate content from different environments and found an issue if the sites use Taxonomy Fields.

The problem in a nutshell

The content is exported from a site using the PowerShell command Export-SPWeb or with code. The site contains Taxonomy fields included in content types. The taxonomy fields are wired to a Managed Metadata Service Term Store. The taxonomy fields have id's to both a TaxonomyHiddenList at the root web level and to the Term stores themselves. These ID's are migrated along with the list. However in the next environment you try to import the content to the Managed Metadata Service has a different ID for the Term Store, and as such the taxonomy fields fail to work properly.

Note: I'm not sure if this is an issue if you use the default Managed Metadata Service, I am using a new one created from a PowerShell script.

Visible Issues

If the site was accessed as an anonymous user, some pages would error with:

 image

The error message might as well just say “Something bad’s happened, but you have to figure out what. Good luck!”  (Well, I could turn off custom errors but that would be too easy and harder to make fun of!)

So the Slightly more helpful are the trace logs which showed:

"Hidden list not found, creating new one"

"Creating taxonomy hidden list"

"Unknown SPRequest error occurred. More information: 0x80070005"

"System.ArgumentException: Value does not fall within the expected range."

Strangely, the first page would access correctly under anonymous and subsequent requests to other pages would fail.  I have a theory about why but wild speculation is probably best left unsaid.

Other Symptoms

The publishing root web contains a list called "TaxonomyHiddenList". This list is used by the taxonomy fields to store terms in use for the site. After using Import-SPWeb many extra entries where added to this list with an empty title.

If you use SharePoint Manager (very useful BTW), you will see a bunch of entries with “(no title)”.  As far as I can see this shouldn’t happen, normally when the taxonomy fields create these entries they include the label as the title. 

image

The above screen shot of SharePoint Manager shows the bad entries.  Notice as well that there are lots, 64 in this example.  If this was working properly you should see the following:

image

In this screen shot the entries have a valid title.  These values originally come from the terms in the term store.  These entries where created by a powershell script and added to the term store, they are some of the ones required for a dublin core and AGLS metadata implementation.  (Which is probably about as interesting as xml namespaces or writing documentation but I felt I had to explain them anyway)

The cause

If you look at the definition of the Microsoft.SharePoint.Taxonomy.TaxonomyField you will see a property called SspId. This property is the Guid of the Term Store that its connected to. After importing, this value points to the old term store. It also has a side effect of creating bad entries in the /Lists/TaxonomyHiddenList list and is likely the cause of the entries in the trace logs. The data in the field also points to the bad entry within that list rather than the entry it should point to. There is a good write up on how the entries are created here so I won't go into that part too much.

The fix

The fix for this is much simpler than tracking it down I assure you. After many frustrating hours and some stern words aimed directly at SharePoint I managed to coble together a script that suitably cleans my site collections after content migration.

The following is a PowerShell script that will clean up your publishing site.  Use at your own risk. Backup your data, etc, etc! Wish you the best of luck! Like a Microsoft error message Smile

Note: This script will utilise the field definitions in the root web.  If you are migrating an entire web then you should fix your content types prior to running this script.

Code Snippet
  1. #Script: CleanTaxonomyFields.ps1
  2. #Author: Tim Wheeler (http://tjwheeler.blogspot.com/)
  3. #Created: 17/11/2010
  4. #Notes:
  5. #When content in imported into a new environment the Term Store ID is kept on the
  6. #taxonomy fields.  This causes a problem as the new term store has a different id.
  7. #These functions clean up the taxonomy list which has bad entries following an import
  8. #and modify all the items and resets the correct values.
  9. param
  10.     (    $siteCollectionUrl = (Read-Host "Please enter a site collection url") )
  11.     
  12. #region SharePoint Snappin Setup
  13. $snapin="Microsoft.SharePoint.PowerShell"
  14. if (get-pssnapin $snapin -ea "silentlycontinue") {
  15.     write-host -f Green "PSsnapin $snapin is loaded"
  16. }
  17. else {
  18.     if (get-pssnapin $snapin -registered -ea "silentlycontinue") {
  19.         write-host -f Green "PSsnapin $snapin is registered"
  20.         Add-PSSnapin $snapin
  21.         write-host -f Green "PSsnapin $snapin is loaded"
  22.     }
  23.     else {
  24.         write-host -f Red "PSSnapin $snapin not found"
  25.     }
  26. }
  27. #endregion
  28.     
  29. #region Data Clean Up
  30.  
  31. function CleanTaxonomyList($site)
  32. {
  33.     Write-Host "Cleaning /Lists/TaxonomyHiddenList"
  34.     $web = $site.RootWeb
  35.       $taxlist =$web.GetList("Lists/TaxonomyHiddenList")
  36.     for($count = $taxlist.Items.Count - 1; $count -ge 0; $count--)
  37.     {
  38.          $item = $taxlist.Items[$count]
  39.         if([string]::IsNullOrEmpty($item.Title))
  40.         {
  41.             $item.Delete()
  42.             if($?)
  43.             {
  44.                 #Write-Host "Deleted Item with bad title"
  45.             }
  46.             else
  47.             {
  48.                 Write-Host -ForegroundColor Red "Failed to deleted Item with bad title"
  49.             }
  50.         }
  51.         
  52.     }
  53. }
  54. #This function will reset the term store id if it is wrong, and will fix the WssId.
  55. #If the correct term cannot be located, the default one will be used.
  56. function ResetTaxonomyDefaults($site, [Microsoft.SharePoint.Publishing.PublishingWeb] $web)
  57. {
  58.     $txs = New-Object "Microsoft.SharePoint.Taxonomy.TaxonomySession" -ArgumentList $site
  59.    $pages = $web.GetPublishingPages()
  60.    foreach($page in $pages)
  61.    {
  62.         Write-Host -ForegroundColor Cyan "Checking publishing page " $page.Title
  63.        if($page.ListItem.File.CheckOutStatus -ne "None")
  64.         {
  65.             $page.CheckIn("Checked in by data clean process");
  66.         }
  67.         $page.CheckOut();
  68.        foreach ($field in $page.ListItem.Fields)
  69.        {
  70.            if($field.GetType().Name -eq "TaxonomyField")
  71.            {
  72.                $taxField = [Microsoft.SharePoint.Taxonomy.TaxonomyField] $field
  73.                 
  74.                 Write-Host "Found field to update:" $taxField.Title
  75.                 $currentValue = $page.ListItem.Properties[$taxField.InternalName]
  76.                 Write-Host "Current Value is" $currentValue
  77.                 $templateField = $page.ListItem.ParentList.ParentWeb.Site.RootWeb.Fields[$field.Id]
  78.                 $defaultValue = $templateField.DefaultValue
  79.                 $termStore = $txs.TermStores[$templateField.SspId]
  80.                 $termSet = $termStore.GetTermSet($templateField.TermSetId)
  81.                 if($taxField.SspId -ne $templateField.SspId)
  82.                 {
  83.                     Write-Host "TaxField SspId is not correct, updating"
  84.                     $taxField.SspId = $templateField.SspId
  85.                     $taxField.Update()
  86.                 }
  87.                 if($taxField.TermSetId -ne $templateField.TermSetId)
  88.                 {
  89.                     Write-Host "TaxField TermSetId is not correct, updating"
  90.                     $taxField.TermSetId = $templateField.TermSetId
  91.                     $taxField.Update()
  92.                 }
  93.                 $fieldValue = $templateField.GetFieldValue($currentValue)
  94.                   if(($fieldValue.GetType().Name -eq "TaxonomyFieldValue") -and($fieldValue -eq $null -or [string]::IsNullOrEmpty($fieldValue.TermGuid)))
  95.                 {
  96.                     $fieldValue = $templateField.GetFieldValue($defaultValue)
  97.                 }
  98.                   if(($fieldValue.GetType().Name -ne "TaxonomyFieldValue") -and($fieldValue[0] -eq $null -or [string]::IsNullOrEmpty($fieldValue[0].TermGuid)))
  99.                 {
  100.                     $fieldValue = $templateField.GetFieldValue($defaultValue)
  101.                 }
  102.                 if($fieldValue.GetType().Name -eq "TaxonomyFieldValue")
  103.                 {
  104.                     try
  105.                     {
  106.                         $term = $termSet.GetTerm($fieldValue.TermGuid)
  107.                     }
  108.                     catch
  109.                     {
  110.                           Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
  111.                     }
  112.                 }
  113.                 else
  114.                 {
  115.                     try
  116.                     {
  117.                         $term = $termSet.GetTerm($fieldValue[0].TermGuid)
  118.                     }
  119.                     catch
  120.                     {
  121.                           Write-Host -red ("Failed to update field {0} for page {1} in web {2}"-f $taxField.InternalName, $page.Title, $web.Url)
  122.                     }
  123.                 }
  124.                 $taxField.SetFieldValue($page.ListItem, $term)
  125.            }
  126.        }
  127.         $page.ListItem.Update()
  128.        $page.CheckIn("Data clean process")
  129.    }
  130.  
  131. }
  132. function CleanSiteCollection ($siteColUrl)
  133. {
  134.     [Microsoft.SharePoint.SPSite] $site = get-spsite -Limit ALL | where-object {$_.Url -ieq $siteColUrl}
  135.     if($site -eq $null)
  136.     {
  137.         Write-Host -ForegroundColor Red "Unable to find site collection"
  138.         throw "Unable to find site collection"
  139.     }
  140.     CleanTaxonomyList $site
  141.    $site | Get-SPWeb -limit all | ForEach-Object {
  142.             #Check to see if site is a publishing site
  143.             if ([Microsoft.SharePoint.Publishing.PublishingWeb]::IsPublishingWeb($_))
  144.             {
  145.                 Write-Host "Cleaning pages in `"$($_.Title)`" site."
  146.                 #Get the Publishing Web and pages within it
  147.                 $publishingWeb = [Microsoft.SharePoint.Publishing.PublishingWeb]::GetPublishingWeb($_)
  148.                 ResetTaxonomyDefaults $site $publishingWeb    
  149.             }
  150.             $_.Dispose()
  151.         }
  152.     $site.Dispose()
  153. }
  154. #endregion
  155. CleanSiteCollection $siteCollectionUrl