Return SPListItems using CSOM and PowerShell without writing CAML


I recently tweeted about my triumph when trying to accomplish returning all SharePoint List Items without the use of CAML or LINQ. The reason I wanted to do this might be strange, but I’ll try to explain my process and methodology and why I got to a point of wanting to make SharePoint do things it didn’t want to do. 🙂

As a very advanced SharePoint Scripter, I do a TON of PowerShell. If you’ve read this blog, follow me on Twitter or talk to me in person you probably knew that. Having said that, there are times when I want to iterate through all list items – either looking for a match or just to return all values. It’s not super common but it happens. Well it’s a very easy thing to do with server side code (read: PowerShell and the Microsoft.SharePoint.PowerShell snap-in).

Getting List Items using standard PowerShell

$web = Get-SPWeb http://someSiteUrl
$list = $web.Lists["SomeList"]
$list.Items #do something with them here

As you see on line 3, there is an Items collection property on the List object. This is great, end of story – IF you’re using server-side PowerShell code. However, I’ve been focusing a lot lately on integrating PowerShell into the CSOM space – specifically to manage SharePoint Online with PowerShell. If you’ve written any CSOM code, you know that things are done a little differently. Here’s an example of how to return a list item using PowerShell on the client-side – note that I’ve omitted about 12 lines of code that must take place before this to load the Microsoft.SharePoint.Client* assemblies, create ClientContext, get the SPWeb, etc.:

Getting a List Item using PowerShell Client-Side Code

$list = $GLOBAL:Web.Lists.GetByTitle("BigListLotsOfItems")
$item = $list.GetItemById(5)
$GLOBAL:Context.Load($item)
$GLOBAL:Context.ExecuteQuery()
$item["Title"]

That’s great, but if I wanted to return all items – the Client Object Model won’t give me List.Items. In reading MSDN, there are C# examples of how to write CAML queries to return all list items – but the whole point is that I don’t want to write CAML. Here’s the C# example – which I could have certainly translated to PowerShell if I wanted to do so:

Using C# to get all List Items

// Starting with ClientContext, the constructor requires a URL to the
// server running SharePoint.
ClientContext context = new ClientContext("http://SiteUrl");

// Assume the web has a list named "Announcements".
List announcementsList = context.Web.Lists.GetByTitle("Announcements");

// This creates a CamlQuery that has a RowLimit of 100, and also specifies Scope="RecursiveAll"
// so that it grabs all list items, regardless of the folder they are in.
CamlQuery query = CamlQuery.CreateAllItemsQuery(100);
ListItemCollection items = announcementsList.GetItems(query);

// Retrieve all items in the ListItemCollection from List.GetItems(Query).
context.Load(items);
context.ExecuteQuery();
foreach (ListItem listItem in items)
{
    // We have all the list item data. For example, Title.
    label1.Text = label1.Text + ", " + listItem["Title"];
}

So that’s great, but here’s how I went about getting all list items without writing CAML! I’ll break it down section by section, but if you’re impatient the whole solution is at the bottom. 🙂

Creating our Context

The first thing we need to do when working with SharePoint from the Client Side is create ClientContext. To do so using PowerShell, here is some example code:

$GLOBAL:Context = New-Object Microsoft.SharePoint.Client.ClientContext("http://someWebUrl")
$GLOBAL:Credentials = Get-Credential -UserName $EmailAddress -Message "Please enter your Office 365 Password"
$Context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Credentials.UserName,$Credentials.Password)
$GLOBAL:Web = $GLOBAL:Context.Web
$GLOBAL:Context.Load($GLOBAL:Web)
$GLOBAL:Context.ExecuteQuery()

Getting our SPWeb

Once we’ve created our ClientContext variable, we’ve already got the SPWeb. But just for transparency, here’s how you would/could access methods and properties of the SPWeb object:

$GLOBAL:Web.Title #returns Web.Title
$GLOBAL:Web.Description #returns Web.Description

Getting our SPList

Now, to get the List object – we’ll do the following:

$SPList = $GLOBAL:Web.Lists.GetByTitle("BigListLotsOfItems") #Note that BigListLotsOfItems is my List Title

Using the For loop to iterate through all items

And this is where the magic happens as they say. For anyone who has done any level of development or scripting, you know that the best way to loop while also allowing for modification of the array with which you’re looping is to use a For loop.

For ($i=0; $i -le $SPList.ItemCount; $i++)
{
}

This is a pretty standard implementation of the For loop, what I want to call out though is the use of Try/Catch/Finally. As a dabbling developer, I’m not sure if this is the intended use of Try/Catch/Finally – so if this is bad practice forgive me. 🙂

Essentially I start out by creating the ListItem variable, and then calling the Load method of the ClientContext variable. I do that before entering the Try/Catch/Finally. I then use Try {} to run the Context.ExecuteQuery method, and if that fails the loop will go into the Catch {} block – where I will increment the array by 1 ($i++) and then run the GetItemById(), Load() and ExecuteQuery() methods against the next $i in the loop.

$ListItem = $SPList.GetItemById($i)
$GLOBAL:Context.Load($ListItem)
    Try
    {
        $GLOBAL:Context.ExecuteQuery()
    }
    Catch
    {
        $i++
        $ListItem = $SPList.GetItemById($i)
        $GLOBAL:Context.Load($ListItem)
        $GLOBAL:Context.ExecuteQuery()
    }

Why did I do this? Well it’s really pretty simple. Since I have to use the GetItemById() method, I have to know the ListItemID of each ListItem. If someone has deleted an item, the ItemCount property won’t match the highest ID in the list – right? So if there are 5 items but someone deleted a few at one point or another – the highest ID might be 7 or 8, without incrementing by 1 on a failure, we’ll never get to 7 or 8…

And so now that we’ve discussed all of that, here’s the final example script.

The Final Solution

$GLOBAL:Context = New-Object Microsoft.SharePoint.Client.ClientContext("http://someWebUrl")
$GLOBAL:Credentials = Get-Credential -UserName $EmailAddress -Message "Please enter your Office 365 Password"
$Context.Credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($Credentials.UserName,$Credentials.Password)
$GLOBAL:Web = $GLOBAL:Context.Web
$GLOBAL:Context.Load($GLOBAL:Web)
$GLOBAL:Context.ExecuteQuery()
$SPList = $GLOBAL:Web.Lists.GetByTitle("BigListLotsOfItems")
For ($i=0; $i -le $SPList.ItemCount; $i++)
{
    $ListItem = $SPList.GetItemById($i)
    $GLOBAL:Context.Load($ListItem)
    Try
    {
        $GLOBAL:Context.ExecuteQuery()
    }
    Catch
    {
        $i++
        $ListItem = $SPList.GetItemById($i)
        $GLOBAL:Context.Load($ListItem)
        $GLOBAL:Context.ExecuteQuery()
    }
    Finally
    {
        #Return item here and do something with it.
    }
}

And that – my friends – is how you can get all List Items from a SharePoint list using PowerShell and CSOM. Hopefully this is helpful for someone else.

Create a SharePoint Site (SPWeb) using REST in SPD 2013 Workflow


I have been struggling with an issue for several days, and it’s one I’ve been trying to solve for an upcoming SharePoint Saturday presentation. That issue is creating SharePoint sub sites/SPWebs automatically via the “Call HTTP Web Service” action in SharePoint Designer 2013 workflow. There were several sources of information that ultimately provided the solution, so I will be sure to credit those folks in this article.

There are several steps which are required to make this happen, and the coolest part is; this will work both on-premise and in Office 365/SharePoint Online!
Continue reading “Create a SharePoint Site (SPWeb) using REST in SPD 2013 Workflow”

Workflow failed on start (retrying) when anonymous users start workflows


Here is an interesting issue and subsequent workaround – anonymous users can create list items (if granted the appropriate permissions), but workflows will not start for anonymous users.

If you do some research on this topic, you’ll see lots of people trying to allow this, so that emails can be sent for example – but the answer is almost always “here is some code you can deploy.” That’s an unfortunate answer, as not all of us have the ability or skillset to write and deploy Visual Studio code to our environments or our client’s environments.

However, after lots of trial and error I came up with a solution using PowerShell.

Essentially, we need to do the following:

  1. Grant anonymous users the ability to add items
  2. Setup a SharePoint Designer workflow to do the e-mailing, or what have you – set the workflow to start when items are changed, not when items are created…
  3. Configure a PowerShell script to run against the site or sites so that the items get updated if they have not yet had their Workflow Status updated to “Complete”

The script is currently configured to iterate through a single site collection and all of it’s subwebs, looking for any “GenericList” (Custom List) that has items which have not yet been marked as “Completed” by the workflow engine…

Here is my PowerShell Script to update items:

Start-SPAssignment -Global
$Site = Get-SPSite http://siteurl
$Webs = $Site.AllWebs
	foreach ($Web in $Webs) {
		foreach ($list in $Web.Lists | where {$_.BaseType -eq "GenericList"}){
			foreach ($item in $list.Items | where {$_.Workflows.InternalState -ne "Completed"}) {
			$item.Update()
			}
		$list.Update()	
		}
	$Web.Dispose()		
	}
$Site.Dispose()
Stop-SPAssignment -Global

Setup a Windows Scheduled Task to run this every half-hour or so, and it will iterate through your site – kicking off any yet-to-be-started workflows under the context of whatever account you use in the Scheduled Task.

Disclaimer: This may cause heavy utilization of server resources if you have a large site collection, lots of lists, big lists, or if you run this against multiple web applications. I’m sure there are other (perhaps better) solutions, but this works for me. No warranty or guarantee is implied. 🙂

Enjoy!
RD

Remove ‘ReadOnlyField’ lock from columns created in InfoPath using PowerShell


Have you ever created an InfoPath Form Template and published fields from the template to SharePoint Columns in your content type?

I’m sure you have, otherwise you probably wouldn’t be reading this.

Essentially, the columns will be grayed out in the list settings:

And if you try to set the value of the field using SharePoint Designer, you won’t see it in your list of available columns:

However, luckily we have PowerShell – and we can simply edit the ReadOnlyField value from $true to $false – thus allowing us to edit this field. Here’s how!

$web = Get-SPWeb http://weburl
$list = $web.Lists["List Name"]
$field = $list.Fields["Field Name"]
$field.ReadOnlyField = $false
$field.Update()
$list.Update()
$web.Update()
$web.Dispose()

After doing so, the column is not only editable in the browser – we can choose it in Designer:

Get a Web, List and Library Inventory using PowerShell


A colleague of mine recently asked me “Ryan, can you give me some PowerShell code that can give me a list of all sites and sub sites as well as all lists and libraries within each of those sites – for an entire web application?”

“Of course”, I said…

I had some other scripts and functions that were similarly constructed, so I simply took one that was close and adapted it to make it work.

This function, which I’ve called “Get-SPSiteInventory” will run against either an entire Web Application (using the -WebApplication switch param) or a single Site Collection (using the -SiteCollection switch param).

I’ve tested this both by sending the output straight to a file (using the Out-File cmdlet) as well as just running in the shell – both work pretty nicely.

I’ve excluded comment-based help for better readability, and the syntax is as follows…

To run against a site collection:

Get-SPSiteInventory -Url http://spsite -SiteCollection

To run against a web application:

Get-SPSiteInventory -Url http://spwebapp -WebApplication

The entire function:

function Get-SPSiteInventory {
Param(
[string]$Url,
[switch]$SiteCollection,
[switch]$WebApplication
)
Start-SPAssignment -Global
	if ($SiteCollection) {
		$site = Get-SPSite $Url
		$allWebs = $site.allwebs
		foreach ($spweb in $allWebs) {
			" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "
			$spweb.Url
			$spweb.Lists | select Title, BaseType
			$spweb.dispose()
		}
		$site.dispose()
	} 
	elseif ($WebApplication) {
		$wa = Get-SPWebApplication $Url
		$allSites = $wa | Get-SPSite -Limit all
		foreach ($spsite in $allSites) {
			$allWebs = $spsite.allwebs
			foreach ($spweb in $allWebs) {
			" - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "
			$spweb.Url
			$spweb.Lists | select Title, BaseType
			$spweb.dispose()
			}
		}
	}
Stop-SPAssignment -Global
}