NullReferenceException in ResolveGrammarActivity

Today I ran into a problem with the ResolveGrammarActivity. I'm not sure if it's a bug, but before I post it to the FIM forum, I thought I'd describe it here so that I'm reminded to post the solution (and to host an image).

I know this used to work in ILM 2 RC0, but in FIM 2010 RC1, I'm getting a NullReferenceException. Here's the deal; I can write a parameter to the workflow dictionary, but when I try to use that in a ResolveGrammarActivity, I get the exception.

Here's an example of adding myself to the workflow dictionary (WorkflowData):



So far, so good. Now I try passing [//WorkflowData/JoeZamora] into the grammar resolver. Here's my debug log:

2010-01-25 17:18:10,477 --6-- DEBUG [Ensynch.FIM.Workflow.Activities.ChangeAttributeActivity]

Source Class : System.Workflow.ComponentModel.Activity
Source Instance : 7. Remove Joe from Group Members
Source Method : RaiseEvent
Current user : INFO\svc.fimws

Passing these data into the ResolveGrammarActivity:
NewGrammarExpression : [//WorkflowData/JoeZamora]
NewResolvedExpression :
NewWorkflowDictionaryKey :

2010-01-25 17:18:10,535 --6-- ERROR [Ensynch.FIM.Workflow.Activities.ChangeAttributeActivity]

Source Class : System.Workflow.ComponentModel.ActivityExecutor`1[T]
Source Instance : 7. Remove Joe from Group Members
Source Method : HandleFault
Current user : INFO\svc.fimws

System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.ResourceManagement.WFActivities.Resolver.GetDisplayStringFromGuid(Guid id, String[] expansionAttributes)
at Microsoft.ResourceManagement.WFActivities.Resolver.ReplaceGuidWithTemplatedString(Match m)
at System.Text.RegularExpressions.RegexReplacement.Replace(MatchEvaluator evaluator, Regex regex, String input, Int32 count, Int32 startat)
at System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator)
at Microsoft.ResourceManagement.WFActivities.Resolver.GetStringAttributeValue(Object attribute)
at Microsoft.ResourceManagement.WFActivities.Resolver.ResolveEvaluatorWithoutAntiXSS(Match m)
at Microsoft.ResourceManagement.WFActivities.Resolver.ResolveEvaluatorForBodyWithAntiXSS(Match m)
at System.Text.RegularExpressions.RegexReplacement.Replace(MatchEvaluator evaluator, Regex regex, String input, Int32 count, Int32 startat)
at System.Text.RegularExpressions.Regex.Replace(String input, MatchEvaluator evaluator)
at Microsoft.ResourceManagement.WFActivities.Resolver.ResolveBody(String input)
at Microsoft.ResourceManagement.Workflow.Hosting.ResolverEvaluationServiceImpl.ResolveLookupGrammar(Guid requestId, Guid targetId, Guid actorId, Dictionary`2 workflowDictionary, Boolean encodeForHTML, String expression)
at Microsoft.ResourceManagement.Workflow.Activities.ResolveGrammarActivity.Execute(ActivityExecutionContext executionContext)
at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(T activity, ActivityExecutionContext executionContext)
at System.Workflow.ComponentModel.ActivityExecutor`1.Execute(Activity activity, ActivityExecutionContext executionContext)
at System.Workflow.ComponentModel.ActivityExecutorOperation.Run(IWorkflowCoreRuntime workflowCoreRuntime)
at System.Workflow.Runtime.Scheduler.Run()

Hmmm, not a very helpful message. I'm pretty sure this is a bug, because ordinarily you try to handle NullReferenceExceptions in code that's exposed to the public.

Anyway, I'll post this to the forum and get back to you.

Update: I posted this on the FIM forum to see if it got any bites:

http://social.technet.microsoft.com/Forums/en-US/ilm2/thread/39d887bf-638c-4539-8f0e-afd9c0ff4490

Joe Schulman mentioned that someone already logged a similar problem:

https://connect.microsoft.com/site433/feedback/ViewFeedback.aspx?FeedbackID=523776&wa=wsignin1.0#tabs

If you run into this same problem, please visit the Connect link and vote it as important!

NullReferenceException in EnumerationResultEnumerator.Dispose()

Still working with the unsupported web service client for RC1, and I ran into the following error:

System.NullReferenceException

Object reference not set to an instance of an object.

at Microsoft.ResourceManagement.Client.EnumerationResultEnumerator.Dispose() in C:\FIM2010Dev\Microsoft.ResourceManagement.Samples\Microsoft.ResourceManagement.Client\EnumerationResultEnumerator.cs:line 46

The problem was pretty easy to find, but I thought I'd at least change the code to throw a more helpful exception. The problem happened because I naively tried to use the LINQ methods Count() and First() consecutively:

IEnumerable<RmResource> objects =
    client.Enumerate(xpath, selection.ToArray());
if (objects != null && objects.Count() > 0)
{
    string result = objects.First()[ATTRIBUTE_DISPLAY_NAME].Value.ToString();
    if (!string.IsNullOrEmpty(result))
    {
        displayName = result;
        break;
    }
}

However, much like LINQ to SQL behavior, the queries are run on-the-fly as the results are enumerated, and then they're disposed. So, you guessed it, we can't enumerate the results more than once (or at least we should avoid it). You've probably seen this exception from the LINQ to SQL libraries, "The query results cannot be enumerated more than once."

Here's my modified code for the EnumerationResultEnumerator class. I'm throwing a more helpful exception with the message above. I've highlighted my changes:

using System;
using System.Collections.Generic;
using System.Xml.Schema;
using System.Text;

using Microsoft.ResourceManagement.Client.WsEnumeration;
using Microsoft.ResourceManagement.ObjectModel;

namespace Microsoft.ResourceManagement.Client
{
    class EnumerationResultEnumerator : IEnumerator<RmResource>, IEnumerable<RmResource>
    {
        WsEnumerationClient client;
        List<RmResource> results;
        int resultIndex;
        bool endOfSequence;
        EnumerationContext context;
        String filter;
        String[] attributes;
        RmResource current;
        RmResourceFactory resourceFactory;
        bool disposed = false;

        internal EnumerationResultEnumerator(WsEnumerationClient client, RmResourceFactory factory, String filter, String[] attributes)
        {
            results = new List<RmResource>();
            this.client = client;
            this.filter = filter;
            this.resourceFactory = factory;
            this.attributes = attributes;
        }

        #region IEnumerator<RmResource> Members

        public RmResource Current
        {
            get { return current; }
        }

        #endregion

        #region IDisposable Members

        public void Dispose()
        {
            if (!disposed)
            {
                this.context = null;
                this.results.Clear();
                this.results = null;
                this.disposed = true;
            }
        }

        #endregion

        #region IEnumerator Members

        object System.Collections.IEnumerator.Current
        {
            get { return current; }
        }

        public bool MoveNext()
        {
            if (disposed)
            {
                throw new InvalidOperationException("The query results cannot be enumerated more than once.");
            }

            lock (this.client)
            {
                if (resultIndex < results.Count)
                {
                    this.current = results[resultIndex++];
                    return true;
                }
                else
                {
                    PullResponse response;
                    if (this.context == null)
                    {
                        if (resultIndex > 0)
                        {
                            // case: previous pull returned an invalid context
                            return false;
                        }
                        EnumerationRequest request = new EnumerationRequest(filter);
                        if (attributes != null)
                        {
                            request.Selection = new List<string>();
                            request.Selection.AddRange(this.attributes);
                        }
                        response = client.Enumerate(request);
                        this.endOfSequence = response.EndOfSequence != null;
                    }
                    else
                    {
                        if (this.endOfSequence == true)
                        {
                            // case: previous pull returned an end of sequence flag
                            this.current = null;
                            return false;
                        }
                        PullRequest request = new PullRequest();
                        request.EnumerationContext = this.context;
                        response = client.Pull(request);
                    }
                  
                    if (response == null)
                        return false;
                    resultIndex = 0;
                    this.results = resourceFactory.CreateResource(response);
                    this.context = response.EnumerationContext;
                    this.endOfSequence = response.IsEndOfSequence;
                    if (this.results.Count > 0)
                    {
                        this.current = results[resultIndex++];
                        return true;
                    }
                    else
                    {
                        this.current = null;
                        return false;
                    }
                }
            }
        }

        public void Reset()
        {
            if (!disposed)
            {
                this.results.Clear();
                this.context = null;
            }
        }

        #endregion

        #region IEnumerable<RmResource> Members

        public IEnumerator<RmResource> GetEnumerator()
        {
            return this;
        }

        #endregion

        #region IEnumerable Members

        System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
        {
            return this;
        }

        #endregion
    }
}

Paolo Tedesco's changes for object count in enumeration responses

As one final note, Paolo has posted code for including the object count in enumeration responses. I haven't tried it yet, but it looks like something I could have used here. Here's the link (note that I'm also posting links to my changes on this thread):

http://social.technet.microsoft.com/Forums/en-US/ilm2/thread/ffc16720-0dfb-4131-b676-9225f15b4f72?prof=required

Multi-valued attributes aren't multi-valued

I'm back on FIM after a brief hiatus, and I've begun updating my FIM Query Tool with the new unsupported web service client for RC1.

Today I noticed that my multi-valued attributes didn't have multiple values. I'm calling Enumerate on the DefaultClient to check the computed members of a Group. After some digging, I discovered that the attributes weren't recognized as multi-valued because I didn't refresh the schema after instantiating the DefaultClient.

Here's a code snippet from the unsupported WS client sample program:

// First need to construct the client
// We will assume all default contracts
DefaultClient client = new DefaultClient();
// We set the client credentials since often the test cases or client apps run under different accounts
client.ClientCredential = Credential.GetAdminCredential();
// We refresh the schema so that the web service put operations are better informed
client.RefreshSchema();

I interpreted the last comment as, "We only need to call RefreshSchema() when using Put operations." Since I'm only using Enumerate/Pull operations, I just left it out of my code. Well, I was wrong. Turns out that the RmFactory needs a schema refresh before it can determine whether an attribute is multi-valued.

Since this is a potential pitfall every time you use the DefaultClient, I decided to refactor its constructors. Originally, there were three constructors. I added three more that accept an additional NetworkCredential, and now they all call RefreshSchema(). Of course, if you use the original constructors (without the NetworkCredential), they'll use the caller's credentials. Since the FIM Query Tool is a Windows app, it'll use your credentials.

Here are the refactored constructors (note the additional helper method):

public DefaultClient() : this(null)
{
}

public DefaultClient(NetworkCredential clientCredential)
{
    this.wsTransferClient = new WsTransferClient();
    this.wsTransferFactoryClient = new WsTransferFactoryClient();
    this.wsEnumerationClient = new WsEnumerationClient();
    this.mexClient = new MexClient();

    this.resourceFactory = new RmResourceFactory();
    this.requestFactory = new RmRequestFactory();

    init(clientCredential);
}

public DefaultClient(
    String wsTransferConfigurationName,
    String wsTransferFactoryConfigurationName,
    String wsEnumerationConfigurationName,
    String mexConfigurationName
    ) : this(
    null,
    wsTransferConfigurationName,
    wsTransferFactoryConfigurationName,
    wsEnumerationConfigurationName,
    mexConfigurationName
    )
{
}

public DefaultClient(
    NetworkCredential clientCredential,
    String wsTransferConfigurationName,
    String wsTransferFactoryConfigurationName,
    String wsEnumerationConfigurationName,
    String mexConfigurationName
    )
{
    this.wsTransferClient = new WsTransferClient(wsTransferConfigurationName);
    this.wsTransferFactoryClient = new WsTransferFactoryClient(wsTransferFactoryConfigurationName);
    this.wsEnumerationClient = new WsEnumerationClient(wsEnumerationConfigurationName);
    this.mexClient = new MexClient(mexConfigurationName);

    this.resourceFactory = new RmResourceFactory();
    this.requestFactory = new RmRequestFactory();

    init(clientCredential);
}

public DefaultClient(
    String wsTransferConfigurationName,
    String wsTransferEndpointAddress,
    String wsTransferFactoryConfigurationName,
    String wsTransferFactoryEndpointAddress,
    String wsEnumerationConfigurationName,
    String wsEnumerationEndpointAddress,
    String mexConfigurationName,
    String mexEndpointAddress
    ) : this(
    null,
    wsTransferConfigurationName,
    wsTransferEndpointAddress,
    wsTransferFactoryConfigurationName,
    wsTransferFactoryEndpointAddress,
    wsEnumerationConfigurationName,
    wsEnumerationEndpointAddress,
    mexConfigurationName,
    mexEndpointAddress
    )
{
}

public DefaultClient(
    NetworkCredential clientCredential,
    String wsTransferConfigurationName,
    String wsTransferEndpointAddress,
    String wsTransferFactoryConfigurationName,
    String wsTransferFactoryEndpointAddress,
    String wsEnumerationConfigurationName,
    String wsEnumerationEndpointAddress,
    String mexConfigurationName,
    String mexEndpointAddress
    )
{
    this.wsTransferClient = new WsTransferClient(wsTransferConfigurationName, wsTransferEndpointAddress);
    this.wsTransferFactoryClient = new WsTransferFactoryClient(wsTransferFactoryConfigurationName, wsTransferFactoryEndpointAddress);
    this.wsEnumerationClient = new WsEnumerationClient(wsEnumerationConfigurationName, wsEnumerationEndpointAddress);
    this.mexClient = new MexClient(mexConfigurationName, mexEndpointAddress);

    this.resourceFactory = new RmResourceFactory();
    this.requestFactory = new RmRequestFactory();

    init(clientCredential);
}

private void init(NetworkCredential clientCredential)
{
    if (clientCredential != null)
    {
        ClientCredential = clientCredential;
    }
    RefreshSchema();
}

Extra Credit

Can anyone tell me why there are warning messages on the following methods in the RmFactory class?

• IsMultiValued
• IsReference
• IsRequired
• RequiredAttributes

No, really, please tell me; I don't know why they're there. For example:

/// <summary>
/// DO NOT USE THIS METHOD -- FOR TESTING ONLY!
/// </summary>
/// <param name="attributeName"></param>
/// <returns></returns>
public bool IsMultiValued(RmAttributeName attributeName)
{
    RmAttributeInfo retValue = null;
    RmAttributeCache.TryGetValue(attributeName, out retValue);
    if (retValue == null)
    {
        return false;
    }
    else
    {
        return retValue.IsMultiValue;
    }
}

Auditing with the FIM Query Tool

Brad Turner recently received a question from a blog reader:

"I am interested in knowing how can we track/audit which user did a certain change on a user/group account through the ILM portal. Have you written a previous post about this issue? Do you have any information that might help me?"

There are a few ways you could approach this challenge. First, you could find all requests on an object. Here's how you can do that with the FIM Query Tool:

1. Run the FIM Query Tool and filter for "Request" object types.



2. Select the following attributes to capture in your audit:
   
   • Created Time
   • Creator
   • Display Name
   • Operation
   • Request Parameters
   • Target


3. Change the Reference Format to DisplayName, so that you're not just looking at GUIDs.



4. Finally, use the following XPath filter:
   
   /Request[Target = /Person[DisplayName = 'Joe Zamora']]
   
   To kind of translate this XPath, we're looking for Request objects whose Target matches the following condition: a Person whose display name is "Joe Zamora". In a production scenario, you'd probably want to use the object's GUID to do the search (ObjectId = '12345678-ABCD-1234-ABCD-1234567890AB'), but I use the display name to make it more readable.

One nice feature of the FIM Query Tool is that, because the results are displayed in a data grid view, you can sort results without re-running the query. Just click on a column header to sort by that column.



One additional note on the results set: to see the details of the request, you'll want to pay attention to the RequestParameters attribute. This is where you'll find which attributes were updated and their new values. This is also where the FIM Query Tools falls a bit short. The attribute is stored in XML, and isn't formatted neatly for quick review. There's a good enhancement request!

Now, this query is pretty handy, but if the object has been updated many times, you may find yourself waiting longer than you'd bargained for to see the results of the audit. Brad suggested that we use the XPath historical query functions to narrow the results set down to a certain time window.

So, the second approach is to use the "betweenTime" XPath function to plug in the time window of interest. Try this in the FIM Query Tool with the rest of the settings remaining the same as above:

betweenTime(/Request[Target = /Person[DisplayName = 'Joe Zamora']], '2008-10-31', '2008-12-31')

Voila! Now you see all the users who made updates to the object during your desired time period. Brad also mentioned a few other XPath functions that he and David Lundell presented at TEC 2009:

• allTime(filter) - Show me the objects that ever satisfied this filter


• betweenTime(filter, begin datetime, end datetime) - Show me the objects that ever satisfied this filter during the time range specified


• atTime(filter, datetime) - Show me the object that satisfied the filter at the specified date and time

David builds some good examples here:

Who were payroll admins at the precise moment of the theft?
atTime(/Person[ObjectID = /Group[DisplayName = 'Payroll Admins']/ComputedMember, '2009-02-01T00:00')

Who were the payroll admins in the merry merry month of of May?
betweenTime(/Person[ObjectID = /Group[DisplayName = 'Payroll Admins']/ComputedMember, '2008-05-01T00:00' , '2008-05-31T23:59:59')

Webinar: Geneva (aka WIF)

Ensynch will be co-presenting a webinar with Quest next week on the Geneva framework (now called Windows Identity Foundation).

When: Wednesday, July 29, 2009 10:30 to 11:30 (PST) 12:30 to 1:30 (CST) 1:30 to 2:30 (EST) Where: Web/Online Live Meeting Information will be sent to attendees Presenters: David Lundell, Identity Management Practice Leader, Ensynch Jonathan Sander IAM and Security Analyst Quest Software	Webinar: How Microsoft Geneva Streamlines Business - Learn How to Reap the Benefits of True Web  Single-Sign-On and Federation Has your organization been forced to deploy one-off solutions to solve login or compliance problems with a newly deployed technology? Are your employees tired of using multiple logins for all kinds of access needs? Having trouble managing shared resources users both inside and outside of your organization? Using open platform identity management solution Microsoft Geneva, you can save money and make your business more efficient today, and also make it more easily scalable for the future. I would like to invite you to our latest exclusive "no frills" webinar: "How Microsoft Geneva Streamlines Business," the 1st in a 4-part Identity Management Webinar Series from Ensynch's Identity Management Practice Leader and Microsoft Identity Management MVP, David Lundell, and Quest Software IAM and Security Analyst, Jonathan Sander. This webinar is designed for business leaders, and will present business value propositions for the Microsoft Geneva framework. Whether identity management is a major concern for your organization or if you are simply curious about using Microsoft Geneva as an asset to help your business, this webinar is for you. Webinar Agenda: - Yikes! The business pain points of managing lots of identities - High level discussion of Microsoft Geneva - Business value of Geneva - Gaps of the Geneva framework - Possible solutions to the gaps - ROI of Geneva versus other Single-Sign-On solutions - Geneva and the Cloud - Q & A Stay Tuned for the other three parts of this webinar series: A Technical Overview of the Microsoft Geneva Infrastructure Thursday, August 20, 2009 Using the Microsoft Geneva Framework to Solve Your Federation Needs Thursday, September 10, 2009 Accelerate Your Businesses for the Future with Microsoft Geneva and the Cloud Thursday, October 1, 2009   [Register Now]

What is SRS?

SRS is the (misused) abbreviation of SSRS ([Microsoft] SQL Server Reporting Services). Yes, that's right; an abbreviation of an acronym. Good grief, folks.

The reason for this short & sweet post is that I see SRS used all the time in certain circles. But if you Google it (or Bing it, whatever your preference), you'll find many other definitions before you come across this one (if you ever do, that is).

Another reason for this post is, of course, that I admittedly once went on a wild goose chase to figure out what someone was asking of me. :)

Introducing the FIM Query Tool

I've had a bit of downtime recently, so I decided to make good on a statement that I made to my colleagues at lunch one day, "I should create an interface for querying the ILM2 web service." Well, I just polished off a first draft and published it to CodePlex. Please take it for a test drive, kick the tires, and leave me some feedback!

FIM Query Tool

As I mention on the CodePlex site, this tool is a Windows Forms front end to the ILM2 enumeration client. It's intended to be a one-stop shop for testing XPath filters on the ILM2 web service. And although it's called 'FIM Query Tool', it's currently written for the only available version of FIM, which is ILM2 RC0. Obviously, I'm expecting this tool to evolve with the technology.

Here's a first glance at the tool.



There are a few bells & whistles on this first draft. First, it populates the object & attribute lists when you first run it, but then it caches those lists so that subsequent sessions are faster. If you create a new object or attribute, you can always refresh the schema with the corresponding buttons.

Next, it uses the extensions formerly known as TEIMO (now called MS-WSTIM) to filter the attributes returned from the web service, so that you can cut down on the SOAP message size and save a bit of time on each query.

The tool displays the results in a table, and although it's not obvious in this first draft, you can use Ctrl-A, Ctrl-C to copy all the cells so that you can paste them into Excel. The tool also gives you the raw XML for your perusing pleasure, as well as some verbose messages on separate tabs.

Finally, you can choose to dereference GUIDs when displaying the output. This means that it will resolve GUIDs to their display names, but if you choose this option, you'll get a warning that performance may be poor.

Now let me mention the biggest limitation of the first draft: there's no filter builder to help you with the XPath syntax. Thus, you're sort of on your own when typing up the XPath filter that you'd like to test. I do give you the underlying attribute name when you skate your mouse over the attributes in the list. I hope this helps you out for now.

One quick note on the application settings. You can find all of the settings in the FIMQueryTool.exe.config file. For example, the enumeration endpoint is set to http://localhost:526/... If you have a different URL for your server/port, you'll have to update this in the config file. Note that I set the SOAP message size to the max (maxReceivedMessageSize="0x7fffffff"), but you may want to tweak other settings like WsEnumerationDefaultPull (batch size).

Oh! I forgot to mention that since this project is on CodePlex, you have access to the source code. Enjoy! Try not to blow anything up. :)

As I mentioned, please download it, try it out, and leave me feedback either through the Discussions section of the CodePlex project or on this blog.