Function evaluator custom expression escape double quote

Curiously, there's no need to escape double quotes inside custom expressions!

ReplaceString(
    [//WorkflowData/XMLEncoded],
    """,
    """
)

Unable To Create New WorkflowInstance For WorkflowDefinition

I discovered another variant that causes the titular error message.  I was copying the FIM policy config from one data source to another, and that meant making copies of a couple activities within the same workflow.  After some troubleshooting and head-scratching, I realized that FIM was choking on the duplicate activity names (see highlights below).  As you can see, when you build a workflow from the portal, FIM names all activities "authenticationGateActivityX."  And interestingly, if you made the same mistake as I did, and you try to fix it by opening all of the activities within the portal and click "Save," FIM won't fix your mistakes!

<ns0:SequentialWorkflow

  x:Name="SequentialWorkflow"

  ActorId="00000000-0000-0000-0000-000000000000"

  WorkflowDefinitionId="00000000-0000-0000-0000-000000000000"

  RequestId="00000000-0000-0000-0000-000000000000"

  TargetId="00000000-0000-0000-0000-000000000000"

  xmlns:ns1="clr-namespace:IDMware.FIM.Workflow.Activities;Assembly=IDMware.FIM.Workflow, Version=1.0.0.0, Culture=neutral, PublicKeyToken=b6cdb71f2a5e8e65"

  xmlns:ns2="clr-namespace:FimExtensions.FimActivityLibrary;Assembly=FimExtensions.FimActivityLibrary, Version=2.0.0.0, Culture=neutral, PublicKeyToken=68dc29282b77eca9"

  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/workflow"

  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"

  xmlns:ns0="clr-namespace:Microsoft.ResourceManagement.Workflow.Activities;Assembly=Microsoft.ResourceManagement, Version=4.1.3559.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35">

  <ns1:GenerateRandomPasswordActivity

    x:Name="authenticationGateActivity1"

    DestinationValue="{x:Null}"

    MinPasswordLength="8"

    DestinationString="RandomPassword"

    TheActivityName="Generate random password"

    MaxPasswordLength="10" />

  <ns0:FunctionActivity

    x:Name="authenticationGateActivity6"

    FunctionExpression="<fn id="SingleValueAssignment"  isCustomExpression="false"><arg>[//Target/AccountName]</arg></fn>"

    Description="Grab Target account name"

    Destination="[//WorkflowData/TargetAccountName]" />

  <ns2:PowerShellActivity

    x:Name="authenticationGateActivity5"

    TheActivityName="Powershell: Set AD password"

    PowerShellScript="C:\Scryptography\SetADPwd.ps1" />

  <ns0:FunctionActivity

    x:Name="authenticationGateActivity7"

    FunctionExpression="<fn id="SingleValueAssignment"  isCustomExpression="false"><arg>[//Target/LMSID]</arg></fn>"

    Description="Grab LMS ID"

    Destination="[//WorkflowData/LMSID]" />

  <ns1:ExecuteSQLActivity

    SQLStatement="EXECUTE [dbo].[SetLMSNewUserPassword]    @id   ,@password" x:Name="authenticationGateActivity8"

    SQLConnectionStringName="LMS"

    ParamData="<DocumentElement>   <Parameters>     <ParamName>@id</ParamName>     <ParamValue>[//WorkflowData/LMSID]</ParamValue>     <DataType>Integer</DataType>   </Parameters>   <Parameters>     <ParamName>@password</ParamName>     <ParamValue>[//WorkflowData/RandomPassword]</ParamValue>     <DataType>String</DataType>   </Parameters> </DocumentElement>"

    WorkflowKey="Results"

    TheActivityName="SQL: Set LMS password" />

  <ns0:FunctionActivity

    x:Name="authenticationGateActivity9"

    FunctionExpression="<fn id="SingleValueAssignment"  isCustomExpression="false"><arg>[//Target/VLMID]</arg></fn>"

    Description="Grab VLM ID"

    Destination="[//WorkflowData/VLMID]" />

  <ns1:ExecuteSQLActivity

    SQLStatement="EXECUTE [dbo].[SetVLMNewUserPassword]    @id   ,@password"

    x:Name="authenticationGateActivity10"

    SQLConnectionStringName="VLM"

    ParamData="<DocumentElement>   <Parameters>     <ParamName>@id</ParamName>     <ParamValue>[//WorkflowData/VLMID]</ParamValue>     <DataType>String</DataType>   </Parameters>   <Parameters>     <ParamName>@password</ParamName>     <ParamValue>[//WorkflowData/RandomPassword]</ParamValue>     <DataType>String</DataType>   </Parameters> </DocumentElement>"

    WorkflowKey="VLMResults"

    TheActivityName="SQL: Set VLM password" />

  <ns0:FunctionActivity

    x:Name="authenticationGateActivity9"

    FunctionExpression="<fn id="SingleValueAssignment"  isCustomExpression="false"><arg>[//Target/PDSID]</arg></fn>"

    Description="Grab PDS ID"

    Destination="[//WorkflowData/PDSID]" />

  <ns1:ExecuteSQLActivity

    SQLStatement="EXECUTE [dbo].[SetPDSNewUserPassword]    @id   ,@password"

    x:Name="authenticationGateActivity10"

    SQLConnectionStringName="PDS"

    ParamData="<DocumentElement>   <Parameters>     <ParamName>@id</ParamName>     <ParamValue>[//WorkflowData/PDSID]</ParamValue>     <DataType>String</DataType>   </Parameters>   <Parameters>     <ParamName>@password</ParamName>     <ParamValue>[//WorkflowData/RandomPassword]</ParamValue>     <DataType>String</DataType>   </Parameters> </DocumentElement>"

    WorkflowKey="PDSResults"

    TheActivityName="SQL: Set PDS password" />

  <ns1:ExpressionEvaluator

    x:Name="authenticationGateActivity4"

    DestinationValue="False"

    TheUpdateMode="0"

    Notes=""

    DestinationString="[//Target/SetPassword]"

    ActorID="31415926-5358-9793-2384-626433832795"

    ThisActivityName="Turn off SetPassword flag"

    ValueType="String" />

</ns0:SequentialWorkflow>

FIM Powershell Module: Remove/unset/clear a single-valued reference attribute

In the latest version of the FIM Powershell Module (2016-05-18), in order to remove/unset/clear a single-valued reference attribute, you're supposed to do this:

New-FimImportChange -Operation 'Replace' -AttributeName "Manager"

Note that you just don't supply the -AttributeValue paramter.  However, in my script, I don't want to perform the extra step of checking whether my value is present; so I'd like to do this:

New-FimImportChange -Operation 'Replace' -AttributeName "Manager" -AttributeValue "$newManager"

In order to do that, I had to make a small change to New-FimImportChange in FimPowerShellModule.psm1:

###

### Process the AttributeValue Parameter

###

if (!$AttributeValue)

{

    # Allow the caller to pass an empty AttributeValue to unset it, but DO NOT set the AttributeValue on the ImportChange object.

}

elseif ($AttributeValue -is [String])

{

    $importChange.AttributeValue = $AttributeValue

}

elseif ($AttributeValue -is [DateTime])

MIM metaverse SQL query - manager contributing MA

This is a sequel (no pun intended) to my old post, FIM metaverse SQL query - employeeID contributing MA.  Since 'manager' is a reference attribute, you need a slightly different query than for scalar attributes.

set transaction isolation level read uncommitted

SELECT TOP 1000

       mv.object_type

       ,mv.accountName

       ,mv.domain

       ,l.attribute_name

       ,ma_mgr.ma_name as [manager MA]

FROM [FIMSynchronizationService].[dbo].[mms_mv_link] l

join [FIMSynchronizationService].dbo.mms_metaverse mv

on l.object_id = mv.object_id

left join [FIMSynchronizationService].[dbo].[mms_lineage_cross_reference] cr_mgr

on cr_mgr.lineage_id = l.lineage_id

left join [FIMSynchronizationService].[dbo].[mms_management_agent] ma_mgr

on ma_mgr.ma_id = cr_mgr.ma_id

where object_type = 'person'

and l.attribute_name = 'manager'

Lessons learned - FIM_TemporalEventsJob

Here are some lessons learned while troubleshooting the titular SQL Agent job.  If you think to yourself, "Self, why would I ever need to care about this?" then you're either (a) lucky or (b) blissfully ignorant.  Don't worry though, you can blame FIM for your ignorance.  ;)  This is one of those features of FIM that won't actually tell you that something's wrong until you stumble upon the fact that, for example, your managers stopped getting notifications when contractor accounts expired.

So, where do you start troubleshooting such a problem?  Well, the keyword "expiration" may trigger thoughts of whether you remembered to renew your car's registration, but in FIMLand it should make you think, "temporal set."  And how are temporal events triggered?  Well, one way is when a datetime attribute on a user is updated and it falls into the scope of a temporal set.  But think about the other way that temporal events are triggered: the user's datetime attribute has been the same for several weeks, and now it's been long enough that the user now satisfies some expiration criteria.  If you've never had to consider how that happens, then I envy you!  But if you really want to know, or if you've read this far and feel like you can't turn back now, then I'll tell you that those events are triggered by a SQL Agent job that's installed with the FIM Service product.

If you run SQL Server Management Studio (SSMS) and connect to the database engine on the FIM SQL server, you can expand the SQL Server Agent -> Jobs branch, right click on FIM_TemporalEventsJob and select View History.  If you see all green checkmarks, then congratulations on keeping your set criteria simple enough for FIM to handle!  However, if you see red icons like below, then congratulations on giving FIM a challenge!  (But unfortunately you're going to have to fix it.)

Lesson 1: FIM_TemporalEventsJob SQL Agent job continues after error in step 1, but fails if error in step 2

The steps of the FIM_TemporalEventsJob SQL Agent job are shown below.  The first lesson I learned in this troubleshooting exercise is that, if the job encounters an error in step 1, the job still continues on to the next step.  However, if an error is encountered in step 2, the job is stopped and the last two steps aren't executed.  (And in case you're not SQL literate, the steps below all execute SQL stored procedures.)

Step 1	EXECUTE [fim].[TriggerTemporalEvents] @extendedOutput = 0
Step 2	EXECUTE [fim].[MaintainSets]
Step 3	EXECUTE [fim].[MaintainGroups]
Step 4	EXECUTE [fim].[OptimizeSetMembershipConditionsUsingPartitions]

Step 1 sample error

Here's an example of an error that you might see in step 1 of the job history.  If you've got a keen eye, you may ask yourself, "What the heck is a SetKey???"  Don't worry, keep reading... 

Executed as user: DOMAIN\SVC-SQLAgent. Reraised Error 50000, Level 16, State 1, Procedure TriggerTemporalEvents, Line 667, Message: <_x0040_failedTable SetKey="6062948" ErrorMsg="Reraised Error 50000, Level 16, State 1, Procedure ReRaiseException, Line 37, Message: Reraised Error 8623, Level 16, State 1, Procedure ?, Line 2, Message: The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information."/> [SQLSTATE 42000] (Error 50000).  The step failed.

Step 2 sample error

Here's a similar error that you might see in step 2 of the job history.  This is essentially the same error, but it occurred in a different stored procedure.

Executed as user: DOMAIN\SVC-SQLAgent. Reraised Error 50000, Level 16, State 1, Procedure MaintainSets, Line 556, Message: <Failures><_x0040_failedSetCorrections SetKey="6062948" ErrorMessage="Reraised Error 8623, Level 16, State 1, Procedure ?, Line 2, Message: The query processor ran out of internal resources and could not produce a query plan. This is a rare event and only expected for extremely complex queries or queries that reference a very large number of tables or partitions. Please simplify the query. If you believe you have received this message in error, contact Customer Support Services for more information."/></Failures> [SQLSTATE 42000] (Error 50000).  The step failed.

Lesson 2: Step 2 (fim.MaintainSets) will continue after individual errors

Okay, we've learned that the job will continue after errors in step 1 but not step 2.  But what about the substeps within each step?  I.e., if an error occurs while evaluating one set, will it stop or continue to evaluate the rest of the sets?  The answer is that it will evaluate *all* sets, even if one set causes an error.

To prove that to myself, I added some print statements to the fim.MaintainSets stored procedure and ran it manually...

Sample of adding print statements to fim.MaintainSets procedure

SUCCESS: @setKey                         = 2732

SUCCESS: @setKey                         = 2733

SUCCESS: @setKey                         = 2734

...
SUCCESS: @setKey                         = 6062663

SUCCESS: @setKey                         = 6062802

SUCCESS: @setKey                         = 6062814

FAIL: @setKey                         = 6062948

SUCCESS: @setKey                         = 6063066

SUCCESS: @setKey                         = 7185483

SUCCESS: @setKey                         = 7185525

...
SUCCESS: @setKey                         = 10711150

SUCCESS: @setKey                         = 11024517

SUCCESS: @setKey                         = 11596415

Lesson 3: Find problematic set

Okay, so how do you find which set is causing the problem?  In the error message above we saw a "SetKey," but what the heck do you do with that?  That's a great question, because all identifiers in the FIM Service are GUIDs.  Well, the FIM product group certainly did you no favors here.  In order to find the problematic set, you have to run a SQL query against the FIMService database:

SELECT

    s.SetKey

    ,o.ObjectID

    ,x.DisplayName

FROM [fim].[Set] s

join fim.[Objects] o on s.SetKey = o.ObjectKey

CROSS APPLY

(

    SELECT

        ValueString as DisplayName

    FROM fim.ObjectValueString ovs

    join fim.AttributeInternal ai on ai.[Key] = ovs.AttributeKey

    WHERE Name = 'DisplayName'

    and ovs.ObjectKey = o.ObjectKey

) x

where s.SetKey = '6062948'

Et voila!  Here's your problematic set!

SetKey	ObjectID	DisplayName
6062948	0DD436A3-C7B5-4D9E-974C-B24E546ED789	Contractor Extended Date 56 days without login

"Great!  Thanks Joe!  ...  Wait, how do I fix it???"  Well, this is out of scope of this blog post, but here are a few web resources to help you.  Good luck!

Troubleshooting FIM Service: Set creation fails when using negative conditions

Criteria-Based Distribution Group Limitations

AddMembersToSet.ps1

• Download utility script: https://technet.microsoft.com/en-us/library/ff720152(v=ws.10).aspx
• Start with Groups script: https://technet.microsoft.com/en-us/library/ff720141(v=ws.10).aspx
• Change 'Group' to 'Set' and fix script to actually work (see code below).
• Obtain comma-separated values list of persons of interest (hint: use FIM Query Tool)

How to use:

. C:\PS\FIMPowerShell.ps1; . C:\PS\AddMembersToSet.ps1; Import-Csv "C:\PS\PersonsOfInterest.csv" | select 'Resource ID' | % { AddMembersToSet -IdentifierName 'ObjectID' -SetIdentifier '7ee30a29-ca1c-463f-a4ff-0375020b8843' -PersonIdentifiers $_.'Resource ID'  -Verbose }

PersonsOfInterest.csv

"Display Name","Account Name","Resource ID"
"Garth Maul","GMaul","e330d85f-effd-442b-b78f-d6f7681a44e7"
"Sky Walker","SWalker","291959a2-4794-402b-a5f4-f8734326fbe6"
"Don Solo","DSolo","e2ae68a9-6b54-43d7-acb8-6b3ed6a82e45"

AddMembersToSet.ps1

$DefaultUri = "http://localhost:5725"
function AddMembersToSet
{
       [CmdletBinding()]
    PARAM($SetIdentifier, $PersonIdentifiers, $IdentifierName="Email", $Uri = $DefaultUri, [switch]$WhatIf)
    END
    {
        Write-Verbose "`$ResolveSet = ResolveObject-ObjectType 'Set' -AttributeName $IdentifierName -AttributeValue$SetIdentifier"
        $ResolveSet = ResolveObject -ObjectType "Set" -AttributeName $IdentifierName -AttributeValue $SetIdentifier
        $ResolveSet | Import-FIMConfig -Uri $Uri
             Write-Verbose "`$ResolveSet: "
             $ResolveSet
        $ImportObjects = $NULL
        $AddedMembers = $NULL
        foreach($PersonIdentifier in $PersonIdentifiers)
        {
                    Write-Verbose "`$ImportObject = ResolveObject -ObjectType 'Person'-AttributeName $IdentifierName -AttributeValue $PersonIdentifier"
            $ImportObject = ResolveObject -ObjectType "Person" -AttributeName $IdentifierName -AttributeValue $PersonIdentifier
                    $ImportObject | Import-FIMConfig -Uri $Uri
                    Write-Verbose "`$ImportObject: "
                    $ImportObject
            if($AddedMembers -eq $NULL)
            {
                $AddedMembers = @($ImportObject.SourceObjectIdentifier)
            }
            else
            {
                $AddedMembers += $ImportObject.SourceObjectIdentifier
            }
            if($ImportObjects -eq $NULL)
            {
                $ImportObjects = @($ImportObject)
            }
            else
            {
                $ImportObjects += $ImportObject
            }
        }
        
        $ModifyImportObject = ModifyImportObject -TargetIdentifier $ResolveSet.TargetObjectIdentifier -ObjectType "Set"
        $ModifyImportObject.SourceObjectIdentifier = $ResolveSet.SourceObjectIdentifier
             Write-Verbose "`$ModifyImportObject: "
             $ModifyImportObject
        
        foreach($AddedMember in $AddedMembers)
        {
            $newValue = $AddedMember
            #The followingline adds all of the Person resources to the Set (if not commented out).
            AddMultiValue -ImportObject $ModifyImportObject -AttributeName "ExplicitMember" -NewAttributeValue $newValue -FullyResolved 0
            #The followingline removes all of the Person resources from the Set (if not commented out).
            #RemoveMultiValue-ImportObject $ModifyImportObject -AttributeName "ExplicitMember"-NewAttributeValue $newValue -FullyResolved 0
        }
        $ImportObjects += $ModifyImportObject
             if (!$WhatIf) {
              #The following line will update the Setobject with the added members (if not commented out). 
                    $ImportObjects | Import-FIMConfig -Uri $Uri
                #$ImportObjects | % { Import-FIMConfig$_ -Uri $Uri; Break }
             } else {
              #The following line returns a referenceto the ImportObject collection (if not commented out).
              $ImportObjects
             }
    }
}