Wednesday, June 26, 2013

Dude, where's my flow?

Got burned by FIM's wacky function behavior again, so I thought I'd actually create some documentation on it.

I set up a pretty common flow: Name = First Middle Last

However, when the middle name is empty, I don't want two blanks between first and last name.  So, how's this for a custom expression?  (Note that I'm assuming that first and last name are always present.)

Trim(givenName+" "+HR_MIDDLE_NAME)+" "+sn

Submit, import, sync.  Now I run a full sync from my target MA.  No errors.  We're good, right?  Not so fast; let's check the attribute flow.  No flow into Name.

Okay, so let's run a full sync preview on one of the connectors.  The status of the flow is Error.  (Good luck finding an error message.)
  • Full sync preview
    • Connector updates
      • Export attribute flow
        • Outbound sync rules
Status
MV attribute
Mapping type
Data source attribute
Initial value
Final value
Error

sync-rule-mapping - expression
NAME

(Unchanged)

So what happened?  Evidently, FIM doesn't like it when you concatenate an attribute that isn't present on the metaverse object.  (Has a funny way of showing it though, doesn't it?)

For my reference and yours, here are the results of other experiments that I ran on this flow.
  • HR_MIDDLE_NAME is not present in MV
  • (Assume givenName and sn are always present)

Description
Function
Flow definition
Result/status
MV attribute not present, embedded expr in Trim
CustomExpression
Trim(givenName+" "+HR_MIDDLE_NAME)+" "+sn
Error
Same expression but using concat feature in portal interface
Concat
Trim(CustomExpression(givenName+" "+HR_MIDDLE_NAME))+" "+sn
Error
Added IsPresent check and removed Trim
CustomExpression
givenName+IIF(IsPresent(HR_MIDDLE_NAME)," "+HR_MIDDLE_NAME,"")+" "+sn
Success
Trim function with no assumptions
CustomExpression
Trim(
""+
IIF(IsPresent(givenName),givenName,"")+
" "+
IIF(IsPresent(HR_MIDDLE_NAME),HR_MIDDLE_NAME,"")+
" "+
IIF(IsPresent(sn),sn,"")
)
Parse failure:
Parameter of function Trim does not match
(Test embedded expr in Trim)
CustomExpression
Trim(givenName+" "+sn)
(Success)
(Test MV attribute not present in Trim)
CustomExpression
givenName+" "+HR_MIDDLE_NAME+" "+sn
(Error)

Friday, June 14, 2013

sync-rule-flow-provisioning-failed Microsoft.MetadirectoryServices.ProvisioningBySyncRuleException: attribute is read-only

Ran into this problem the other day, and I was a bit confused.

sync-rule-flow-provisioning-failed
Microsoft.MetadirectoryServices.ProvisioningBySyncRuleException: attribute is read-only

I Googled the error message, but I didn't get any direct hits.  Turns out, I'd forgotten to check the Initial Flow Only box on the anchor attribute in the outbound sync rule (OSR).

The following links helped find the answer.
http://social.technet.microsoft.com/Forums/en-US/ilm2/thread/1aa13147-e16c-4e99-a7da-76e3c9e8c10d/
http://social.technet.microsoft.com/Forums/en-US/ilm2/thread/6ac2bed1-9704-4fcc-94d2-3be73c5a7f47

I guess this is one of those subtle features that I'd taken for granted.  Essentially, this means that there's no built-in "anchor rename" feature on the SQL MAs (this is an Oracle MA, BTW).

Here's another subtle feature that's closely related to this one.  As Markus mentions in his article, "In case of a SQL management agent, the anchor attribute is also the DN."  However, this doesn't mean that the CS object DN is exported to the anchor column.  In fact, you can arbitrarily initialize the CS object DN with the csObjectID (as Markus mentions); and assuming your SQL anchor is generated automatically upon export, you can join on the anchor (like userid) during inbound sync, and (if I recall correctly) the CS object DN will be set equal to the anchor.

Thursday, June 13, 2013

More FIM Powershell Module updates

More updates to the FIM Powershell Module for end-to-end flow rules. In case you're wondering, yes, I've asked the Codeplex project team to add me as a developer. :)

1. Fixed a bug with OSR-Expression export flows.
2. Added the following info to the output:
    a. Allow nulls
    b. Initial flow only
    c. Is existence test

Reference version:
FIM PowerShell Modules v2.1
http://fimpowershellmodule.codeplex.com/releases/view/98374

<#
  .SYNOPSIS
  Gets the Export Attribute Flow Rules from Sync Server Configuration
  .DESCRIPTION
  Reads the server configuration from the XML files, and outputs the Export Attribute Flow rules as PSObjects
  .OUTPUTS
  PSObjects containing the synchronization server export attribute flow rules
  
  .EXAMPLE
  Get-ExportAttributeFlow -ServerConfigurationFolder "E:\sd\IAM\ITAuthorize\Source\Configuration\FimSync\ServerConfiguration"
#>
Function Get-ExportAttributeFlow
{
  Param
  (       
        [parameter(Mandatory=$false)]
            [String]
            [ValidateScript({Test-Path $_})]
            $ServerConfigurationFolder
  )
  End
  {       
           ### This is where the rules will be aggregated before we output them
            $rules = @()
           
            ### Export attribute flow rules are contained in the ma-data nodes of the MA*.XML files
            $maFiles = @(get-item (Join-Path $ServerConfigurationFolder "MA-*.xml"))
           
           
            foreach ($maFile in $maFiles)
            {
                  ### Get the MA Name and MA ID
                  $maName = (select-xml $maFile -XPath "//ma-data/name").Node.InnerText
              
                foreach($exportFlowSet in (Select-Xml -path $maFile -XPath "//export-flow-set" | select -ExpandProperty Node))
                {
                    $mvObjectType = $exportFlowSet.'mv-object-type'
                    $cdObjectType = $exportFlowSet.'cd-object-type'
                   
                    foreach($exportFlow in $exportFlowSet.'export-flow')
                    {
                        $cdAttribute = $exportFlow.'cd-attribute'
                        [bool]$allowNulls = $false
                              if ([bool]::TryParse($exportFlow.'suppress-deletions', [ref]$allowNulls))
                              {
                                    $allowNulls = -not $allowNulls
                              }
                        [string]$initialFlowOnly = $null
                        [string]$isExistenceTest = $null
                       
                        if ($exportFlow.'direct-mapping' -ne $null)
                        {
                        ###
                        ### Handle src-attribute that are intrinsic (<src-attribute intrinsic="true">object-id</src-attribute>)
                        ###
                        if ($exportFlow.'direct-mapping'.'src-attribute'.intrinsic)
                        {
                            $srcAttribute = "<{0}>" -F $exportFlow.'direct-mapping'.'src-attribute'.'#text'
                        }
                        else
                        {
                                $srcAttribute = $exportFlow.'direct-mapping'.'src-attribute'
                        }
                           
                            $rule = New-Object PSObject
                            $rule | Add-Member -MemberType noteproperty -name 'RuleType' -value 'DIRECT'
                            $rule | Add-Member -MemberType noteproperty -name 'MAName' -value $maName               
                            $rule | Add-Member -MemberType noteproperty -name 'MVObjectType' -value $mvObjectType
                            $rule | Add-Member -MemberType noteproperty -name 'MVAttribute' -value $srcAttribute
                            $rule | Add-Member -MemberType noteproperty -name 'CDObjectType' -value $cdObjectType
                            $rule | Add-Member -MemberType noteproperty -name 'CDAttribute' -value $cdAttribute
                                    $rule | Add-Member -MemberType noteproperty -name 'ScriptContext' -value $null
                                    $rule | Add-Member -MemberType noteproperty -name 'AllowNulls' -value $allowNulls.ToString()
                                    $rule | Add-Member -MemberType noteproperty -name 'InitialFlowOnly' -value $initialFlowOnly
                                    $rule | Add-Member -MemberType noteproperty -name 'IsExistenceTest' -value $isExistenceTest
                           
                            $rules += $rule
                        }
                        elseif ($exportFlow.'scripted-mapping' -ne $null)
                        {               
                            $scriptContext = $exportFlow.'scripted-mapping'.'script-context'                          
                                    $srcAttributes = @()
                                   
                        ###
                        ### Handle src-attribute that are intrinsic (<src-attribute intrinsic="true">object-id</src-attribute>)
                        ###
                        $exportFlow.'scripted-mapping'.'src-attribute' | ForEach-Object {
                            if ($_.intrinsic)
                            {
                                $srcAttributes += "<{0}>" -F $_.'#text'
                            }
                            elseif ($_) # Do not add empty values.
                            {
                                    $srcAttributes += $_
                            }
                        }
                        # (Commented) Leave as collection.
                        #if ($srcAttributes.Count -eq 1)
                        #{
                        #    $srcAttributes = $srcAttributes -as [String]
                        #}
                               
                            $rule = New-Object PSObject
                            $rule | Add-Member -MemberType noteproperty -name 'RuleType' -value 'SCRIPTED'
                            $rule | Add-Member -MemberType noteproperty -name 'MAName' -value $maName
                                    $rule | Add-Member -MemberType noteproperty -name 'MVObjectType' -value $mvObjectType
                            $rule | Add-Member -MemberType noteproperty -name 'MVAttribute' -value $srcAttributes
                            $rule | Add-Member -MemberType noteproperty -name 'CDObjectType' -value $cdObjectType
                            $rule | Add-Member -MemberType noteproperty -name 'CDAttribute' -value $cdAttribute   
                            $rule | Add-Member -MemberType noteproperty -name 'ScriptContext' -value $scriptContext
                                    $rule | Add-Member -MemberType noteproperty -name 'AllowNulls' -value $allowNulls.ToString()
                                    $rule | Add-Member -MemberType noteproperty -name 'InitialFlowOnly' -value $initialFlowOnly
                                    $rule | Add-Member -MemberType noteproperty -name 'IsExistenceTest' -value $isExistenceTest
                                           
                            $rules += $rule                       
                        }
                              elseif ($exportFlow.'sync-rule-mapping' -ne $null)
                              {
                                    $srcAttribute = $exportFlow.'sync-rule-mapping'.'src-attribute'
                        $initialFlowOnly = $exportFlow.'sync-rule-mapping'.'initial-flow-only'
                        $isExistenceTest = $exportFlow.'sync-rule-mapping'.'is-existence-test'
                                    if($exportFlow.'sync-rule-mapping'.'mapping-type' -eq 'direct')
                                    {
                                          $rule = New-Object PSObject
                                          $rule | Add-Member -MemberType noteproperty -name 'RuleType' -value 'OSR-Direct'
                                          $rule | Add-Member -MemberType noteproperty -name 'MAName' -value $maName
                                          $rule | Add-Member -MemberType noteproperty -name 'MVObjectType' -value $mvObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'MVAttribute' -value $srcAttribute
                                          $rule | Add-Member -MemberType noteproperty -name 'CDObjectType' -value $cdObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'CDAttribute' -value $cdAttribute                                                                               
                                          $rule | Add-Member -MemberType noteproperty -name 'ScriptContext' -value $null
                                          $rule | Add-Member -MemberType noteproperty -name 'AllowNulls' -value $allowNulls.ToString()
                                    $rule | Add-Member -MemberType noteproperty -name 'InitialFlowOnly' -value $initialFlowOnly
                                    $rule | Add-Member -MemberType noteproperty -name 'IsExistenceTest' -value $isExistenceTest
                                                                 
                                          $rules += $rule            
                                    }
                                    elseif ($exportFlow.'sync-rule-mapping'.'mapping-type' -eq 'expression')
                                    {
                                          $scriptContext = $exportFlow.'sync-rule-mapping'.'sync-rule-value'.'export-flow'.InnerXml
                                          $cdAttribute = $exportFlow.'sync-rule-mapping'.'sync-rule-value'.'export-flow'.dest
                                          $rule = New-Object PSObject
                                          $rule | Add-Member -MemberType noteproperty -name 'RuleType' -value 'OSR-Expression'
                                          $rule | Add-Member -MemberType noteproperty -name 'MAName' -value $maName
                                          $rule | Add-Member -MemberType noteproperty -name 'MVObjectType' -value $mvObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'MVAttribute' -value $srcAttribute
                                          $rule | Add-Member -MemberType noteproperty -name 'CDObjectType' -value $cdObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'CDAttribute' -value $cdAttribute                                                                               
                                          $rule | Add-Member -MemberType noteproperty -name 'ScriptContext' -value $scriptContext
                                          $rule | Add-Member -MemberType noteproperty -name 'AllowNulls' -value $allowNulls.ToString()
                                    $rule | Add-Member -MemberType noteproperty -name 'InitialFlowOnly' -value $initialFlowOnly
                                    $rule | Add-Member -MemberType noteproperty -name 'IsExistenceTest' -value $isExistenceTest
                                                                 
                                          $rules += $rule            
                                    }
                                    elseif ($exportFlow.'sync-rule-mapping'.'mapping-type' -eq 'constant')
                                    {                          
                                        $srcAttributes = @()
                                        $scriptContext = $exportFlow.'sync-rule-mapping'.'sync-rule-value'
                                          $rule = New-Object PSObject
                                          $rule | Add-Member -MemberType noteproperty -name 'RuleType' -value 'OSR-Constant'
                                          $rule | Add-Member -MemberType noteproperty -name 'MAName' -value $maName
                                          $rule | Add-Member -MemberType noteproperty -name 'MVObjectType' -value $mvObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'MVAttribute' -value $srcAttributes
                                          $rule | Add-Member -MemberType noteproperty -name 'CDObjectType' -value $cdObjectType
                                          $rule | Add-Member -MemberType noteproperty -name 'CDAttribute' -value $cdAttribute                                                                               
                                          $rule | Add-Member -MemberType noteproperty -name 'ScriptContext' -value "'$scriptContext'"
                                          $rule | Add-Member -MemberType noteproperty -name 'AllowNulls' -value $allowNulls.ToString()
                                    $rule | Add-Member -MemberType noteproperty -name 'InitialFlowOnly' -value $initialFlowOnly
                                    $rule | Add-Member -MemberType noteproperty -name 'IsExistenceTest' -value $isExistenceTest
                                                                 
                                          $rules += $rule             
                                    }
                                    else
                                    {
                                          throw "Unsupported Export Flow type '$($exportFlow.'sync-rule-mapping'.'mapping-type')'"
                                    }
                            
                              }
                    }
                }
            }
           
            Write-Output $rules
  }#End
}
<#
  .SYNOPSIS
  Gets the Joined Rules where the IAF rules are joined to the EAF rules based on the MV Attributes and Object Types
  .DESCRIPTION
  Reads the server configuration from the XML files, and outputs the Joined IAF and EAF Rules as PSObjects
  .OUTPUTS
  PSObjects containing the synchronization server attribute flow rules
  
  .EXAMPLE
  Join-ImportToExportAttributeFlow -ServerConfigurationFolder "E:\sd\IAM\ITAuthorize\Source\Configuration\FimSync\ServerConfiguration"
  
#>
Function Join-ImportToExportAttributeFlow
{
    [CmdletBinding()]
     Param
     (       
        [parameter(Mandatory=$false)]
            [String]
            [ValidateScript({Test-Path $_})]
            $ServerConfigurationFolder
     )
      End
      {
            ### Get the Import Attribute Flow Rules
            $IAF = Get-ImportAttributeFlow-ServerConfigurationFolder $ServerConfigurationFolder
           
            ### Get the Export Attribute Flow Rules
            $EAF = Get-ExportAttributeFlow-ServerConfigurationFolder $ServerConfigurationFolder
       
#        foreach ($eafTest in $EAF)
#        {
##            Write-Verbose "$(' ' + $eafTest.'MVAttribute' + ' ') $($eafTest.'MVAttribute'.GetType())"
##            Write-Verbose "$($eafTest.'MVAttribute' -join ',') $($eafTest.'MVAttribute'.GetType())"
#            if (($eafTest.'RuleType') -eq 'OSR-Expression')
#            {
#                Write-Host $eafTest
##                Write-Verbose "$eafTest $($eafTest.'MVAttribute'.GetType()) $($eafTest.'MVAttribute'.count) $($eafTest.'MVAttribute' -join ',')"
#            }
#        }
            ### This is where the rules will be aggregated before we output them
            $e2eFlowRules = @()
            foreach ($iafRule in $IAF)
            {
                ### Look for a corresponding EAF rule   
                $eafMatches = @($EAF | where {$_.'MVAttribute' -contains $iafRule.'MVAttribute' -and $_.'MVObjectType' -eq $iafRule.'MVObjectType'})
                  ### There may be multiple EAF rule for each IAF rules
                if ($eafMatches.count -gt 0)
                {
                    foreach($eafRule in $eafMatches)
                    {                        
                        $e2eFlowRuleProperties = @{           
                            'IAFRuleType'          = $iafRule.'RuleType'
                            'IAFSourceMA'          = $iafRule.'SourceMA'
                            'IAFCDObjectType'      = $iafRule.'CDObjectType'
                            'IAFCDAttribute'       = $iafRule.'CDAttribute'
                            'IAFScriptContext'     = $iafRule.'ScriptContext'
                            'IAFPrecedenceType'    = $iafRule.'PrecedenceType'
                            'IAFPrecedenceRank'    = $iafRule.'PrecedenceRank'
                            'MVObjectType'         = $iafRule.'MVObjectType'
                            'MVAttribute'          = $iafRule.'MVAttribute'
                            'EAFMVAttribute'       = $eafRule.'MVAttribute'
                            'EAFCDAttribute'       = $eafRule.'CDAttribute'
                            'EAFTargetMA'          = $eafRule.'MAName'
                            'EAFCDObjectType'      = $eafRule.'CDObjectType'
                            'EAFRuleType'          = $eafRule.'RuleType'
                            'EAFScriptContext'     = $eafRule.'ScriptContext'
                            'EAFAllowNulls'        = $eafRule.'AllowNulls'
                            'EAFInitialFlowOnly'   = $eafRule.'InitialFlowOnly'
                            'EAFIsExistenceTest'   = $eafRule.'IsExistenceTest'
                        }
                       
                    $e2eFlow = New-Object PSObject -Property $e2eFlowRuleProperties
                        $e2eFlowRules += $e2eFlow
                    }
                }
                  ### It is possible there are NO EAF rules for an IAF rule
                  ### here we stuff $null into the EAF side to make our output easy to consume for Out-GridView and Compare-Object
                  ### otherwise jagged objects seem to confuse things
                  ###
                  ### In this case the rule may be useless
                  ### Or the use of that MV attribute may not be visible here because some rules extension calls it (need to check the source code to confirm)
                else
                {
                   $e2eFlowRuleProperties = @{           
                            'IAFRuleType'          = $iafRule.'RuleType'
                            'IAFSourceMA'          = $iafRule.'SourceMA'
                            'IAFCDObjectType'      = $iafRule.'CDObjectType'
                            'IAFCDAttribute'       = $iafRule.'CDAttribute'
                            'IAFScriptContext'     = $iafRule.'ScriptContext'
                            'IAFPrecedenceType'    = $iafRule.'PrecedenceType'
                            'IAFPrecedenceRank'    = $iafRule.'PrecedenceRank'
                            'MVObjectType'         = $iafRule.'MVObjectType'
                            'MVAttribute'          = $iafRule.'MVAttribute'
                            'EAFMVAttribute'       = $null
                            'EAFCDAttribute'       = $null
                            'EAFTargetMA'          = $null
                            'EAFCDObjectType'      = $null
                            'EAFRuleType'          = $null
                            'EAFScriptContext'     = $null
                            'EAFAllowNulls'        = $null
                            'EAFInitialFlowOnly'   = $null
                            'EAFIsExistenceTest'   = $null
                        }
                       
                $e2eFlow = New-Object PSObject -Property $e2eFlowRuleProperties
                  $e2eFlowRules += $e2eFlow
                }
            }
        ### There's one more case in which the MV attribute is blank for an EAF.
        foreach ($eafRule in @($EAF | where {$_.'MVAttribute'.count -eq 0}))
        {
            Write-Verbose $eafRule
            $e2eFlowRuleProperties = @{           
                'IAFRuleType'          = $null
                'IAFSourceMA'          = $null
                'IAFCDObjectType'      = $null
                'IAFCDAttribute'       = $null
                'IAFScriptContext'     = $null
                'IAFPrecedenceType'    = $null
                'IAFPrecedenceRank'    = $null
                'MVObjectType'         = $null
                'MVAttribute'          = $null
                'EAFMVAttribute'       = $eafRule.'MVAttribute'
                'EAFCDAttribute'       = $eafRule.'CDAttribute'
                'EAFTargetMA'          = $eafRule.'MAName'
                'EAFCDObjectType'      = $eafRule.'CDObjectType'
                'EAFRuleType'          = $eafRule.'RuleType'
                'EAFScriptContext'     = $eafRule.'ScriptContext'
                'EAFAllowNulls'        = $eafRule.'AllowNulls'
                'EAFInitialFlowOnly'   = $eafRule.'InitialFlowOnly'
                'EAFIsExistenceTest'   = $eafRule.'IsExistenceTest'
            }
                       
            $e2eFlow = New-Object PSObject -Property $e2eFlowRuleProperties
            $e2eFlowRules += $e2eFlow
        }
            $e2eFlowRules | select `
            'IAFSourceMA',`
            'IAFCDObjectType',`
            'IAFCDAttribute',`
            'IAFRuleType',`
            'IAFScriptContext',`
            'IAFPrecedenceType',`
            'IAFPrecedenceRank',`
            'MVObjectType',`
            'MVAttribute',`
            'EAFMVAttribute',`
            'EAFTargetMA',`
            'EAFCDObjectType',`
            'EAFCDAttribute',`
            'EAFRuleType',`
            'EAFScriptContext',`
        'EAFAllowNulls',`
        'EAFInitialFlowOnly',`
        'EAFIsExistenceTest'`
      }
}