Quantcast
Channel: BAM – SANDRO PEREIRA BIZTALK BLOG
Viewing all 35 articles
Browse latest View live

BAM – Limitation of Tracking Profile Editor (TPE)

$
0
0

BizTalk artifacts only!

The most important limitation of TPE is that it can only be used to capture data from BizTalk artifacts. If you want to use Bam to capture data from custom code, you will need to use the BAM API.

Messages and elements limitations

Some messages and elements can’t be tracked with TPE. You can only track messages that have a schema that is known at design time, so you can’t track messages of type string or System.XML.XMLDocument.

Also you can’t capture data from repeating fields.

Some limitations exist on the length of data item names. The combined length of a folder name and data item instance value cannot be longer that 128 characters.

Orchestration shapes limitations

You also can’t track any of the following orchestration shapes:

  • Group (Task)
  • Loop (While)
  • Message Assignment
  • Suspend
  • Terminate
  • Throw Exception
  • Transform

You can work around these limitations for some shapes by wrapping them in a nontransactional scope shape and tracking the scope. However this don’t work for the Suspend and Terminate shapes, as they never fire the Shape End event.

Tags: BizTalk | BAM | TPE



BAM API – Using MessagingEventStream to write BAM Events from pipeline components

$
0
0

MessagingEventStream (MES) is used inside a BizTalk pipeline component to write Bam as part of the messaging transactions ensuring that your BAM event persistence remains in sync with the BizTalk pipeline transactions.

Messaging Event Streams are asynchronous and store tracking data first in the BizTalk MessageBox database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS). This class is derived from the base class EventStream.

MES is available since BizTalk 2004 with SP1.

Normally, we write the code to produce BAM events in the Execute method of the pipeline component. Execute method have two parameters:

  • An IPipelineContext: contains the pipeline context
  • An IBaseMessage: contains the message being processed.

The pipeline context has a GetEventStream method that returns a MessagingEventStream.

Code sample:

public IBaseMessage Execute(IPipelineContext context, IBaseMessage message)
{
   //Get the Messaging Event Stream
   MessagingEventStream eventStream = context.GetEventStream();
   …

   //Write to BAM
   //Create a new, unique activity identifier to use as the ActivityID in BAM
   string activityId = Guid.NewGuid().ToString() + "_" + DateTime.Now;

   //Start the activity record identified by activityId
   eventStream.BeginActivity("MyTrackingDemo", activityId);

   // Updates the activity record.
   eventStream.UpdateActivity("MyTrackingDemo", activityId, "MESSAGE_ID", message.MessageID, "TRANSACTION_CREATED", DateTime.UtcNow);

   eventStream.EndActivity("MyTrackingDemo", activityId);

   // Optional Flush
   eventStream.Flush();
}

Tags: BizTalk | BAM | MessagingEventStream


BAM API – Using OrchestrationEventStream to write BAM Events from BizTalk orchestration

$
0
0

The OrchestrationEventStream (OES) is one of the artifacts developers should consider in order to programmatically populate a BAM activity model from within a BizTalk orchestration.

The OES writes BAM Activity data into the state of the BizTalk orchestration and does not require a database roundtrip each time. When the orchestration completes or reaches a persistence point, the BAM activity data is written into the MessageBox along with the orchestration state.

Because this data is part of the orchestration state, all data is rolled back along with the orchestration if an error condition is reached. This method gives you complete transactional integrity, unlike the other event stream options, and offers the best performance for an orchestration-based solution.

The OES API stores tracking data first in the BizTalk MessageBox database. Periodically the data is processed and persisted to the BAM Primary Import database by the Tracking Data Decode Service (TDDS).

It is found in the Microsoft.BizTalk.Bam.EventObservation namespace. To use the API you must add the Microsoft.Biztalk.BAM.Xlangs.dll to your project.

So in your BizTalk project, include references to the following assemblies:

Microsoft.BizTalk.Bam.EventObservation and Microsoft.BizTalk.Bam.XLANGs.

These assemblies may be found in your “Microsoft BizTalk Server 2006\Tracking” folder.

The OrchestrationEventStream is utilized by the TPE tool. In BizTalk Server 2004, it was an internal component used by TPE and not exposed for general use. The transactional integrity requirement came up during a customer engagement, so it was later exposed in the 2004 release by a hotfix and is in the box with BizTalk Server 2006.

Sample code:

Note that because the OrchestrationEventStream is implemented as a static class, there is no need to manually create an event stream each time, and a connection string isn’t required, because the OrchestrationEventStream piggybacks the persistence of the orchestration.

//Create a new, unique activity identifier to use as the ActivityID in BAM
string activityId = Guid.NewGuid().ToString() + "_" + DateTime.Now;

//Start the activity record identified by activityId
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.BeginActivity(“MyTrackingDemo”, activityID);

// Updates the activity record.
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.UpdateActivity(“MyTrackingDemo”, activityID, “MESSAGE_ID”, msg(BTS.MessageID), "TRANSACTION_CREATED", DateTime.UtcNow);

// End the activity record.
Microsoft.BizTalk.Bam.EventObservation.OrchestrationEventStream.EndActivity(“MyTrackingDemo”, activityID);

Tags: BizTalk | BAM | OrchestrationEventStream


BAM – The BAM Databases

$
0
0

BAMAlertsApplication and BAMAlertsNSMain

Are used by SQL Notification Services to support BAM Alerts.

  • BAM Notification Services Application database (BAMAlertsApplication) Contains alert information for BAM notifications.
  • BAM Notification Services Instance database (BAMAlertsNSMain) Contains instance information specifying how the notification services connect to the system that BAM is monitoring.

BAMPrimaryImport

The BAM Primary Import database (BAMPrimaryImport) table is primarily designed to store data for current and recently completed. You will see several tables inside:

  • the table bam_<activity>_Active and bam_<activity>_Completed contains the activities;
  • the other tables are used by relationships and continuations

BAMStarSchema

The BAM Star Schema database (BAMStarSchema) Contains the staging table, and the measure and dimension tables.

BAMAnalysis

BAM Analysis database (BAMAnalysis) contains BAM OLAP cubes for both online and offline analysis.

BAMArchive

BAM Archive database (BAMArchive) archives old business activity data. You can create a BAM Archive database to minimize the accumulation of business activity data in the BAM Primary Import database.

Tags: BizTalk | BAM | Databases


BAM – SQL Server Views in the BAM Primary Import Database

$
0
0

When you issue the deploy-all command using “bm.exe” tool, BAM automatically creates the underlying data model for you activities and views.

Microsoft has committed to maintaining the structure of the views in futures versions of the product, so you can read the view data using any programming tool that works with SQL Server.

The term view in the content of BAM is used with two different meanings. A SQL Server view is a virtual table that is defined using a SELECT statement. A BAM view is derived from an activity by subsetting, aggregation, or aliasing. SQL Server view are used to implement both BAM activities and BAM views

The views that are directly derived from the deployment of a BAM activity:

  • bam_<Activity Name>_ActiveInstance: Returns the list of the active activity instances based on the records stored in the active activity table.
  • bam_<Activity Name>_AllInstances: Joins the active and complete activity views, returning a list of all the BAM activity instances, in other words, return all instances of the activity regardless of state.
  • bam_<Activity Name>_AllRelationships: Joins the active and complete activity relationship tables, returning a list of all the relationships of the BAM activity instances.
  • bam_<Activity Name>_CompletedInstances: Returns the list of the completed activity instances based on the records stored in the completed activity table.
  • bam_<Activity Name>_InstancesForArchive: return all instance of the activity that will be archived on the next execution of the DTS Package.

Additionally, a new set of views are created for each activity view that we include in the BAM activity model:

  • bam_<ActivityName>_<ActivityView Name>_ActiveView: Returns the list of active activity instances including the calculations of the required dimension levels.
  • bam_<ActivityName>_<ActivityView Name>_ActiveAliasView: Return the list of items currently in the active state by alias.
  • bam_<ActivityName>_<ActivityView Name>_CompletedView: Returns the list of completed activity instances including the calculations of the required dimension levels.
  • bam_<ActivityName>_<ActivityView Name>_CompletedAliasView: Items currently in completed state by alias.
  • bam_<ActivityName>_<ActivityView Name>_View: Joins the two previous views (ActiveView and CompletedView), returning the list of all activity instances included the calculation of the required dimension levels, in other words, all items in the view.

These views enable two key functionalities in the BAM infrastructure: At the first level, activity views are the fundamental mechanism to expose BAM data to other applications, including other components of the BAM infrastructure like the aggregation process. Additionally, these views include some of the calculations required by elements such as measures that are not directly included in the activity tables.

References:

Tags: BizTalk | BAM | SQL | Views


BizTalk BAM – Concepts

$
0
0

What is BAM used for?

BAM is used to monitor business milestones and key metrics in near real-time throughout a process in BizTalk.

Description

Business Activity Monitoring (BAM) provides visibility on business processes by tracking process milestones and business data (KPIs). BAM allows business users and decision makers to gain insight on the current health of in-flight processes they are responsible for. Up-to-date information is accessible to users via Microsoft Office BI tools or a designated portal and through automatic business alerts. This allows users to streamline and improve daily business operations by proactively escalating issues and mitigating problems sooner.

BAM is composed of a collection of tools and runtime components to layer business tracking without having to write any code. A set of runtime interceptors capture interesting business events inside of BizTalk Application or other applications and publish those events into the BAM infrastructure.

It also provides a set of tools for managing this infrastructure, tools for aggregating and viewing the data as well as APIs to optionally publish data to through custom code.

BAM is tied directly to the central BizTalk engine (that is, processes and databases) through OLAP cubes. These cubes are created automatically when a developer deploys a BAM report and profile, typically by using a combination of Microsoft Excel pivot tables and the BizTalk Tracking Profile Editor.

BAM Activities

BAM activities identify the milestones and tracking data an individual is interested in tracking. Milestones are the steps in an activity that are measured in time, and tracking data is the key data points in a process you are interested in tracking (such as a customer ID or name).

BAM View

A BAM view is a representation of the milestones and business data tracked in one or more activities.

  • The Group milestone allows related milestones to be treated as a single milestone. For example, grouping the EndSuccess and Exception milestones is useful to indicate the completion of the process regardless of whether it completed successfully or failed.
  • A duration calculates the time between two business milestones and is useful when reporting the time elapsed between two milestones is important.
  • A progress dimension defines milestones and stages for a process. A progress dimension allows you to display how many processes are at an existing status at a given time. For example, a progress dimension will display how many processes are in the middle of execution, how many encountered an exception, or how many processed without an exception.
  • Alias Allows the referencing of a previously created milestone

BAM alerts

BAM alerts allow you to configure and receive alerts related to specific changes in business data. Alerts are set up per BAM view.

BAM is based on SQL Server Reporting Services, and events are fundamentally triggered via SQL Server Notification Services.

Deployment

Use the deploy-all command to deploy the BAM activity and view defined in your BAM Excel workbook:

  • C:\Program Files\Microsoft BizTalk Server 2006\Tracking\BM.exe deploy-all -DefinitionFile:"MyBAMWorkbook.xls"
    

This command reads the BAM Excel workbook specified in the DefinitionFile parameter and creates SQL Server database artifacts based on the activities and view defined in the spreadsheet.

Related links:

Tags: BizTalk | BAM


BizTalk BAM activity Deploy problem – Old format or invalid type library. (Exception from HRESULT: 0×80028018 (TYPE_E_INVDATAREAD)).

$
0
0

When trying to deploy BAM activity: “C:\Program Files\Microsoft BizTalk Server 2009\Tracking>bm.exe deploy-all -DefinitionFile:BAMSample.xls”, you can get “Old format or invalid type library” error:

Microsoft (R) Business Activity Monitoring Utility Version 3.8.368.0 Copyright (C) 2006 Microsoft Corporation.  All rights reserved.
Using ‘BAMPrimaryImport’ BAM Primary Import database on server ‘…’…
ERROR: Failed to open BAM Excel workbook file – ‘C:\Program Files\Microsoft BizTalk Server 2009\Tracking\MyBAMSample.xls’. Old format or invalid type library. (Exception from HRESULT: 0×80028018 (TYPE_E_INVDATAREAD))

CAUSE

  • When you use a locale setting other than English (United States) while installing the English version of BizTalk Server, your deployment of the Bam.xls file generates an "fnGetString" or "Old format or invalid type library" error.

SOLUTION

  • Change the computer locale to English (United States) and re-run the bm deploy command at the command prompt.
  • To change your computer locale to English (United States)
    • Click Start, click Control Panel, and then click Regional and Language Options.
    • In the Regional and Language Options dialog, on the Regional Options tab, choose the English (United States) locale from the drop down box, and click OK.

 

Other Solution:

  • To work around this problem, you can either install the Office Multilingual User Interface (MUI) pack which can be found on http://www.microsoft.com/ or change your locale to US-EN just for the duration of the deployment.

Tags: BizTalk | BAM | Deploy | Errors and Warnings, Causes and Solutions


BizTalk BAM activity Deploy problem – The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

$
0
0

When trying to deploy BAM activity into a new BizTalk Environment:

  • “C:\Program Files (x86)\Microsoft BizTalk Server …\Tracking>bm.exe deploy-all -DefinitionFile:BAM.SMSExpress.xls”

It happened to me the following error:

“Microsoft (R) Business Activity Monitoring Utility Version 3.9.469.0
Copyright (C) Microsoft Corporation. All rights reserved.
Using ‘BAMPrimaryImport’ BAM Primary Import database on server ‘BTS2010LAB01′…
Deploying Activity… Done.
Deploying View… ERROR: The BAM deployment failed.
The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

CAUSE

  • One of the possible causes of the problem is that Analysis Services form BAM aggregations was not configured.

SOLUTION

  • Open BizTalk Server Configuration Console. (All Programs → Microsoft BizTalk Server … → BizTalk Server Configuration)
  • Select the option “BAM Tools” in the left menu and check the option “Enable Analysis Services for BAM Aggregations”
  • Configure “BAM Analysis Database” and “BAM Star Schema Database”
    BAM-Tools-Configuration
  • Click in “Apply Configuration”
  • On the Summary screen, Click “Next”
    BAM-Tools-Configuration-summary
  • On the completed screen, click “Finish”

Tags: BizTalk | BAM | Deploy | Errors and Warnings, Causes and Solutions



BizTalk 2010 – BAM Tools configuration error: Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine.

$
0
0

When trying to configure BAM features in BizTalk Server 2010 (64 bit) with a remote SQL Server 2008 R2 (64 bit) I was getting these two errors:

When trying to configure “BAM Archive Database”

Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine. Please install Microsoft SQL Server 2008 Integration Services. (Microsoft.BizTalk.BAM.CfgExtHelper.ToolsHelper)

BAM-Archive-Database-error

And when trying to configure “BAM Analysis Database”

Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine. Please install Microsoft SQL Server Integration Services 2008 with SP1 or higher. (Microsoft.BizTalk.BAM.CfgExtHelper.ToolsHelper)

BAM-Analysis-Database-error

Both errors with the same additional information:

——————————
ADDITIONAL INFORMATION:
Could not load file or assembly ‘Microsoft.SqlServer.ManagedDTS, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91’ or one of its dependencies. The system cannot find the file specified. (Microsoft.BizTalk.Bam.CfgExtHelper)

CAUSE

As Kent Weare said in his post: “This error makes it sound like you need to install SQL Server Integration Services (SSIS) which is a little misleading.”

The documentation on MSDN indicates the following:

What 64-bit SQL Server components are required to configure BAM tools?

The configuration wizard is a 32-bit process; therefore it requires certain components which allow it to communicate with 64-bit SQL Server. You must install the following SQL Server client components to enable configuration of BAM tools:

  • Connectivity Components
  • Management Tools
  • Legacy Components

SOLUTION

  • Install the SQL Server 2008 R2 Management Tools (Basic & Complete) features as shown in the screenshot onto the BizTalk Server 2010:

SQL-Server-2008-R2-Management-Tools

Tags: BizTalk | BAM | Configuration | Errors and Warnings, Causes and Solutions


BAM Portal Errors – Failed to list permissions for BAM view. System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values

$
0
0

I decided to put the name of the error in the post title … but this post also could be called “Why you shouldn’t delete Active Directory accounts (if you don’t know where it is being used!)”… but before I tell you why, let me try to explain the problem and put some context on it.

Last week I had a funny surprised when I tried to access my BAM Portal in my BizTalk Server 2006 environment

An unspecified error has occured.
Use the navigation bar on the left to access Business Activity Monitoring views.
If the problem persist, contact you System Administrator.

BAM-portal-error

I just love this type of errors! Because I’m also one of the System Administrator, meaning that I was f*$#%& Sorriso.

My first reaction is that it could have been some connectivity problem, and as the error indicates, I tried to navigate the remaining views to see if the problem remained… and surprisingly this problem occurred only in certain views!

BAM-portal

PROBLEM

So at this point I knew it was happening some problem. Of course the first thing we should do is to check the Event Viewer to see if we can find more details about the error and I found three errors related between themselves and associated with BAM:

BAM-Portal-Errors-Event-Viewer

Starting from below:

  • The first error was:

Current User: DOMAIN\sandro
EXCEPTION:
Microsoft.BizTalk.Bam.Management.BamManagerException: Failed to list permissions for BAM view. —> System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values.
at System.Data.SqlClient.SqlBuffer.get_String()
at System.Data.SqlClient.SqlDataReader.GetString(Int32 i)
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
— End of inner exception stack trace —
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager, Boolean throwIfNoPermissions)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager)
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)

Observation: Impossible, I’m BizTalk Administrator but most important my user is the owner of this views!!!

  • The second error was:

Current User: DOMAIN\sandro
EXCEPTION: System.Web.Services.Protocols.SoapException: Internal Server Error.

Observation: says absolutely nothing … trash!

  • And finally the third error was:

(BAMPortal.PortalApplication) Void LogAllErrors(System.Exception[]): System.Web.HttpException: Error executing child request for /BAM/Pages/Search.aspx. —> System.Web.HttpUnhandledException: Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> System.Web.Services.Protocols.SoapException: System.Web.Services.Protocols.SoapException: Internal Server Error.
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Microsoft.BizTalk.Bam.WebServices.ManagementService.BamManagementService.GetViewDetailsAsXml(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.FetchViewDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.GetBamDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.BuildColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.ColumnsOfType(ColumnTypes type)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureInstanceColumns()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.get_InstanceColumns()
at BAMPortal.ColumnsChooser_ascx.GetColumns()
at BAMPortal.ColumnsChooser_ascx.GetAvailableColumns()
at BAMPortal.ColumnsChooser_ascx.ReconcileColumns()
at BAMPortal.ColumnsChooser_ascx.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
— End of inner exception stack trace —
at System.Web.UI.Page.HandleError(Exception e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_search_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
— End of inner exception stack trace —
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
at System.Web.HttpServerUtility.Execute(String path, TextWriter writer, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path)
at BAMPortal.navbar_ascx.TreeViewNav_NodeClicked(Object sender, TreeNodeEventArgs eventArgs)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.OnTreeNodeClicked(TreeNode node)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.RaisePostBackEvent(String eventArgument)
at System.Web.UI.WebControls.TreeView.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_view_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Observation: much information … yet without giving me many tips on the problem that was happening.

Actually the first error is what brings us a better sense of the error… it’s a permission problem to access the view… but why? and what was really causing this problem?

CAUSE
  • BAM Management Utility (BM.exe) doesn’t provide the capability to grant group permissions to BAM views instead you need to use user’s accounts. So these issues may occur if the user account which was granted permission to BAM objects are deleted from Active Directory or from Local Computers.

When you access to a certain view in BAM Portal the services invoked by the Portal will try to check the permission for ALL the users associated to this specific view and not only my user, so if a user was deleted from AD or from the local computer without first removing it from the view, the services will fail while attempting to map the account name with Security ID and you will get with this annoying problem: “Data is Null. This method or property cannot be called on Null values.”

You may experience any one or more of the following symptoms:

  • When you access to certain views in the BAM Portal
  • When you try to execute any kind of operation using BM.exe tool against certain views, like “bm.exe get-accounts”; “bm.exe remove-account” or “bm.exe remove-view”
SOLUTION
  • You have to manually delete those user accounts from SQL Server.
How can I really solved this problem?

So now I know the problem, the cause and the solution… but is it that simple?
… NO of course!

First problem: How can I really know with account(s) is causing the problem?

You can have many users in your organization associated with the view and may have passed months or years since the last time we associate the users account to this view… so it is really a problem to find with user is causing the problem.

  • Option 1: you can ask!
    • I know that if I ask who was deleting accounts in the AD or which accounts have been deleted, I will get the typical response… no one or no account has been deleted!!! Don’t go there is an endless road.
  • Option 2: You can use BM.exe!
    • Unfortunately we also can’t use BM tool to ask with account have permission to this view (bm.exe get-accounts), we get the following error:
      • ERROR: Failed to list permissions for BAM view.
        Data is Null. This method or property cannot be called on Null values.
  • Option 3: See in the documentation
    • Another endless road Sorriso. This type of documentation should exist, but few companies actually have them!

I finally decided to try to make a query directly in the BAM Primary Import database (BAMPrimaryImport) to attempt to get a list of all the account that were associated with this view in order to validate with my system administrators if they all existed in AD, this was the result:

WITH Query AS (
SELECT
    [UserName] = CASE princ.[type]
                    WHEN 'S' THEN princ.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE princ.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = princ.[name],
    [Role] = null,
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --database user
    sys.database_principals princ
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on princ.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = princ.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col ON col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
WHERE
    princ.[type] in ('S','U')
UNION
--List all access provisioned to a sql user or windows user/group through a database or application role
SELECT
    [UserName] = CASE memberprinc.[type]
                    WHEN 'S' THEN memberprinc.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE memberprinc.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = memberprinc.[name],
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Role/member associations
    sys.database_role_members members
JOIN
    --Roles
    sys.database_principals roleprinc ON roleprinc.[principal_id] = members.[role_principal_id]
JOIN
    --Role members (database users)
    sys.database_principals memberprinc ON memberprinc.[principal_id] = members.[member_principal_id]
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on memberprinc.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
UNION
--List all access provisioned to the public role, which everyone gets by default
SELECT
    [UserName] = '{All Users}',
    [UserType] = '{All Users}',
    [DatabaseUserName] = '{All Users}',
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Roles
    sys.database_principals roleprinc
LEFT JOIN
    --Role permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
JOIN
    --All objects
    sys.objects obj ON obj.[object_id] = perm.[major_id]
WHERE
    --Only roles
    roleprinc.[type] = 'R' AND
    --Only public role
    roleprinc.[name] = 'public' AND
    --Only objects of ours, not the MS objects
    obj.is_ms_shipped = 0
)
SELECT * From  Query
WHERE Role like '%name_of_the_view%' AND ObjectType like 'View'

BAM-query-result

It may not be sophisticated or the best way but it served my purposes and was much easier and effective than be looking one by one in the SQL Server management Studio.

Now that I had a list of users, it was easy to validate with the system administrators which account had been deleted.

After so much effort and work trying to find the user that was causing the problem I found a simple and more effective solution to this problem in this post on MSDN Blogs: BizTalk Error: BAM Management Utility Errors, using this query:

USE BAMPrimaryImport
GO
Select Name,SID,SUser_SName(SID) as UserAccount from sysusers
WHERE ISLogin = 1 AND issqluser = 0 AND isntuser = 1

In the result for the DB roles, the third column for Windows users (and groups) should display the User Name (or group name) and can’t be null.

So if you execute the above query it will give the user(s) account(s) which was deleted from Domain Controller or Local Computer.

Second problem: How can I manually delete the account?

Yep … this question may seem a bit ridiculous … but the truth is, manually delete the account was not so simple after all… at least for me it wasn’t.

You can’t just delete that account from SQL Server. To do that you need to follow the steps:

  • Go to SQL Management Studio, expand the BAMPrimaryImport Database
  • Go to Security –> Roles –> Database Roles –> BAM_ManagementWS
  • Right click BAM_ManagementWS and go to properties option which will open a new window (Database Role Properties – BAM_ManagementWS)

BAM-ManagementWS

  • In that window go to Securables, choose the NT User account which needs to be deleted.

BAM-ManagementWS-properties-Securables

  • In the bottom portion you will see that NT User Account having “View Definition” Permission. Uncheck that option then click OK.
  • Now you should able to delete the user from Security –> User –> <NT User Account>

BAM-User-delete

However if you try to delete the user without following these steps, like I did :) , you get stuck in another huge problem:

TITLE: Microsoft SQL Server Management Studio
——————————
Drop failed for User ‘domain\user’.  (Microsoft.SqlServer.Smo)
——————————
ADDITIONAL INFORMATION:
An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)
——————————
The database principal has granted or denied permissions to objects in the database and cannot be dropped. (Microsoft SQL Server, Error: 15284)

So we need now to manual revoke the Grant access to BAM_ManagementWS by execute the following query:

REVOKE VIEW DEFINITION ON USER::[domain\user] TO [BAM_ManagementWS] AS [domain\user]
GO
Security Considerations for the BAM Portal

Using the principle of least privilege, user accounts should have restrictive permissions to perform routine tasks in the BAM portal. Keep the following points in mind as you set up your user accounts for BAM to balance security with appropriate access for users.

User accounts

User accounts with minimum permissions are not able to use the BAM portal distributed navigation feature. To be able to use this feature, these accounts must have sufficient permissions to allow access to the Web services on the remote computer as well as on the local computer.

User accounts for the BAM Web services must have permissions to access all referenced databases and must be a member of the BAM_ManagementWS role in the referenced databases.

For the following user types, you should be aware of these considerations:

  • Domain Users: These users must have access permissions on remote computers that host BAM Primary Import databases that are being accessed.
  • Local User: Users who are assigned this role cannot use distributed navigation.
Administrator accounts

Administrators must be members of the securityadmin or sysadmin groups to grant permissions to domain users.

To run the BAM Management utility, you must be at least a database operator for the BAM databases.

Final Notes

I would like to thank Nino Crudele for the help that he gave me to solve this problem and to my coworker and friend José Barbosa who helped me creating these SQL scripts and as result solving the problem.

And a final note for all system administrator… please don’t delete Active Directory accounts… instead disable them!!


BizTalk BAM Portal URL authorization failed for the request (Event code 4007)

$
0
0

Last day I was able to convince my client to use, for the first time, BAM for tracking and monitoring of specific processes. As I anticipated, 1-hour work resulted in a major impact (for better) on the people responsible for these tasks in the organization.

However after I deploy my BAM Definition and tracking profile in production environment and everything was working well, i.e., processes were running successfully and data was being tracking (I was able to see the tracking data in database)…

If you don’t know, and contrary to what I thought also, the only user that always has access to the view and cannot be added to or removed from the view(s) is the Database Owner (BAMPrimaryImport). So the user that made the deploy of this BAM Definition don’t necessarily have access to this particular view in the BAM Portal!

In this particular case I’m also the Database Owner, but when I try to access to BAM Portal the access was constantly being denied me and always asking to enter my credentials when browsing to it… even when I gave access to another domain user to this view and tested with these credentials the problem remained.

After examining the logs in event I found this information message:

BAM-Web-Event-Information

With the following details:

Event code: 4007
Event message: URL authorization failed for the request.
Event time: 23-04-2013 15:55:52
Event time (UTC): 23-04-2013 14:55:52
Event ID: 053c6e752b6a4de8ae400a9a9d7d26b1
Event sequence: 10
Event occurrence: 9
Event detail code: 0

Application information:
Application domain: /LM/W3SVC/1/ROOT/BAM-1-130112015742350508
Trust level: BAMPortal_Minimal
Application Virtual Path: /BAM
Application Path: D:\Program Files (x86)\Microsoft BizTalk Server 2010\BAMPortal\
Machine name: MyMachine

Process information:
Process ID: 9560
Process name: w3wp.exe
Account name: DOMAIN\bts-bam-ap

Request information:
Request URL: http://localhost/BAM
Request path: /BAM
User host address: 192.168.***.***
User: DOMAIN\MYUSER
Is authenticated: True
Authentication Type: Negotiate
Thread account name: DOMAIN\MYUSER

Custom event details:

CAUSE

Well, unfortunately, this problem or similar problems can happen for many reasons:

Fortunately for me I have an E2E test environment which is an almost exact replica of PROD with which I could compare to see what was the problem and that was working well Sorriso.

One thing I was sure: I had permission problems!

The first thing was to analyze the basic settings of the application pool like: credentials or .net version and so on… however everything was properly configured and equal to the test environment.

After a few minutes I remembered the basics… if you remember the BizTalk Configuration experience, you use the BizTalk Server configuration tool to specify whether BAM is enabled, and to specify the Web service accounts, the Windows groups that can view portal, and the Web site that will host the portal.

BAM-Portal-Configuration

That you also can see in “.Net Authorization Rules” under BAM website:

BAM-IIS-Net-Authorization-Rules

Using the principle of least privilege, user accounts should have restrictive permissions to perform routine tasks in the BAM portal. BizTalk BAM Portal Users is the group, at least for me but this may change according to your configurations, where you defined the users or groups that can access to BAM Portal Web site.

SOLUTION

In my case, after checking in AD, there was no one configured to have access to BAM Portal.

To solve this problem you have to configure the users or groups that you want to have access to BAM Portal under the “BizTalk BAM Portal Users” in your Active Directory.

After this operation everything work fine! Exactly as it should.


BizTalk Server: Cannot see my BAM View in BAM Portal after I deploy it

$
0
0

It´s a common factor that before users can view data from a BAM model in the BAM Portal website, they must be granted access to the views. However, and contrary to what I thought, the user that makes the deploy of the BAM Definition don’t necessarily have access to this particular view in the BAM Portal! (I was being misled because usually I am the owner… hehe)

BAM-Portal-Missing-View

Instead the only user that always has access to the view and cannot be added to or removed from the view(s) is the Database Owner (BAMPrimaryImport)!

You can check who the database owner is by running the following query:

SELECT SUSER_SNAME(owner_sid),name
FROM sys.DATABASES

So after I run this query I realize that another user was the owner and to solve this “problem” I had to give permissions to my user to access this View in BAM Portal by running the following BAM Management Utility command:

bm.exe add-account -AccountName:DOMAIN\USER -View:MyView

BAM-Portal-With-View

Mystery solved Sorriso


BAM Portal: This website uses a data provider that may be unsafe. If you trust the website, click OK, otherwise click Cancel

$
0
0

When we try to access BAM Portal, sometimes Office Web Components Display Cross-Domain warning messages such as:

“This Web site uses a data provider that may be unsafe. If you trust the Web site, click OK, otherwise click Cancel.”

This-website-uses-data-provider-that-may-be-unsafe

Or this other common warning messages

“This page accesses data on another domain. Do you want to allow this?”

Or even:

“Failed to get data. If available, errors returned from the provider are listed below… Safety settings on this machine prohibit accessing a data source on another domain.”

CAUSE

This are some of the normal warning message that the Microsoft Office 2003 Web Components displays when a user try to accesses to BAM Portal, some because a scripted Web page tries to run ADO code from an untrusted provider (accessing data from a domain other than the one from which the Web page originates)

For ADO 2.7 and earlier you may receive: “This page accesses data on another domain. Do you want to allow this?”

However, for ADO 2.8 and later, the preceding message no longer appears. Instead, the following message appears in this context: “This Web site uses a data provider that may be unsafe. If you trust the Web site, click OK, otherwise click Cancel.”

The purpose for the alert is to prevent a Web page author from directing users to a potentially malicious page that uses the security context of the user to access data for which the author does not have access.

SOLUTION

To avoid this messages in Microsoft Internet Explorer, you can add a secure Web site to your Trusted Sites zone on the Security tab of the Internet Options dialog box:

  • In the Internet Explorer window, click “Tools”, then click “Internet Options”.
  • Click the “Security” tab, and then select the “Trusted sites” zone.

Add-BAM-Portal-Trusted-Sites-Zone-Security-tab

  • Click “Sites” button, and then “Add” the BAM Portal to the Trusted sites zone

Add-BAM-Portal-Trusted-Sites-Zone


BAM Portal: “Failed to get data. If available, errors returned from the provider are listed below” when using IE10

$
0
0

After installing some upgrades in the BizTalk Server machine, in this particular case after installing Internet Explorer 10 I started to receive the following error message when I try to access BAM Portal:

“Failed to get data. If available, errors returned from the provider are listed below”

BAM-Portal-Failed-to-get-data

However without no error was listing… so what’s the problem and how can I solve it?

Additional I also get this error messages and bad behaviors:

BAM-Portal-Pivot-Table-Provider-MSOLAP-error

BAM-Portal-Pivot-Table-Provider-MSOLAP-error-2

CAUSE

Sometimes a website you’re visiting, in this case BAM Portal, doesn’t look like you expect it to. Images might not show up, menus might be out of place, and text boxes could be jumbled together. This can be caused by a compatibility problem between Internet Explorer and the site you’re on. When a site is incompatible with Internet Explorer, you’ll see the Compatibility View button in the Address bar.

Note: Not all website display problems are caused by browser incompatibility. Interrupted Internet connections, heavy traffic or network connections can also affect how a page is displayed.

SOLUTION

To solve this problem you must turn on Compatibility View for BAM Portal in the IE10. You can do this by:

  • Clicking in the “Compatibility View” button that appears in the Address bar to display the site in Compatibility View.

BAM-Portal-tap-Compatibility-View

Note: If you don’t see the button, there’s no need to turn on Compatibility View.

Once you turn on Compatibility View, Internet Explorer will automatically show that site in Compatibility View each time you visit. You can turn it off by tapping or clicking the button again. Or, you can clear the entire list of sites using Compatibility View by deleting your browsing history.


BAM: The cube "MyView" was not processed. Please run the cube DTS to process the cube

$
0
0

After I deploy a BAM activity to a new BizTalk Server 2010 environment I create a job called “BAM Generic Import data” to import all BAM data information to OLAP Cubes that will be presented on Aggregations tab in the BAM Portal.

However when I try to manual execute the job I got an error messages saying:

“The cube “MyView” was not processed. Please run the cube DTS to process the cube”

When I went to the event viewer I saw several Bam Event Provider warning messages with the following details:

Bam-Event-Provider-warning-messages

Log                Job History (BAM Generic Import data)
Step ID                0

Server                MyServer\BIZTALK
Job Name                BAM Generic Import data
Step Name                (Job outcome)
Duration                00:00:00
Sql Severity                0
Sql Message ID                0
Operator Emailed
Operator Net sent
Operator Paged
Retries Attempted                0

Message

The job failed. Unable to determine if the owner (Domain\User) of job BAM Generic Import data has server access (reason: Could not obtain information about Windows NT group/user ‘Domain\User’, error code 0×2. [SQLSTATE 42000] (Error 15404)).

CAUSE

This error most likely when the machine account that runs the job (BAM Generic Import data) doesn’t have permission to query the AD.

Job-BAM-Generic-Import-Data-ower

SOLUTION 1

I would recommend requesting to the AD administrator access to this user or change owner that runs this job to a low-privileged domain account that has proper permissions on the AD – Members of the Domain should be enough.

SOLUTION 2

However in my case the AD administrator was in vacation and the other unavailable and my user account indeed didn’t have permission to query the AD (don’t ask me why) so the only solution that I found to try to solve the problem was to change the owner to the SQL Login System administrator (sa) that, lucky, wasn’t locked.

Job-BAM-Generic-Import-Data-ower-fixed

And problem solved … at least momentarily until we can apply the solution 1



BizTalk BAM activity Deploy problem – The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

$
0
0

When trying to deploy BAM activity into a new BizTalk Environment:

  • “C:\Program Files (x86)\Microsoft BizTalk Server …\Tracking>bm.exe deploy-all -DefinitionFile:BAM.SMSExpress.xls”

It happened to me the following error:

“Microsoft (R) Business Activity Monitoring Utility Version 3.9.469.0
Copyright (C) Microsoft Corporation. All rights reserved.
Using ‘BAMPrimaryImport’ BAM Primary Import database on server ‘BTS2010LAB01’…
Deploying Activity… Done.
Deploying View… ERROR: The BAM deployment failed.
The BAM Star Schema database has not been configured. Run bm.exe setup-databases to configure the database.

CAUSE

  • One of the possible causes of the problem is that Analysis Services form BAM aggregations was not configured.

SOLUTION

  • Open BizTalk Server Configuration Console. (All Programs → Microsoft BizTalk Server … → BizTalk Server Configuration)
  • Select the option “BAM Tools” in the left menu and check the option “Enable Analysis Services for BAM Aggregations”
  • Configure “BAM Analysis Database” and “BAM Star Schema Database”
    BAM-Tools-Configuration
  • Click in “Apply Configuration”
  • On the Summary screen, Click “Next”
    BAM-Tools-Configuration-summary
  • On the completed screen, click “Finish”

Tags: BizTalk | BAM | Deploy | Errors and Warnings, Causes and Solutions


BizTalk 2010 – BAM Tools configuration error: Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine.

$
0
0

When trying to configure BAM features in BizTalk Server 2010 (64 bit) with a remote SQL Server 2008 R2 (64 bit) I was getting these two errors:

When trying to configure “BAM Archive Database”

Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine. Please install Microsoft SQL Server 2008 Integration Services. (Microsoft.BizTalk.BAM.CfgExtHelper.ToolsHelper)

BAM-Archive-Database-error

And when trying to configure “BAM Analysis Database”

Microsoft SQL Server Data Transformation Services (DTS) 2008 with SP1 or higher for BAM Archiving is not installed on the local machine. Please install Microsoft SQL Server Integration Services 2008 with SP1 or higher. (Microsoft.BizTalk.BAM.CfgExtHelper.ToolsHelper)

BAM-Analysis-Database-error

Both errors with the same additional information:

——————————
ADDITIONAL INFORMATION:
Could not load file or assembly ‘Microsoft.SqlServer.ManagedDTS, Version=10.0.0.0, Culture=neutral, PublicKeyToken=89845dcd8080cc91’ or one of its dependencies. The system cannot find the file specified. (Microsoft.BizTalk.Bam.CfgExtHelper)

CAUSE

As Kent Weare said in his post: “This error makes it sound like you need to install SQL Server Integration Services (SSIS) which is a little misleading.”

The documentation on MSDN indicates the following:

What 64-bit SQL Server components are required to configure BAM tools?

The configuration wizard is a 32-bit process; therefore it requires certain components which allow it to communicate with 64-bit SQL Server. You must install the following SQL Server client components to enable configuration of BAM tools:

  • Connectivity Components
  • Management Tools
  • Legacy Components

SOLUTION

  • Install the SQL Server 2008 R2 Management Tools (Basic & Complete) features as shown in the screenshot onto the BizTalk Server 2010:

SQL-Server-2008-R2-Management-Tools

Tags: BizTalk | BAM | Configuration | Errors and Warnings, Causes and Solutions


BAM Portal Errors – Failed to list permissions for BAM view. System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values

$
0
0

I decided to put the name of the error in the post title … but this post also could be called “Why you shouldn’t delete Active Directory accounts (if you don’t know where it is being used!)”… but before I tell you why, let me try to explain the problem and put some context on it.

Last week I had a funny surprised when I tried to access my BAM Portal in my BizTalk Server 2006 environment

An unspecified error has occured.
Use the navigation bar on the left to access Business Activity Monitoring views.
If the problem persist, contact you System Administrator.

BAM-portal-error

I just love this type of errors! Because I’m also one of the System Administrator, meaning that I was f*$#%& Sorriso.

My first reaction is that it could have been some connectivity problem, and as the error indicates, I tried to navigate the remaining views to see if the problem remained… and surprisingly this problem occurred only in certain views!

BAM-portal

PROBLEM

So at this point I knew it was happening some problem. Of course the first thing we should do is to check the Event Viewer to see if we can find more details about the error and I found three errors related between themselves and associated with BAM:

BAM-Portal-Errors-Event-Viewer

Starting from below:

  • The first error was:

Current User: DOMAIN\sandro
EXCEPTION:
Microsoft.BizTalk.Bam.Management.BamManagerException: Failed to list permissions for BAM view. —> System.Data.SqlTypes.SqlNullValueException: Data is Null. This method or property cannot be called on Null values.
at System.Data.SqlClient.SqlBuffer.get_String()
at System.Data.SqlClient.SqlDataReader.GetString(Int32 i)
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
— End of inner exception stack trace —
at Microsoft.BizTalk.Bam.Management.SecurityModule.ListViewPermissions(String viewName, String& dboUsername)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager, Boolean throwIfNoPermissions)
at Microsoft.BizTalk.Bam.WebServices.SecurityHelper.VerifyViewPermissions(String viewName, IPrincipal user, BamManager bamManager)
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)

Observation: Impossible, I’m BizTalk Administrator but most important my user is the owner of this views!!!

  • The second error was:

Current User: DOMAIN\sandro
EXCEPTION: System.Web.Services.Protocols.SoapException: Internal Server Error.

Observation: says absolutely nothing … trash!

  • And finally the third error was:

(BAMPortal.PortalApplication) Void LogAllErrors(System.Exception[]): System.Web.HttpException: Error executing child request for /BAM/Pages/Search.aspx. —> System.Web.HttpUnhandledException: Exception of type ‘System.Web.HttpUnhandledException’ was thrown. —> System.Web.Services.Protocols.SoapException: System.Web.Services.Protocols.SoapException: Internal Server Error.
at Microsoft.BizTalk.Bam.WebServices.Management.BamManagementService.GetViewDetailsAsXml(String viewName)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at Microsoft.BizTalk.Bam.WebServices.ManagementService.BamManagementService.GetViewDetailsAsXml(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.FetchViewDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.BamDefinitionCache.GetBamDefinition(String viewName)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.BuildColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureColumnsCollection()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.ColumnsOfType(ColumnTypes type)
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.EnsureInstanceColumns()
at Microsoft.BizTalk.Bam.Portal.DataAccess.Activity.get_InstanceColumns()
at BAMPortal.ColumnsChooser_ascx.GetColumns()
at BAMPortal.ColumnsChooser_ascx.GetAvailableColumns()
at BAMPortal.ColumnsChooser_ascx.ReconcileColumns()
at BAMPortal.ColumnsChooser_ascx.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
— End of inner exception stack trace —
at System.Web.UI.Page.HandleError(Exception e)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_search_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
— End of inner exception stack trace —
at System.Web.HttpServerUtility.ExecuteInternal(IHttpHandler handler, TextWriter writer, Boolean preserveForm, Boolean setPreviousPage, VirtualPath path, VirtualPath filePath, String physPath, Exception error, String queryStringOverride)
at System.Web.HttpServerUtility.Execute(String path, TextWriter writer, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path, Boolean preserveForm)
at System.Web.HttpServerUtility.Transfer(String path)
at BAMPortal.navbar_ascx.TreeViewNav_NodeClicked(Object sender, TreeNodeEventArgs eventArgs)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.OnTreeNodeClicked(TreeNode node)
at Microsoft.BizTalk.Bam.Portal.ClickableTreeView.RaisePostBackEvent(String eventArgument)
at System.Web.UI.WebControls.TreeView.System.Web.UI.IPostBackEventHandler.RaisePostBackEvent(String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(IPostBackEventHandler sourceControl, String eventArgument)
at System.Web.UI.Page.RaisePostBackEvent(NameValueCollection postData)
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at System.Web.UI.Page.ProcessRequest()
at System.Web.UI.Page.ProcessRequestWithNoAssert(HttpContext context)
at System.Web.UI.Page.ProcessRequest(HttpContext context)
at ASP.pages_view_aspx.ProcessRequest(HttpContext context)
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

Observation: much information … yet without giving me many tips on the problem that was happening.

Actually the first error is what brings us a better sense of the error… it’s a permission problem to access the view… but why? and what was really causing this problem?

CAUSE
  • BAM Management Utility (BM.exe) doesn’t provide the capability to grant group permissions to BAM views instead you need to use user’s accounts. So these issues may occur if the user account which was granted permission to BAM objects are deleted from Active Directory or from Local Computers.

When you access to a certain view in BAM Portal the services invoked by the Portal will try to check the permission for ALL the users associated to this specific view and not only my user, so if a user was deleted from AD or from the local computer without first removing it from the view, the services will fail while attempting to map the account name with Security ID and you will get with this annoying problem: “Data is Null. This method or property cannot be called on Null values.”

You may experience any one or more of the following symptoms:

  • When you access to certain views in the BAM Portal
  • When you try to execute any kind of operation using BM.exe tool against certain views, like “bm.exe get-accounts”; “bm.exe remove-account” or “bm.exe remove-view”
SOLUTION
  • You have to manually delete those user accounts from SQL Server.
How can I really solved this problem?

So now I know the problem, the cause and the solution… but is it that simple?
… NO of course!

First problem: How can I really know with account(s) is causing the problem?

You can have many users in your organization associated with the view and may have passed months or years since the last time we associate the users account to this view… so it is really a problem to find with user is causing the problem.

  • Option 1: you can ask!
    • I know that if I ask who was deleting accounts in the AD or which accounts have been deleted, I will get the typical response… no one or no account has been deleted!!! Don’t go there is an endless road.
  • Option 2: You can use BM.exe!
    • Unfortunately we also can’t use BM tool to ask with account have permission to this view (bm.exe get-accounts), we get the following error:
      • ERROR: Failed to list permissions for BAM view.
        Data is Null. This method or property cannot be called on Null values.
  • Option 3: See in the documentation
    • Another endless road Sorriso. This type of documentation should exist, but few companies actually have them!

I finally decided to try to make a query directly in the BAM Primary Import database (BAMPrimaryImport) to attempt to get a list of all the account that were associated with this view in order to validate with my system administrators if they all existed in AD, this was the result:

WITH Query AS (
SELECT
    [UserName] = CASE princ.[type]
                    WHEN 'S' THEN princ.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE princ.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = princ.[name],
    [Role] = null,
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --database user
    sys.database_principals princ
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on princ.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = princ.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col ON col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
WHERE
    princ.[type] in ('S','U')
UNION
--List all access provisioned to a sql user or windows user/group through a database or application role
SELECT
    [UserName] = CASE memberprinc.[type]
                    WHEN 'S' THEN memberprinc.[name]
                    WHEN 'U' THEN ulogin.[name] COLLATE Latin1_General_CI_AI
                 END,
    [UserType] = CASE memberprinc.[type]
                    WHEN 'S' THEN 'SQL User'
                    WHEN 'U' THEN 'Windows User'
                 END,
    [DatabaseUserName] = memberprinc.[name],
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Role/member associations
    sys.database_role_members members
JOIN
    --Roles
    sys.database_principals roleprinc ON roleprinc.[principal_id] = members.[role_principal_id]
JOIN
    --Role members (database users)
    sys.database_principals memberprinc ON memberprinc.[principal_id] = members.[member_principal_id]
LEFT JOIN
    --Login accounts
    sys.login_token ulogin on memberprinc.[sid] = ulogin.[sid]
LEFT JOIN
    --Permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
LEFT JOIN
    sys.objects obj ON perm.[major_id] = obj.[object_id]
UNION
--List all access provisioned to the public role, which everyone gets by default
SELECT
    [UserName] = '{All Users}',
    [UserType] = '{All Users}',
    [DatabaseUserName] = '{All Users}',
    [Role] = roleprinc.[name],
    [PermissionType] = perm.[permission_name],
    [PermissionState] = perm.[state_desc],
    [ObjectType] = obj.type_desc,--perm.[class_desc],
    [ObjectName] = OBJECT_NAME(perm.major_id),
    [ColumnName] = col.[name]
FROM
    --Roles
    sys.database_principals roleprinc
LEFT JOIN
    --Role permissions
    sys.database_permissions perm ON perm.[grantee_principal_id] = roleprinc.[principal_id]
LEFT JOIN
    --Table columns
    sys.columns col on col.[object_id] = perm.major_id
                    AND col.[column_id] = perm.[minor_id]
JOIN
    --All objects
    sys.objects obj ON obj.[object_id] = perm.[major_id]
WHERE
    --Only roles
    roleprinc.[type] = 'R' AND
    --Only public role
    roleprinc.[name] = 'public' AND
    --Only objects of ours, not the MS objects
    obj.is_ms_shipped = 0
)
SELECT * From  Query
WHERE Role like '%name_of_the_view%' AND ObjectType like 'View'

BAM-query-result

It may not be sophisticated or the best way but it served my purposes and was much easier and effective than be looking one by one in the SQL Server management Studio.

Now that I had a list of users, it was easy to validate with the system administrators which account had been deleted.

After so much effort and work trying to find the user that was causing the problem I found a simple and more effective solution to this problem in this post on MSDN Blogs: BizTalk Error: BAM Management Utility Errors, using this query:

USE BAMPrimaryImport
GO
Select Name,SID,SUser_SName(SID) as UserAccount from sysusers
WHERE ISLogin = 1 AND issqluser = 0 AND isntuser = 1

In the result for the DB roles, the third column for Windows users (and groups) should display the User Name (or group name) and can’t be null.

So if you execute the above query it will give the user(s) account(s) which was deleted from Domain Controller or Local Computer.

Second problem: How can I manually delete the account?

Yep … this question may seem a bit ridiculous … but the truth is, manually delete the account was not so simple after all… at least for me it wasn’t.

You can’t just delete that account from SQL Server. To do that you need to follow the steps:

  • Go to SQL Management Studio, expand the BAMPrimaryImport Database
  • Go to Security –> Roles –> Database Roles –> BAM_ManagementWS
  • Right click BAM_ManagementWS and go to properties option which will open a new window (Database Role Properties – BAM_ManagementWS)

BAM-ManagementWS

  • In that window go to Securables, choose the NT User account which needs to be deleted.

BAM-ManagementWS-properties-Securables

  • In the bottom portion you will see that NT User Account having “View Definition” Permission. Uncheck that option then click OK.
  • Now you should able to delete the user from Security –> User –> <NT User Account>

BAM-User-delete

However if you try to delete the user without following these steps, like I did:), you get stuck in another huge problem:

TITLE: Microsoft SQL Server Management Studio
——————————
Drop failed for User ‘domain\user’.  (Microsoft.SqlServer.Smo)
——————————
ADDITIONAL INFORMATION:
An exception occurred while executing a Transact-SQL statement or batch. (Microsoft.SqlServer.ConnectionInfo)
——————————
The database principal has granted or denied permissions to objects in the database and cannot be dropped. (Microsoft SQL Server, Error: 15284)

So we need now to manual revoke the Grant access to BAM_ManagementWS by execute the following query:

REVOKE VIEW DEFINITION ON USER::[domain\user] TO [BAM_ManagementWS] AS [domain\user]
GO
Security Considerations for the BAM Portal

Using the principle of least privilege, user accounts should have restrictive permissions to perform routine tasks in the BAM portal. Keep the following points in mind as you set up your user accounts for BAM to balance security with appropriate access for users.

User accounts

User accounts with minimum permissions are not able to use the BAM portal distributed navigation feature. To be able to use this feature, these accounts must have sufficient permissions to allow access to the Web services on the remote computer as well as on the local computer.

User accounts for the BAM Web services must have permissions to access all referenced databases and must be a member of the BAM_ManagementWS role in the referenced databases.

For the following user types, you should be aware of these considerations:

  • Domain Users: These users must have access permissions on remote computers that host BAM Primary Import databases that are being accessed.
  • Local User: Users who are assigned this role cannot use distributed navigation.
Administrator accounts

Administrators must be members of the securityadmin or sysadmin groups to grant permissions to domain users.

To run the BAM Management utility, you must be at least a database operator for the BAM databases.

Final Notes

I would like to thank Nino Crudele for the help that he gave me to solve this problem and to my coworker and friend José Barbosa who helped me creating these SQL scripts and as result solving the problem.

And a final note for all system administrator… please don’t delete Active Directory accounts… instead disable them!!


BizTalk BAM Portal URL authorization failed for the request (Event code 4007)

$
0
0

Last day I was able to convince my client to use, for the first time, BAM for tracking and monitoring of specific processes. As I anticipated, 1-hour work resulted in a major impact (for better) on the people responsible for these tasks in the organization.

However after I deploy my BAM Definition and tracking profile in production environment and everything was working well, i.e., processes were running successfully and data was being tracking (I was able to see the tracking data in database)…

If you don’t know, and contrary to what I thought also, the only user that always has access to the view and cannot be added to or removed from the view(s) is the Database Owner (BAMPrimaryImport). So the user that made the deploy of this BAM Definition don’t necessarily have access to this particular view in the BAM Portal!

In this particular case I’m also the Database Owner, but when I try to access to BAM Portal the access was constantly being denied me and always asking to enter my credentials when browsing to it… even when I gave access to another domain user to this view and tested with these credentials the problem remained.

After examining the logs in event I found this information message:

BAM-Web-Event-Information

With the following details:

Event code: 4007
Event message: URL authorization failed for the request.
Event time: 23-04-2013 15:55:52
Event time (UTC): 23-04-2013 14:55:52
Event ID: 053c6e752b6a4de8ae400a9a9d7d26b1
Event sequence: 10
Event occurrence: 9
Event detail code: 0

Application information:
Application domain: /LM/W3SVC/1/ROOT/BAM-1-130112015742350508
Trust level: BAMPortal_Minimal
Application Virtual Path: /BAM
Application Path: D:\Program Files (x86)\Microsoft BizTalk Server 2010\BAMPortal\
Machine name: MyMachine

Process information:
Process ID: 9560
Process name: w3wp.exe
Account name: DOMAIN\bts-bam-ap

Request information:
Request URL: http://localhost/BAM
Request path: /BAM
User host address: 192.168.***.***
User: DOMAIN\MYUSER
Is authenticated: True
Authentication Type: Negotiate
Thread account name: DOMAIN\MYUSER

Custom event details:

CAUSE

Well, unfortunately, this problem or similar problems can happen for many reasons:

Fortunately for me I have an E2E test environment which is an almost exact replica of PROD with which I could compare to see what was the problem and that was working well Sorriso.

One thing I was sure: I had permission problems!

The first thing was to analyze the basic settings of the application pool like: credentials or .net version and so on… however everything was properly configured and equal to the test environment.

After a few minutes I remembered the basics… if you remember the BizTalk Configuration experience, you use the BizTalk Server configuration tool to specify whether BAM is enabled, and to specify the Web service accounts, the Windows groups that can view portal, and the Web site that will host the portal.

BAM-Portal-Configuration

That you also can see in “.Net Authorization Rules” under BAM website:

BAM-IIS-Net-Authorization-Rules

Using the principle of least privilege, user accounts should have restrictive permissions to perform routine tasks in the BAM portal. BizTalk BAM Portal Users is the group, at least for me but this may change according to your configurations, where you defined the users or groups that can access to BAM Portal Web site.

SOLUTION

In my case, after checking in AD, there was no one configured to have access to BAM Portal.

To solve this problem you have to configure the users or groups that you want to have access to BAM Portal under the “BizTalk BAM Portal Users” in your Active Directory.

After this operation everything work fine! Exactly as it should.


BizTalk Server: Cannot see my BAM View in BAM Portal after I deploy it

$
0
0

It´s a common factor that before users can view data from a BAM model in the BAM Portal website, they must be granted access to the views. However, and contrary to what I thought, the user that makes the deploy of the BAM Definition don’t necessarily have access to this particular view in the BAM Portal! (I was being misled because usually I am the owner… hehe)

BAM-Portal-Missing-View

Instead the only user that always has access to the view and cannot be added to or removed from the view(s) is the Database Owner (BAMPrimaryImport)!

You can check who the database owner is by running the following query:

SELECT SUSER_SNAME(owner_sid),name
FROM sys.DATABASES

So after I run this query I realize that another user was the owner and to solve this “problem” I had to give permissions to my user to access this View in BAM Portal by running the following BAM Management Utility command:

bm.exe add-account -AccountName:DOMAIN\USER -View:MyView

BAM-Portal-With-View

Mystery solved Sorriso


Viewing all 35 articles
Browse latest View live