Azure App Services Series – Finding #6: No possibility to comment triggers and actions in Logic Apps

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

Working with Logic Apps is fun, but in the current version of it is kind of hard to keep track what all the different actions do. Let’s say you use the BizTalk XPath Extractor several times, because you need to extract several values from an XML file content. But take a look at Screenshot 1 below and see what it looks like.

xpath-extractor

Screenshot 1 – Several XPath Extractors

With the announcement that Integration Manager now fully supports logging in Logic Apps it becomes very easy to follow the chain of occurred events in your logic apps, enabling you to see the input and output of the different actions which helps you understanding the different logic apps better – read the announcement by clicking here.

The Problem

As you can see in Screenshot 1 above it is pretty hard to understand what the different extractor actions do, isn’t it? It will be a demanding task to learn and understand logic apps by just looking at them in they designer view if they will look like shown in Screenshot 1, since the information provided is not enough – making it hard to understand.

The Solution

What we would like to see is a way of commenting each action in the workflow for better understanding each action being executed. An example would be a comment field  enabling developers to explain what the action does.

Screenshot 2 - Comment Field

Screenshot 2 – Comment Field

Azure App Services Series – Finding #3: BizTalk transformer API service still requires Visual Studio 2012

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

For systems to communicate with each other data needs to be transformed.

Microsoft has released an API app / Connector that can be used in your Logic Apps enabling you to transform a message from one document type to another by using the BizTalk Services map (see Screenshot 1) – BizTalk Transform Service (see action in Screenshot 2)

Screenshot 1 - BizTalk Services Map in Visual Studio

Screenshot 1 – BizTalk Services Map in Visual Studio

Screenshot 2 - BizTalk Transform Service Action

Screenshot 2 – BizTalk Transform Service Action

With Integration Manager we can see how the message has been transformed without needing to export the input/output files of your operation logs of your logic app (see Screenshot 3 to 5 below).

Screenshot 3 - Logged events of tranformation in Integration Manager

Screenshot 3 – Logged events of tranformation in Integration Manager

 

Screenshot 4 - Message before transformation

Screenshot 4 – Message before transformation

 

Screenshot 5 - Message after transformation

Screenshot 5 – Message after transformation

The Problem

The maps used by the BizTalk Transform Service in your Logic App need to be uploaded to your instance of the BizTalk Transform Service (see screenshot 6).

Screenshot 3 - Uploading BizTalk Services map

Screenshot 6 – Uploading BizTalk Services map

But how do we create them? In Visual Studio, but what version? The answer is 2012 – you can create a BizTalk Services map in Visual Studio 2012, not in 2013, 2015 Preview, just in 2012! In our opinion it is a miss once again here. Microsoft engages us to use the latest techniques/technologies, the latest tools, but when it comes to a certain level of details, Microsoft is just not up-to-date.

What we want

It is obvious – we want a better integration when it comes to Visual Studio 2013 and of course with that Visual Studio 2015 as well.

Key-Takeaways

  1. Creating a BizTalk Services map to use in your BizTalk Transform Service of your Logic App requires you to use Visual Studio 2012 and the BizTalk Services SDK.
  2. Integration Manager gives us end-to-end tracking in the logic apps (not only Logic Apps) for understanding all steps in your integrations

Azure App Services Series – Finding #1: Problems using Workflow Definition Language in Logic Apps

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

With the preview-release of Logic Apps, Microsoft also releases a new language used in the work flows of your Logic Apps – Workflow Defintion Language (WDL).

The language is used in your work flow code in both conditions and action inputs.

Using it in coditions

Conditions are used to perform the action on which you add the condition when the condition matches the requirements. This is especially important when it comes to exception handling.

Per default if one of the action fails in an Azure Logic App, the message is thrown away when one of the actions fail, so what you need to do is to create actions that are performed when error occur in order to not loose data. In most cases the data itself is still accessible in the operation logs (this depends on the API apps!) and can be extracted from there with the Workflow Management API (the Workflow Management API will be covered in an upcoming blog post).

An example:
The above example is a condition for an action that is executed when the status of the action “YOURACTION” equals ‘Failed’.

Using it in an action’s input

WDL is not only used in conditions, but also when sending input data to the defined actions. For instance, you might have an FTP connector that is triggered as soon as a file is available on the FTP, what you want is to send the file’s content or file name to another action (logging purposes maybe). In this case the FTP connector would be the trigger and another API App the action. An example of getting the FTP triggers file name:

The above example is a WDL expression to get the triggers output body, which you can send into an action as input.

The Problems

Problem #1

There is basically no Intellisense/help at all when it comes to writing expressions with WDL . The only Intellisense/help you get is when using a trigger’s/action’s output in another action and the only thing you get there are the available output alternatives (output body of succeeded actions! Let me get back to that later in Problem#2 below!) shown in screenshot 1 below. In many cases this is enough, but what happens if you want to use functions to concatenate strings or you want to write conditions? Well, you are on your own body!

available_trigger_outputs

Working with concat or any other function provided by the WDL is not giving you any alternative in the input fields – no help, no nothing except the output of the triggers or actions that have run before the action you are editing, making it very hard to work with WDL!

Problem #2

As described above and shown in the Screenshot 1, we get the triggers’ and actions’ outputs as alternatives in the actions inputs (NOT IN THE CONDITIONS!!!). But the available outputs are the outputs for succeeded actions! But if you have an action that fails based on a condition and you want to send the error message to third-party tools like Integration Manager using Integration Manager’s connector (Coming soon), what is the WDL expression for that? Well and how can we test the expressions (see Problem#3)? The documentation shows us that there is the errors attribute containing an array of errors, but there is no help in the interface on what the WDL expression is for that.

Problem #3

The third problem we see in the current version of Logic Apps are the not yet existing way of testing your WDL expressions, there is some kind of validation on saving your workflow, but no direct way of testing it.

Our wishes

Problem #1: Full intellisense in both action input text fields and condition text fields.

Problem #2: All available outputs for both when an action failes and succeeds. (when building a condition you might want to choose “action(‘ftpconnector’).status”)

Problem #3: Testing of WDL expressions and improved validation.

 

Vote for our feedback

http://feedback.azure.com/forums/287593-logic-apps/suggestions/7592121-intellisense-help-on-wdl-expressions

Log events and messages from BizTalk to Integration Manager

Get insights and Control!

This post describes the steps necessary to log messages from BizTalk to Integration Manager. Integration Manager has great, out-of-the-box support for Microsoft BizTalk Server. Integration Manager replaces the need for costly and bad performing alternatives like BAM, SCOM and many other 3rd party products.

The need to log events and messages flowing through the systems integration platform is ever growing and growing. This is due to systems integrations becoming more and more important for the business. Providing the business people with the data they need in order to innovate their business is crucial.

Microsoft BizTalk Server is one integration platform for enterprise application integration (EAI) and business-to-business (B2B) integration.

 

 

This blog post assumes you have basic knowledge in administering Microsoft BizTalk Server and Integration Manager, as well as a working instance of BizTalk and Integration Manager.

 


Prepare the BizTalk Server Environment

Simply enable tracking on receive and send ports within your applications in order for Integration Manager to log all the events/messages going through your Microsoft BizTalk Server.

Screenshot 1 - Enable Tracking on Microsoft BizTalk Server

Screenshot 1 – Enable Tracking on a Microsoft BizTalk Server receive port

Integration Manager can also be used from within Orchestrations and pipeline stages. Thiscapability uses our LogAPI which we will cover in another blog post.

The question we often get asked is how we affect existing BizTalk applications. The answer is – we don’t.

  • Integration Manager only requires BizTalk’s tracking to be enabled on ports of interest.
  • No pipelines / pipeline Components required.
  • No BAM activities
    Integration Manager offers you a way to retrieve any kind (XML, EDI, CSV) of logged message content by using either the context values or the messages’ body (This will be covered in another blog post).
  • No solution/project requirements
    • No analysis
    • No design
    • No code
  • No MSMQ queues

Prepare Integration Manager

Integration Manager’s logging service has a thread that copies data from Microsoft BizTalk Server’s tracking database (usually BizTalkDTADb). For Integration Manager to get the data from the database we need to enable the log feature in Integration Manager and provide it with the correct connection information to Microsoft BizTalk Server’s management and tracking database.

You can do this by opening up the configuration application (see screenshot 2 below), open the system settings (see screenshot 3) and check the checkbox saying “Copy messages from BizTalk (BizTalk Tracking Enabled)” and fill in the required connection information for both the management database (BizTalkMgmtDb) and tracking database (BizTalkDTADb) (see screenshot4).

Integration Manager - Configuration Application

Screenshot 2 – Starting Integration Manager’s configuration application

Integration Manager - Configuration Application - system settings

(Screenshot 3 – Open up system settings)

Integration Manager - Configuration Application - biztalk settings

Screenshot 4 – Setting up Microsoft BizTalk Server settings

Finish by pressing the Save button and close the Integration Manager configuration application.

Start the ‘IM Logging’ Windows Service.

Integration Manager will now first sync end Points (Send Ports, Receive ports/locations, etc.) in BizTalk with Integration Manager and then to copy all the data from your tracking database to Integration Manager’s logging databases. This is a continous process meaning that you can have a short windows (3-4 days worth of data within the biztalk tracking database)

Use the admin-view in Integration Manager or create custom log views to search for any event/message logged via Microsoft BizTalk server. You get access to not only an event that indicates that something has been logged, you get also access to the message itself and the context values.

event context BT compared to IM

Screenshot 4 – showing message details in BizTalk Management Console and Integration Manager

Key Takeaways

Using Integration Manager to log & archive events/messages from Microsoft BizTalk Server is easy and gives you the control and insights you need in order to run your business – simply follow the simple steps explained above and you are good to go.

By using Integration Manager and its job for copying events/messages from BizTalk’s tracking database you can keep the “Purge & archive” SQL job to save data for just 2 to 3 days, since the data is copied to the log databases of Integration Manager – keeping the performance affection low.

No BAM activities – Integration Manager as already explained above, does not need any BAM activities. Integration Manager has ways of getting the values you are interested in by using the messages’ body or context values. The values can be retrieved from historic events/messages as well! (Will be covered in an upcoming blog post).

Integration Manager has long time storage – Every transaction can be kept forever as long as you have the disk space. We will create new databases as we go to keep the system performant independent of the total amount of data being kept. There might be legal demands to keep for example outgoing invoices for 7 years – we can guarantee that for you.

To read more about Integration Manager logging click here.

How to use Integration Manager’s log4net log appender

This blog post describes one of the ways/functions we offer to log events/message to Integration Manager by using Integration Manager’s log appender for Log4Net. We assume that you have basic knowledge in Log4Net as well as you have a running instance of Integration Manager. Before continuing reading, make sure you have the correct link to Integration Manager’s Log API. In my exampel the test application and Integration Manager are located on the same server and the address to LogAPI is “http://localhost/IM/LogApi/LogApiService.svc“.

We will work with an example solution – a simple console application that prompts the user to enter a message which will log the message to a file with Log4Net’s file appender. During this how-to we will show you how you can extend your application with Integration Manager’s Log4Net appender. You can download the example here.

Before we start we let us take a look at how the application works.

  1. Start the application
    Console Application - Image 1
  2. Write a message to log – Hello World
    Console Application - Image 2
  3. The application will write into a file called tutorial.log in the application’s folder.
    Console Application - Image 3
  4. The result
    Console Application - Image 4

Let’s start adding Integration Manager’s log appender for Log4Net. We start by downloading the appender here.

The file you downloaded contains readme and DLL files. Start by copying the DLL files from the downloaded ZIP into the applications folder and open your applications configuration file which in the example looks like the code below.

Since we log a message to Integration Manager’s Log API we need to add the required “ServiceModel” configuration – see below.

Copy the above code, replace client/endpoint/@address value “http://localhost/IM/LogAPI/LogApiService.svc” with the address of your Integration Manager’s Log API instance and paste it into the application’s configuration file.

The next step is to add an application settings key called “IM.Log4Net.IsOneWay” which is of type boolean – set it depending on the requirements to either true or false.

If the application setting “IM.Log4Net.IsOneWay” is set to true the application will not ensure deliverance to the Log API, since we have configured the (IM) appender for Log4Net to not wait for a response (one way).

For log4net to understand that it needs to log an event/message with the provided appender you need to extend the configuration file with a reference to the appender as well as an appender configuration (IMAppender) – see below.

All the parameters used in the (IMAppender) appender configuration file are described in the readme.pdf that comes with the appender you downloaded before.

When the configuration is completed you can then start the application, write a text message and press Enter. The application will then log the event/message via the IM appender to Integration Manager – the configured Log Agent, End Point and Message Type will automatically added to Integration Manager. You can then create a log views allowing users to only search for events/messages coming from applications that use this (IM) appender.

You can see an example search result of Integration Manager in the screenshot below.

Console Application - Image 6

 

You can download the final application here. We hope that this blog post gets you started with logging your Log4Net logs to Integration Manager, giving you yet another way to log messages to Integration Manager for a centralized platform for all your logs, no matter what integration platform, application nor technology you use.

Twitter
YouTube
LinkedIn