Supervise the size of your SQL Databases in Azure

Make sure your SQL databases in the cloud stay performant and within the sizing thresholds your business requires. By monitoring the size, you gain awareness and can take actions to keep only the required amounts of data. This will save you money.

With the SQL monitor agent in Integration Manager it is easy to keep track of your SQL Databases in Azure. Today the there are two categories available for monitoring the SQL Databases.

  • Size check
  • Custom statements

This blog post describes how to monitor size checks using Integration Manager.

SQL Monitor Agent

With Integration Manager comes an agent for monitoring SQL Server instances and databases on premise and SQL Databases in the cloud. To read more about the agent and see all supported monitoring capabilities, please visit the Documentation site.

The Agent can be installed in multiple instances on premise or in the cloud/partner location on a virtual machine.

SQL Monitor Agent

 

Global Configuration

Using Remote Configuration the agent can be configured to monitor one or more databases

Size Check Category

Configure

Configure the thresholds for the size check category to get warnings and errors if the limits are reached. An error will also be raised if the resource is unavailable (network error, someone accidently removes the database, changes permissions and so on).

There are two options available for the size check evalution

  1. Percentage
  2. Size in Megabytes

Remote Configuration SQL Server

Read more about configuration in the Integration Manager Documentation, Azure – SQL Size Checks.

Monitor

When the configuration is done it is easy to add the resource to one or more Monitor Views. Within the monitor view you can configure whether an alert should be raised or not when an error or warning occurs. More about Monitor Views in Integration Manager can be found here.

Azure SQL Monitor

This screen shot shows a list of my Monitor Views.

The Azure SQL Monitor view, which supervise the size of the databases are currently OK in the example above. The other view, ‘Azure LogicApps Monitor’ indicates a problem with my Logic Apps (more about monitoring Logic Apps in a future blog post).

Detailed configuration

A specific resource can easily be reconfigured and information about limits are also showed. In the example below one can see that there was indeed a size limit on just 100 MB and percentage was the better monitoring configuration for this specific database.

Azure Database Size Check Configuration

Summary

With Integration Manager it is easy to monitor all your assets independent of where and what they are. This was a short blog post about how to monitor your Azure SQL Databases without logging in to the Azure portal and manually keep track of the size.  A database cannot grow beyond limits and your business critical applications will fail if there’s not enough free space to deal with your transaction needs. Stay proactive using Integration Manager

Videos on  YouTube
Information on  Twitter
Contact on  LinkedIn

Online documentation: Latest version
Latest release notes: Release Notes
Support site: Support Site

Azure App Services Series – Finding #8: No filter in API Apps pane of Logic Apps actions and trigger blade

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

Microsoft has in a recent update released the possibility to provision API apps directly from the actions and triggers blade to your API app service (see Screenshot 1 below). This is an important update, since you no longer need to close the logic apps blade, add all the API apps and then go back to the Logic App. To read more about the update click here.

Screenshot 1 - Create Logic App with list of available API Apps in Marketplace.

Screenshot 1 – Create Logic App with list of available API Apps in Marketplace.

The Problem

As you can see in Screenshot 1, the list of available API apps is already long and will grow and grow. In the today’s version of Logic Apps there is no possibility to search, filter and group all available API apps, making it a hard task to find the API app you need.

The Solution

What I / we would like to see is a text field / select boxes enabling us to filter the API apps by values such as

  • Type (API Apps are separated into groups, such as protocol API apps, integration API apps, etc.) – as selectbox (multi selectbox to support filtering on more than one type)
  • Name – as a text field.
  • Company/Namespace – as a text field or multi-selectbox to simply search for API apps published by the different providers.
  • etc.

This would make provision the required API apps within your Logic Apps easier.

Microsoft Integration Newsletter

Microsoft has an integration newsletter we would like you to inform you about.

The newsletter contains useful information about upcoming/ongoing events as well as past events – making it easy for you to know what is happening in the Microsoft integration world.

To subscribe to the newsletter, please here.

 

Picture taken from Microsoft BizTalk Integration Newsletter – Issue 8.

Azure App Services Series – Finding #7: Startpage widget for Logic Apps in preview portal not showing status

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

The new Azure portal is great when it comes to flexibility and customization support for the user interface or at least when it comes to the start page. You can add shortcuts to virtual machines, websites, etc. to your dashboard, as well as billing information, etc.  And this makes it interesting from a monitoring point of view, since you can add a shortcut to your virtual machine and the dashboard will then provide you with a shortcut link, as well as shows information about the state. This means that if the virtual machine is stopped – the dashboard/home page will tell you as shown below in Screenshot 1.

Screenshot 1 - Virtual Machine widget showing status.

Screenshot 1 – Virtual Machine widget showing status.

The Problem

I wanted to add my Logic Apps to my dashboard to simply provide myself with useful shortcuts as well as I thought that the widget on the dashboard would then show me the state of the application itself – but no (see Screenshot 2 below). Unfortunately the new widgets provided for the logic apps do not show the state yet.

Screenshot 2 - Status of logic app not showing on home.

Screenshot 2 – Status of logic app not showing on home.

Integration Manager – a tool for centralizing logging and monitoring of data and systems in your system integration provides you that shows all logic apps, not only the one in your Azure directory- but all directories you configure it to log from and to monitor.

The Solution (in my/our opinion)

Show the state of the logic apps (running/stopped), as well as the state of all API apps used in the logic apps, since they are important as well for the integrations you’ve build to work.

Azure App Services Series – Finding #6: No possibility to comment triggers and actions in Logic Apps

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

Working with Logic Apps is fun, but in the current version of it is kind of hard to keep track what all the different actions do. Let’s say you use the BizTalk XPath Extractor several times, because you need to extract several values from an XML file content. But take a look at Screenshot 1 below and see what it looks like.

xpath-extractor

Screenshot 1 – Several XPath Extractors

With the announcement that Integration Manager now fully supports logging in Logic Apps it becomes very easy to follow the chain of occurred events in your logic apps, enabling you to see the input and output of the different actions which helps you understanding the different logic apps better – read the announcement by clicking here.

The Problem

As you can see in Screenshot 1 above it is pretty hard to understand what the different extractor actions do, isn’t it? It will be a demanding task to learn and understand logic apps by just looking at them in they designer view if they will look like shown in Screenshot 1, since the information provided is not enough – making it hard to understand.

The Solution

What we would like to see is a way of commenting each action in the workflow for better understanding each action being executed. An example would be a comment field  enabling developers to explain what the action does.

Screenshot 2 - Comment Field

Screenshot 2 – Comment Field

Azure App Services Series – Finding #5: Changing the order of connectors in your Logic APP not possible in designer view

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

Developing Logic Apps requires you to put a chain of API apps in place. This chain can of course be changed.
The reasons why this chain could be changed could be

  • An API app has changed.
  • A connector is not working anymore
  • The connector is not required anymore
  • The flow has changed and therefore maybe the order of API apps.
  • etc.

Some of the points in the list above such as if a connector is still working or not can be monitored by Integration Manager.

The Problem

The designer the current state of Logic Apps is offering does not offer drag and drop and therefore changing the order of API apps or inserting a new API app in between is not possible.

The Solution

Luckily for us the designer view is not the only view available the Logic App designer offers – we also have the code view. You can achieve changing the order of API apps / connectors in your Logic App by changing the JSON workflow definition.

To learn more about the Workflow definition click here.

What we would like is of course the possibility to drag and drop the different connectors and put them in the order we want instead of needing to change the JSON workflow definition.

Azure App Services Series – Finding #4: Hybrid Connections – Could not load file or assembly Newtonsoft.Json

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

Azure BizTalk Services – Hybrid Connections allow you to access on-premise resources, such as SQL instances, files, etc from Azure app services – web, API, mobile and logic apps.

By enabling your app services to use on-premise resources you can together with Integration Manager monitor even more!

Microsoft created and released e.g. a protocol connector enabling you to access on-premise files – the File connector. This connector uses Hybrid Connections to access files located on your on-premise file system.

The Problem

If you follow tutorials / documentations on how to get started with Hybrid Connections you learn that you have to have a BizTalk Services instance in Azure, this instance can have one or several hybrid connections (see pricing of BizTalk Services to learn more about limitations). While setting up and configuring your hybrid connection, it requires you to install a service/agent on your on-premise machine called the Hybrid Connection Manager. Installing the agent sometimes works and sometimes not.

We came across a file not found error during installation. Newtonsoft.Json file could not be found and therefore the installer throws an exception stating “Could not load file or assembly Newtonsoft.json…”.

Unfortunately there is no post from Microsoft about it, there are some forum posts regarding this error message, but we could not find a proper blog post about how to resolve the problem.

The Solution

We solved the issue by installing the proper version of the Newtonsoft.Json.dll into the Global Assembly Cache (GAC). The error message of the installer tells you which version you need, which in our case was version 4.5.x.x. We downloaded the dll with version 4.5.11 from the Newtonsoft.json Github project downloads and installed it into the GAC. Installing the DLL into the GAC can be done by using the Developer Command Prompt for Visual Studio and executing the gacutil.exe with the proper parameters.

The command prompt will show a message that the assembly has successfully been added to the GAC (see screenshot 1).

Screenshot 1 - Gacutil added assembly

Screenshot 1 – Gacutil added assembly

When you are done, restart the installer of the Hybrid Connection Manager and you should be good to go (if there is not another bug hidden somewhere along the installation!).

 

We hope this helps you solving issues during the installation of the Hybrid Connection Manager!

Azure App Services Series – Finding #3: BizTalk transformer API service still requires Visual Studio 2012

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

For systems to communicate with each other data needs to be transformed.

Microsoft has released an API app / Connector that can be used in your Logic Apps enabling you to transform a message from one document type to another by using the BizTalk Services map (see Screenshot 1) – BizTalk Transform Service (see action in Screenshot 2)

Screenshot 1 - BizTalk Services Map in Visual Studio

Screenshot 1 – BizTalk Services Map in Visual Studio

Screenshot 2 - BizTalk Transform Service Action

Screenshot 2 – BizTalk Transform Service Action

With Integration Manager we can see how the message has been transformed without needing to export the input/output files of your operation logs of your logic app (see Screenshot 3 to 5 below).

Screenshot 3 - Logged events of tranformation in Integration Manager

Screenshot 3 – Logged events of tranformation in Integration Manager

 

Screenshot 4 - Message before transformation

Screenshot 4 – Message before transformation

 

Screenshot 5 - Message after transformation

Screenshot 5 – Message after transformation

The Problem

The maps used by the BizTalk Transform Service in your Logic App need to be uploaded to your instance of the BizTalk Transform Service (see screenshot 6).

Screenshot 3 - Uploading BizTalk Services map

Screenshot 6 – Uploading BizTalk Services map

But how do we create them? In Visual Studio, but what version? The answer is 2012 – you can create a BizTalk Services map in Visual Studio 2012, not in 2013, 2015 Preview, just in 2012! In our opinion it is a miss once again here. Microsoft engages us to use the latest techniques/technologies, the latest tools, but when it comes to a certain level of details, Microsoft is just not up-to-date.

What we want

It is obvious – we want a better integration when it comes to Visual Studio 2013 and of course with that Visual Studio 2015 as well.

Key-Takeaways

  1. Creating a BizTalk Services map to use in your BizTalk Transform Service of your Logic App requires you to use Visual Studio 2012 and the BizTalk Services SDK.
  2. Integration Manager gives us end-to-end tracking in the logic apps (not only Logic Apps) for understanding all steps in your integrations

Azure App Services Series – Finding #2: Logic App Workflow API documentation is incomplete

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

With the release of Logic Apps, Microsoft also releases the Workflow Management API. An API to use for getting access to the workflow of your logic apps allowing you to easily enable, disable workflows, get the status, etc.

Microsoft’s documentation of the API is incomplete and not clear which of the API calls are working in the current release of the API.

This API is allowing you to monitor Logic Apps with Integration Manager’s monitoring feature!

The Problem

Problem #1: Incomplete documentation

According to the documentation the URI to the calls is:

https:///subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Logic/workflows/{workflowName}?api-version=2015-02-01-preview

This is wrong of course, you can clearly see that something is missing in the beginning of the URI.

It’s the URI to the region of the Logic App. For example: northeurope.logic.azure.com. The URI to call the API should look like this (example below):

https://northeurope.logic.azure.com/subscriptions/72bad97c-b0ce-40a3-a300-d36e50de34fe/resourceGroups/Default-Web-NorthEurope/providers/Microsoft.Logic/workflows/FTP2BTnoIM?api-version=2015-02-01-preview
Problem #2: Not all of the API calls documented are working (yet)

We did not manage to get many of the API calls to work. In fact, we only did manage to get one call to work – triggering the action by calling the /run method of the API. Most of the others (we did not test all) return 404 not found when calling the API.

Problem #3: There is no documentation about authenticating against the API

The documentation says nothing about how the authentication looks like, which makes the documentation somewhat useless!

For your interest, the authentication type supported is Basic. A username is created on logic app create which is ‘default‘ and the password to use is the primary key (not sure if the secondary works as well) which you can find in the logic apps properties blade of the Azure portal (see screenshot 1).

Screenshot 1 - Primary Access Key

Screenshot 1 – Primary Access Key

So the proper way of authenticating against the API is to send the Authorization header with your Basic username/password value – see example below:

Authorization: Basic YOURENCRYPTED(default:primaryAccessKey)

Azure App Services Series – Finding #1: Problems using Workflow Definition Language in Logic Apps

This blog post is one of many of a series we have about Microsoft Azure App Services. See a list of all our findings here.

Introduction

With the preview-release of Logic Apps, Microsoft also releases a new language used in the work flows of your Logic Apps – Workflow Defintion Language (WDL).

The language is used in your work flow code in both conditions and action inputs.

Using it in coditions

Conditions are used to perform the action on which you add the condition when the condition matches the requirements. This is especially important when it comes to exception handling.

Per default if one of the action fails in an Azure Logic App, the message is thrown away when one of the actions fail, so what you need to do is to create actions that are performed when error occur in order to not loose data. In most cases the data itself is still accessible in the operation logs (this depends on the API apps!) and can be extracted from there with the Workflow Management API (the Workflow Management API will be covered in an upcoming blog post).

An example:
The above example is a condition for an action that is executed when the status of the action “YOURACTION” equals ‘Failed’.

Using it in an action’s input

WDL is not only used in conditions, but also when sending input data to the defined actions. For instance, you might have an FTP connector that is triggered as soon as a file is available on the FTP, what you want is to send the file’s content or file name to another action (logging purposes maybe). In this case the FTP connector would be the trigger and another API App the action. An example of getting the FTP triggers file name:

The above example is a WDL expression to get the triggers output body, which you can send into an action as input.

The Problems

Problem #1

There is basically no Intellisense/help at all when it comes to writing expressions with WDL . The only Intellisense/help you get is when using a trigger’s/action’s output in another action and the only thing you get there are the available output alternatives (output body of succeeded actions! Let me get back to that later in Problem#2 below!) shown in screenshot 1 below. In many cases this is enough, but what happens if you want to use functions to concatenate strings or you want to write conditions? Well, you are on your own body!

available_trigger_outputs

Working with concat or any other function provided by the WDL is not giving you any alternative in the input fields – no help, no nothing except the output of the triggers or actions that have run before the action you are editing, making it very hard to work with WDL!

Problem #2

As described above and shown in the Screenshot 1, we get the triggers’ and actions’ outputs as alternatives in the actions inputs (NOT IN THE CONDITIONS!!!). But the available outputs are the outputs for succeeded actions! But if you have an action that fails based on a condition and you want to send the error message to third-party tools like Integration Manager using Integration Manager’s connector (Coming soon), what is the WDL expression for that? Well and how can we test the expressions (see Problem#3)? The documentation shows us that there is the errors attribute containing an array of errors, but there is no help in the interface on what the WDL expression is for that.

Problem #3

The third problem we see in the current version of Logic Apps are the not yet existing way of testing your WDL expressions, there is some kind of validation on saving your workflow, but no direct way of testing it.

Our wishes

Problem #1: Full intellisense in both action input text fields and condition text fields.

Problem #2: All available outputs for both when an action failes and succeeds. (when building a condition you might want to choose “action(‘ftpconnector’).status”)

Problem #3: Testing of WDL expressions and improved validation.

 

Vote for our feedback

http://feedback.azure.com/forums/287593-logic-apps/suggestions/7592121-intellisense-help-on-wdl-expressions

Twitter
YouTube
LinkedIn