Update – Auto Packaging using CSPack and Azure SDK 1.6

This post is related to two of my previous posts:

Azure 1.5 ate my diagnostics

I had diagnotics working quite happily until SDK 1.5 came out.  Then all of a sudden data was no longer being transferred to  Azure storage.  Even more mysteriously diagnostics would happily transfer data to Azure storage when being emulated locally, but not when on the Azure cloud (in other words a nightmare problem)

I didn’t get around to investigating why till this week.  I saw that several people had the same problem, and assumed that the problem was that I wasn’t configuring the diagnostics correctly in the OnStart method.

Finally I saw this forum thread.  The thead described that if you upload your solution from visual studio diagnostic works correctly, but not when deployed from the build process.  I tried for myself, and yep diagnostics would magically work when the solution was deployed from Visual Studio.  This finally clued me into the fact that the problem had nothing to do with the code, but everything to do with packaging.  Which leads us to this update on Auto Packaging your Azure solution.

Configuring Your Azure Continuous Integration process with CSPack and SDK 1.6

My previous post on using CSPack to automatically build your deployment packages is largely still correct.  But as of (I assume SDK 1.5) there’s a new EntryPoint property.

So you need to specify the name of the DLL that is the entrypoint to your solution.  In mycase HuzuSocial.App.dll.  So my AzureProperties.txt file now looks like this:


Now configured correctly, Diagnostics works as expected from our Continuous Integration process.


Windows Azure Diagnostics with SDK 1.6 for WebRoles

There appears to be a lot of conflicting and confused advice about configuring Diagnostics on Windows Azure.  The situation is not at all helped by Microsoft’s own site which, to paraphrase Morecambe and Wise, has all the right pieces of information, just not necessarily in the right order.

It doesn’t help that what used to work with earlier versions of the Azure SDK, no longer works with later versions.  So here I outline:

  • The steps to get Diagnostics outputting correctly to Windows Azure Storage with SDK 1.6 for WebRoles (although I’d imagine it’s largely the same for WorkerRoles)
  • Azure 1.5 ate my diagnostics – Another post where I update my Auto Packaging post to be compatible with SDK 1.6

Setting up Windows Azure Diagnostics for your WebRole with SDK 1.6

1. Configure Web.Config – required if you are using Trace statements

I use Log4Net for my general logging/tracing needs so don’t use Trace statements, thus the example shown in step 3, below, does not require you to complete this step.

However, if you are using Trace statements,  ie:

System.Diagnostics.Trace.TraceError("Error has occurred");

You’ll need to configure Web.config as described here

            <add type="Microsoft.WindowsAzure.Diagnostics.DiagnosticMonitorTraceListener,
                <filter type="" />

2. Initialise Diagnostics

As outlined here, you’ll need to ensure you add the Import element for the Diagnostics module in your ServiceDefinition.csdef file.  Here’s what mine looks like:

<?xml version="1.0" encoding="utf-8"?>
<ServiceDefinition name="HuzuSocial.Azure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition">
    <WebRole name="HuzuSocial.App" vmsize="Small" >
            <Site name="Web">
                    <Binding name="Endpoint1" endpointName="Endpoint1" />
            <InputEndpoint name="Endpoint1" protocol="http" port="80" />
            <Import moduleName="Diagnostics" />

Secondly you’ll need to add your Azure Storage Account details into your ServiceConfiguration.cscfg, mine looks like this (obviously replace with your account name and key):

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="HuzuSocial.Azure" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*">
        <Role name="HuzuSocial.App">
        <Instances count="2" />
            <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=[youracountnamehere];AccountKey=[youraccountkeyhere]/>

3. Override the OnStart method in WebRole.cs

In the root of your web project you should have a WebRole class.  You’ll need to override the OnStart method to correctly initialise the Diagnostics.  There is loads of different sample code out there, some of it highly dubious.  This is my configuration, and works well for me (I lifted this from a post out there somewhere, unfortunately I forgot to bookmark it and can no longer find it, so thankyou whoever you are)

public override bool OnStart()
    DiagnosticMonitorConfiguration diagConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();

    var perfCounters = new List<string>
        @"\Processor(_Total)\% Processor Time",
        @"\Memory\Available Mbytes",
        @"\TCPv4\Connections Established",
        @"\ASP.NET Applications(__Total__)\Requests/Sec",
        @"\Network Interface(*)\Bytes Received/sec",
        @"\Network Interface(*)\Bytes Sent/sec"

    // Add perf counters to configuration
    foreach (var counter in perfCounters)
        var counterConfig = new PerformanceCounterConfiguration
                                CounterSpecifier = counter,
                                SampleRate = TimeSpan.FromSeconds(5)


    diagConfig.PerformanceCounters.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);

    //Windows Event Logs
    diagConfig.WindowsEventLog.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);
    diagConfig.WindowsEventLog.ScheduledTransferLogLevelFilter = LogLevel.Warning;

    //Azure Trace Logs
    diagConfig.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);
    diagConfig.Logs.ScheduledTransferLogLevelFilter = LogLevel.Warning;

    //Crash Dumps

    //IIS Logs
    diagConfig.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(1.0);

    DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagConfig);

    return base.OnStart();

4. That’s it

When deployed to Azure your diagnostics should be successfully transferred to Azure Storage.  To analyse them in any meaningful way, I’d recommend Cerebrate Diagnostics manager, which gives you a nice dashboard.  See below

CRUD – Destroyer of Worlds

As web developers a large part of what we do is CRUD.  The average Service class in my project is at least 50% CRUD methods, and often a lot more.

If you’re anything like me, you’ve probably not given much thought to CRUD methods, you just write them as required.  However, I recently had a small epiphany, this approach to CRUD was making the code base sprawl, approaches to CRUD from the team were non-standard.  Which was leading to new CRUD methods being created, when not required, because of the difficulty of understanding what was there.

I wouldn’t start from here…

So how did it come to this?  I use the Repository pattern for my DB layer and for whatever reason a lot of the documentation around this pattern have method signatures like this:

Customer GetCustomerById(int customerId)
Customer GetCustomerByAddress(Address address)

Thinking that this was the accepted pattern I just blindly started adding this style of methods into my code base, not only in the Repository but in the Service layer too.

Here’s an interface from out codebase for working with images, were you can see CRUD methods jumbled in with other methods to get PagedLists etc.  All in all not very obvious.

public interface IMediaItemsService
    PagedList<MediaItem> GetForUserId(int userId, MediaTypeEnum mediaType, int page, int pageSize);
    List<MediaItem> GetForUsername(string username, MediaTypeEnum? mediaType, int? numRecords);
    void Save(MediaItem mediaItem);
    void Delete(MediaItem mediaItem);
    void Delete(int mediaItemId);
    void Delete(int mediaItemId, int currentUserId);
    MediaItem GetByMediaItemId(int mediaItemId);
    MediaItem GetByUniqueId(Guid uniqueId);
    IPagedList<MediaItem> GetAll(int page, int pageSize);
    IPagedList<MediaItem> Search(MediaSearchParams mediaSearchParams, int page, int pageSize);
    MediaItem Update(MediaForm mediaModel);
    IEnumerable<MediaItem> GetAllMediaItemsForRequest(Guid uploadRequestId);

The Anti-Pattern

What I’ve come to realise is the Byxxxx is an anti-pattern and makes your interfaces far harder to work with.

Not only that but we had some non-standard approaches to Creates, sometimes called Save (does this do updates), Delete maybe called Remove etc.  The upshot being that when approaching adding some functionality you had to study the Service class to separate the CRUD methods from other methods.  It was at this point I realised there was a much better approach to dealing with CRUD (credit must go to @leegunn for planting the germ of the idea).

Convention over Configuration, Overloading and Areas

In my opinion, the most important message from Convention over Configuration (taken from the Wikipedia article) is:

“… a software design paradigm which seeks to decrease the number of decisions that developers need to make…”

So I came up with the following CRUD conventions for our project

  1. CRUD Methods will be added at the top of each method in a Region called CRUD
  2. All CRUD methods should be overloaded where-ever possible
  3. Overloads should ideally call down to one method containing all the business logic – (hopefully I’ll manage to expand on this in an additional post).
  4. CRUD methods should be entered in the order Create, Read, Update, Delete.
  5. Create Methods will be named Create, and should always return the object that was created
  6. Read Methods will be named Retrieve where they return a single object
  7. Read Methods will be named  RetrieveAll  where they return an IEnumerable<T> collection of objects
  8. Update Methods will be called Update and should always return the object that was updated
  9. Delete methods will be called Delete and return void, an exception may be thrown if an error occurs during the transaction.

The Results

By applying these rules to the interface above we get:

public interface IMediaItemsService
#region CRUD

MediaItem Create(MediaItem mediaItem);
MediaItem Retrieve(int mediaItemId);
MediaItem Retrieve(Guid uniqueId);
IEnumerable<MediaItem> RetrieveAll(Guid uploadRequestId);
IEnumerable<MediaItem> RetrieveAll (string username, int? numRecords);
MediaItem Update(MediaForm mediaModel);
MediaItem Update(MediaItem mediaItem);
void Delete(MediaItem mediaItem, int currentUserId);
void Delete(int mediaItemId, int currentUserId);
void Delete(MediaItem mediaItem, int currentUserId, bool isAdmin);
void Delete(int mediaItemId, int currentUserId, bool isAdmin);


IPagedList<MediaItem> GetAll(int page, int pageSize);
PagedList<MediaItem> GetForUserId(int userId, MediaTypeEnum mediaType, int page, int pageSize);
IPagedList<MediaItem> Search(MediaSearchParams mediaSearchParams, int page, int pageSize);

By leveraging overloading where-ever possible we’ve got rid of Byxxx, which makes working with Reads a lot easier.  I can just do MediaItemsService.Retrieve and study the available overloads.  Same for Updates, Deletes etc.

Additionally by separating your interfaces into CRUD and Non-CRUD methods, it makes it easier to spot when a new class, factory etc might be appropriate.  In our case we can see that the non CRUD methods could do with being refactored, possibly rename to RetrieveAllPaged etc.  The point being the refactoring is now easy to spot.

So by conforming to CoC principles for CRUD methods, we’ve definitely decreased the number of decisions our devs needs to make.  Score! 🙂


While writing this post I came across an intriguing idea mentioned at the bottom of this Stackoverflow question about using lambda expressions within your CRUD interface, to allow you to effectively pass a query through to the DB.  This would drastically cut down on overloads not to mention repository methods.

So interface would be

T Single<T>(Expression<Func<T, bool>> expression) where T : class, new();

Usage would be

var user = _dbService.Single<User>(user => user.ID == 12);

There are a few things that could make this tricky to work with, especially if you needed to enforce business rules etc.  However, as a starting place for working with CRUD it’s a really smart idea, and warrants more investigation.

Update 07/12/11

After comments from @alanjmburns I decided he was right and the Delete methods are better to return void rather than a status, and throw an exception if necessary.

SQL Azure – Calculate Table Size

Useful post from Ryan Dunn on calculating table size.  Reprinted below:

-- Total DB Size
SELECT SUM(reserved_page_count) * 8.0 / 1024
FROM sys.dm_db_partition_stats;

-- Individual Table Sizes
SELECT sys.objects.name, SUM(reserved_page_count) * 8.0 / 1024
FROM sys.dm_db_partition_stats,
WHERE sys.dm_db_partition_stats.object_id = sys.objects.object_id
GROUP BY sys.objects.name;