One of the most common workloads I hear about from customers and developers is video encoding in Windows Azure.  There are certainly many ways to perform video encoding in Windows Azure – in fact, at PDC 2009 I was joined by Mark Richards of Origin Digital who discussed their video transcoding solution on Windows Azure – but until the release of Windows Azure 1.3 SDK, and in particular Startup Tasks and elevated privileges, it was complicated.

As I looked into this workload, I decided that I wanted to prove out a few things in a small spike:

  1. It is not necessary to use the VM Role.  The VM Role was introduced to make the process of migrating existing Windows Server applications to Windows Azure easier and faster, especially when this involves long, non-scriptable or fragile installation steps.  In this particular case, I was pretty sure I could script the setup process using Startup Tasks.
  2. It is possible to use the freely available Expression Encoder 4.
  3. It is possible to use the Web Platform Installer to install Expression Encoder 4.

Before I go any farther, make sure you review two great blog posts by Steve Marx – be sure to read these as they are a great starting point for Startup Tasks:

Okay, now that everyone’s familiar with Startup Tasks, let’s jump into what it takes to get the video encoding solution working.

I decided to put the encoding service in a Worker Role – it makes the most sense, as all I want the role to do is encode my videos.  Consequently, I added a Startup Task to the Service Definition that points to a script:

Code Snippet
  1. <Startup>
  2.   <Task commandLine="Startup\InstallEncoder.cmd" executionContext="elevated" taskType="background" />
  3. </Startup>

Note that I have the task type defined as “background”.  As Steve explains in his post, this makes it easy during development to debug; however, when you’re ready to push this to production, be sure to change this to “simple” so that the script runs to completion before any other code runs.

The next thing to do is add the tools to our project that are needed to configure the instance and install Encoder.  Here’s what you need:

  • WebPICmdLine – the executable and it’s assemblies
  • InstallEncoder.cmd – this is our script that’s used to set everything up
  • PsExec.exe (optional) – I like to include this for debugging purposes (e.g. psexec –s -i cmd)

As I mentioned in my post on Web Deploy with Windows Azure, make sure to mark these files as “Copy to Output Directory” in Visual Studio.  I like to organize these files like this:

image

Now, let’s take a look at the InstallEncoder.cmd script, as that’s where the real work is done.  Here’s the script:

Code Snippet
  1. REM : Install the Desktop Experience
  2. ServerManagerCMD.exe -install Desktop-Experience -restart -resultPath results.xml
  3. REM : Make a folder for the AppData
  4. md "%~dp0appdata"
  5. REM : Change the location of the Local AppData
  6. reg add "hku.defaultsoftwaremicrosoftwindowscurrentversionexploreruser shell folders" /v "Local AppData" /t REG_EXPAND_SZ /d "%~dp0appdata" /f
  7. REM : Install Encoder
  8. "%~dp0webpicmdWebPICmdLine.exe" /accepteula /Products: ExpressionEncoder4 /log:encoder.txt
  9. REM : Change the location of the Local AppData back to default
  10. reg add "hku.defaultsoftwaremicrosoftwindowscurrentversionexploreruser shell folders" /v "Local AppData" /t REG_EXPAND_SZ /d %%USERPROFILE%%AppDataLocal /f
  11. REM : Exit gracefully
  12. exit /b 0

Let’s break it down:

  • Line #2: Turns out that you need to install the Desktop Experience in the instance (it’s Windows Server 2008, after all) before you can run Expression Encoder 4.  You can do this easily with ServerManagerCMD.exe, but you’ll have to restart the machine for it to take affect.
  • Line #4: Create a new folder called “appdata”. (Incidentally, %~dp0 refers the directory where the batch file lives.)
  • Line #6: Update the “Local AppData” folder in the registry to temporarily use our “appdata” folder.
  • Line #8: Install Expression Encoder 4 using the WebPICmdLine. Note that with the latest drop you’ll need to add the “/accepteula” flag to avoid user interaction.
  • Line #10: Revert the “Local AppData” back to the default.
  • Line #12: Exit gracefully, so that Windows Azure doesn’t think there was an error.

If you need to restart your machine before you’ve installed everything in your startup task script, as I did, make sure the script is idempotent.  Specifically in this case, since we restart the machine after installing the Desktop Experience, line #2 will get run again when the machine starts up.  Fortunately ServerManagerCMD.exe has no problem running when the Desktop Experience is installed.

This is all that’s required to setup the Worker Role.  At this point, it has all the software required in order to start encoding videos.  Pretty slick, huh?

Unfortunately, there’s one other thing to be aware of – the Expression Encoder assemblies are all 32-bit, and consequently they will not run when targeting x64 or Any CPU.  This means that you will need to start a new process to host the Encoder assemblies and our encoding process.  Not a big deal.  Here’s what you can do.

Create a new Console project – I called mine “DllHostx86” based on some guidance I gleamed from Hani Tech’s post – and add the Expression Encoder assemblies:

image

Update the platform target so that it’s explicitly targeting x86 – this is required with these assemblies, as I’ve noted elsewhere in my blog (see Using the Expression Encoder SDK to encode lots of videos).

image

Now you can write the code in the Program.cs file to encode the videos using the Encoder APIs.  This itself is pretty easy and straightforward, and I invite you to review the post I mention above to see how.  Also, if you want to do something kind of fun you can add a text overlay on the video, just to show you can.

The next step is to make sure that the DllHostx86 executable is available to include as part of our Worker Role package – we’ll want to run it as a separate process outside of the Worker Role.  Simply update the output path so that it points to a folder that exists in your Worker Role project.

image

This way the any build will create an executable that exists within the Worker Role project folder, and consequently can be included within the Visual Studio project (be sure to set them to copy to the output directory).  The end result will look like this:

image

Now, all we have to do is write some code that will create a new process and execute our DllHostx86.exe file.

Code Snippet
  1. private void ExecuteEncoderHost()
  2. {
  3.     string dllHostPath = @"Redist\DllHostx86.exe";
  4.  
  5.     ProcessStartInfo psi = new ProcessStartInfo(dllHostPath);
  6.     Trace.WriteLine("Starting DllHostx86.exe process", "Information");
  7.  
  8.     using (Process dllHost = new Process())
  9.     {
  10.         dllHost.StartInfo = psi;
  11.         dllHost.Start();
  12.         dllHost.WaitForExit();
  13.     }
  14. }

While I think this is pretty straightforward, a few comments:

  • Line #5: If you want to pass parameters to the applications (which I ultimately do in my solution) you can use an overload of ProcessStartInfo to pass them in.
  • Line #12: Since my simple spike isn’t meant to handle multiple a lot of these processes at once, I use WaitForExit() to ensure that the encoding process completes before continuing.

And there you have it!  That’s all you need to do to setup a Windows Azure Worker Role to use Expression Encoder 4 to encode videos!

If you want to see an example of this running, you can try out my initial spike: http://encoder.cloudapp.net/.

This is just a simple, but hopefully effect, sample that illustrates how simple it is to setup the Worker Role using Startup Tasks.  With this solution in place, you can scale out as many Worker Roles as you deem necessary to process video – in fact, at one point I scaled up to 10 instances and watched them all encoding my videos.  In fact, I bet you could use some of the tips I illustrate in Using the Expression Encoder SDK to Encode Lots of Videos to run multiple encoding session at a time per instance.  Pretty nifty.

I hope this helps!

  • Pingback: Tweets that mention Wade Wegner » Blog Archive » Using Expression Encoder 4 in a Windows Azure Worker Role -- Topsy.com

  • Pingback: Custom performance counters with Windows Azure SDK v1.3 « Vishal's space

  • http://www.snakebytez.com Vikram

    Thanks for the information.

  • http://hypervize.com Brian Blanchard

    Wade,

    We need to encode a large number of live video streams from multiple urls (200 – 3000 simultaneous urls streaming H.264). Once encoded they need to be viewable on PC & IOS.
    The stream doesn’t have to be real time, but we would like it to be as close as possible.

    Can you suggest a way to use the approach in this post and/or http://www.wadewegner.com/2010/08/using-the-expression-encoder-sdk-to-encode-lots-of-videos/ to accomplish this task using Azure?

    Would it make sense to break down the stream into small 1 – 5 minute chunks & encode those instead of attempting to encode the full stream?

    Brian Prince suggested you would be the most likely person to know about a TAP/EAP program for streaming on the Azure CDN. Have you heard of any program that might fit this unique request?

    Feel free to email if you would be interested in learning more about this challenge.

    Thanks in advance for your assistance,
    Brian

    • Wade

      Thanks for the comment, Brian – I’ll follow-up with you via email.

      • Shayaan

        Hey Wade, similar issue here…

        I’m working on a project to allow teachers to interact with students live, using MSN chat services as the medium to ask questions. I want to use Azure for CDN support, but my problem lies in the challenge of getting teachers to have an encoder that will do live smooth streaming with easy access into the site, so students can view the stream and ask questions in real-time. We don’t have a budget for every teacher to have encoder pro, so I’m open to ideas, and using Azure is the way to go for us since there is IIS smooth streaming support as well as infrastructure we don’t have to buy. This is a non-profit site, so ultimately I’d love some ideas.. doesn’t have to be code exactly, but something to lead us in the right direction to get the project off the ground and started as easily as possible, so we can add to the service as we go along. The core right now is going to be pretty simple, just a live video with chat, and move on from there.

        Thanks for any ideas or help!
        Shy

  • HC

    Hi Wade,

    Nice article.

    I’m trying to extend this article to use the expression encoder to encode flv videos in the windows azure.
    The solution is working in my virtual machine (32 bits) but when i deploy to azure(64 bits) isn’t working.
    I’m installing the ffdshow to decode the flv.

    I need some help to overcome this roadblock.

    Here is my StartupTask script:

    REM : Install the Desktop Experience
    ServerManagerCMD.exe -install Desktop-Experience -restart -resultPath results.xml
    REM : Make a folder for the AppData
    md “%~dp0appdata”
    REM : Change the location of the Local AppData
    reg add “hku.defaultsoftwaremicrosoftwindowscurrentversionexploreruser shell folders” /v “Local AppData” /t REG_EXPAND_SZ /d “%~dp0appdata” /f
    REM : Install Encoder
    “%~dp0webpicmdWebPICmdLine.exe” /accepteula /Products: ExpressionEncoder4 /log:encoder.txt
    REM : Install ffdshow
    “%~dp0ffdshow_rev3631_20101115_clsid.exe” /silent
    REM : Change the location of the Local AppData back to default
    reg add “hku.defaultsoftwaremicrosoftwindowscurrentversionexploreruser shell folders” /v “Local AppData” /t REG_EXPAND_SZ /d %%USERPROFILE%%AppDataLocal /f
    REM : Exit gracefully
    exit /b 0

    Thanks in advance for your help,
    Hilário Cunha

    • Wade

      Nice, cool scenario! Did you see the ExecuteEncoderHost method later in the article? This is required in order to run 32 processes, as the worker role itself runs as a 64 bit process.

      • HC

        Hi,

        I’m using the ExecuteEncoderHost technique.
        The solution works for the codecs natively supported by the Expression Encoder 4.

        The problem is with the flv videos, they need a directshow decoder to be installed, i’m trying to use the ffdshow. The ffdshow solution works on 32 bits but not in 64 bits(don’t know why).

        What i need is to find a way of install one flv directshow decoder in the production Azure environment that the expression encoder is able to find and use to encoding the video.

        • HC

          Hi,

          problem solved.

          The user who runs the role was unable to find the ffdshow codecs, i needed to put some entries in the registry.

  • http://www.cloudcasts.net/ Alan Smith

    Hi,

    We are building a Windows Azure solution to encode video files using the H.264 format. As I understand it Expression Encoder 4 Pro is required for this. Do you know if it is possible to install the Pro version using the techniques described in this post.

    Regards,

    Alan

  • Chris Martin

    I’d love to know if it’s possible to use the Pro version as well. Otherwise, I’m going to fall back to VM Role.

  • Chris Martin

    If anyone needs to install the pro version, here’s the (partial) solution:

    download the encoder_en.exe, extract it, and put the setup files in your worker role under a folder called “ExpressionEncoder”.

    Also, there’s no reason to redirect the %appdata% folder in this scenario because the install is silent and control returns immediately. You’ll need to RDP into the instance anyway; if you need to investigate the installation, just head to %appdata%.

    Here’s my startup script.

    REM : Install the Desktop Experience
    ServerManagerCMD.exe -install Desktop-Experience -restart -resultPath results.xml
    REM : Install Encoder
    “%~dp0ExpressionEncoderSetup.exe” -q PROOFOFPURCHASE=[YOUR PRODUCT KEY] FORCEPRODUCTKEY=1

    The only step I can’t figure out is how to activate the product. For now we’ll just RDP into the instance and manually activate it through the UI. We’ll also setup email notifications if an instance needs encoder activation.

    I really hope there’s a better solution to this hiding somewhere at MS.

    Hope this helps someone.

    • Kevin LaChapelle

      To activate on the command line add: ACTIVATIONOPTIN=1

  • Andreas

    Is the project posted for download somewhere?

  • Bret

    Hi, I’m Bret Ahlstrom – Program Manager on the Encoder team. There is a way to force activation to happen when doing a command line installation as Chris Martin describes above.

    Just add the following to that command line:

    ACTIVATIONOPTIN=1

    So the whole command line would become:

    “%~dp0ExpressionEncoderSetup.exe” -q PROOFOFPURCHASE=[YOUR PRODUCT KEY] FORCEPRODUCTKEY=1 ACTIVATIONOPTIN=1

    And activation will happen – assuming that you are net connected, your firewall doesn’t prevent contacting the MS activation servers, the product key is valid, and has not already been activated against more than three times (three is the limit for each product key).

    I hope this helps.

    Thanks,
    Bret Ahlstrom
    Program Manager, Expression Encoder

    • http://www.facebook.com/chris.spanellis Chris D. Spanellis

      I have tried activating that way, and it does not seem to work.  Is that method still valid with SP2?

  • Bret

    OK – I need to amend my comment above. Using Encoder Pro w/activation on an Azure VM will present several problems. While the activation will likely work fine during the initial setup, running as a VM will likely cause activation/licensing checks to fail. There are also other issues as well, such as the question of how accurate royalty accounting can be done in a virtualized environment. We are aware of the desire to have an Azure-based encoding solution, but for now only the free version of Encoder can be used in this way.

    Thanks,
    Bret Ahlstrom
    Program Manager, Expression Encoder

    • http://www.builttoroam.com Nick

      Hi Bret,

      So is there guidance as to how we can encode for smooth streaming w h264 for publishing into azure – it would seem a no brainer for Microsoft to enable this scenario to make full use of their cloud offerings. However, from what you’re saying, even if we have a valid license for Encoder we’ll still run into issues?

      • Bret

        Nick – sorry for the long delay.

        Unfortunately, it isn’t quite as simple as we would hope. The product activation issues are not easy to solve or work around, and that technology is core to the product and is part of our agreement with the royalty partners. I wish I had clear guidance to offer, but there isn’t any right now.

        Thanks,
        Bret

        • http://www.builttoroam.com Nick

          Hi Bret,

          One of the issues we’re running into is that we seem to only be able to install the no-codecs version of Expression Blend. Is this because the codecs version is a different download, or is this a limitation of the product key we’ve entered?

          Thanks