How To

How to set up Remote Desktop Connection

Wed, 06 Feb 2013 18:05:00 GMT

Windows Azure allows you to Remote Desktop to any of your deployment instances. Follow this how to for the setup.

We need a tool called CSEncrypt Command-Line Tool, you can get it by installing Windows Azure SDK for .NET. Download WindowsAzureLibsForNet-x64.msi or x86.msi then you should be able to find it at the default installation path: C:\Program Files\Microsoft SDKs\Windows Azure\.NET SDK\2012-10\bin

Run a Command window and type

CSEncrypt Get-PasswordEncryptionCertificate

The output is something like this, a list of all the certificates that are usable for encryption on your machine.

To create a new one, we type

CSEncrypt New-PasswordEncryptionCertificate -FriendlyName MyFourtyTwoRDPCert

This one has longer output but if you run the first command again, you should see the newly created one listed there.

Now let's encrypt the RDP password:

CSEncrypt Encrypt-Password -Output t:\dong\FourtyTwoRDPPass.txt -Thumbprint DF9913DA63103F0D62C7FC02E5B5A52732E65FA3

Note you might not able to write to the SDK path so use a writable path like C:\Users\somebody; the Thumbprint is what we got from the step before. If success, in the text file we have a long string which is an encrypted password you just typed.

We are done with CSEncrypt, let's switch to PowerShell to get the certificate file itself, you need to upload this file to Windows Azure Portal.

ls cert:\CurrentUser\My

This will list certificates including our newly created one, you can check on the Thumbprint value, then starting from 0, count the position of this cert, e.g. 2, you shall be able to see the same cert if you do

(ls cert:\CurrentUser\My)[2]

Now type those code:

$bytes = (ls cert:\CurrentUser\My)[2].export([System.Security.Cryptography.X509Certificates.X509ContentType]::pfx, "StrongPassword")

[System.IO.File]::WriteAllBytes("t:\dong\FourtytwoRDPCert.pfx", $bytes)

Note this 'StrongPassword' is not the same as our RDP password, this one is used to encrypt the pfx file, so please use a real strong one and keep it somewhere secure. This .pfx file is what we need. You can double check if everything is alright by

Get-PfxCertificate t:\dong\FourtytwoRDPCert.pfx

It will ask for the password and show the output with Thumbprint you can verify. (BTW if you are using Windows Server 2012, there is a new PowerShell comlet you can use for this purpose in one go: Export-PfxCertificate

Summary

After this How to you shall have the following things ready:

a text file holding the encrypted password in a long string
a pfx file holding the cert and private key in one file
the password you used to encrypt the pfx file
the password you typed that's going to be used for RDP login
the Thumbprint for this cert

How to set up Windows Azure Connect

Fri, 08 Feb 2013 16:04:00 GMT

Windows Azure Connect is a modern IPv6 and IPSEC based technology you can use for many advanced purposes. In our case, Fourtytwo uses it to allow your on-premise desktop to be able to communicate with your deployment instances via IPv6 and IPSEC, through your firewall. Unlike the Windows Azure Virtual Network, Connect is Point-to-Point, more on this later.

To deploy Fourtytwo, we only need a 'ActivationToken':

Login to your Windows Azure Portal, click the top right corner you shall see this, then click 'Previous portal', this will take you to the old management portal in Silverlight, so you need to use a browser that supports Silverlight.

After fully loaded, click 'Virtual Network' at bottom left, when you see this, click 'Get Activation Token' button, then we get a GUID string. That's all we need.

For your on-premise machine, you need to 'Install Local Endpoint', you can do this by click the relevant button above. You can also define 'Groups' e.g. 'DEV' for developer machines.

Once Fourtytwo is deployed and running, you can see a 'WorkerRole' showing in the Groups and Roles list, you then need to 'Edit Role Connections', to include in your desktops or groups, after a short while when Connect updated all the policies on your desktops and Fourtytwo instances, they will be able to see and talk to each other.

How to use Windows Azure Portal to deploy Fourtytwo

Fri, 08 Feb 2013 17:47:00 GMT

Here we show in detail how to use Windows Azure Portal to deploy Fourtytwo.

1 - Create a new 'Affinity Group'

After you login the Windows Azure Management Portal, you'll see something like this:

Then click left panel 'NETWORKS', then click 'AFFINITY GROUPS' in the main area; you'll see a 'create' at bottom, click and give a name to your affinity group, and choose the region appropriate to you. By using this, we can group all your Azure resources to the same region/data centre thus having better performance and reduce the cost.

2 - Create a new 'Storage' account

Left panel again, click 'STORAGE', then click the bottom left 'NEW' button, choose 'QUICK CREATE'. Choose a unique name (the URL) for your storage account, in the drop down list of REGION/AFFINITY GROUP, remember to choose our new affinity group other than a region! Then uncheck the 'Enable Geo-Replication'.

Once your new storage account listed in the main area is ready, click on the name, then click 'CONTAINERS', click 'ADD CONTAINER' at bottom, give a name e.g. 'TEMP', leave it as 'Private'.

Then in the main area, click 'DASHBOARD' above your list, at the bottom click 'MANAGE KEYS', if you click on the little icon right side to your 'PRIMARY ACCESS KEY', this key will be copied to clipboard.

Now we need to use the Azure Storage Explorer tool. Run that and click the 'Add Acount' button at top, fill in your storage account name, and paste in your key. Don't tick other checkboxes. You should be able to see your account listed in the dropdown list, and under 'blobs' there should be the container e.g. 'TEMP' showing up. Use this tool please upload the fourtytwo_db.bacpac file into container 'TEMP'.

3 - Create a new 'Database'

At left panel, click 'SQL DATABASES', then click bottom 'IMPORT', click 'BACPAC URL' you should see your container 'TEMP' then the bacpac file, choose that; then give your database a name, e.g. 'fourtytwo'; for the 'SERVER' dropdown, choose 'New SQL Database Server' if you haven't created one. Fill in a login name and give a strong password (you need to keep this information somewhere secure), remember to choose the same region as our affinity group (SQL Azure does not use affinity group yet.) Check the 'CONFIGURE ADVANCED DATABASE SETTINGS', on next page, a 1GB WEB edition is enough.

Please check your new Server - CONFIGURE - allowed services is 'YES' to 'WINDOWS AZURE SERVICES', this will open the firewall to our cloud service.

4 - Create a new 'Cloud Service'

At left panel, click 'CLOUD SERVICES', click bottom 'NEW', choose CUSTOM CREATE. Fill URL with a name, e.g. Fourtyone or Fourtythree, then choose our affinity group and tick 'Deploy a cloud service package now'. On next page, give a name for this deployment, e.g. 'Fourtyone-2013-02-06-d1' for a day's first deployment. Then you can either choose from local or from container for the fourtytwo.cspkg and ServiceConfiguration.Cloud.cscfg file. For the 'ENVIRONMENT' we always use 'PRODUCTION' since 'STAGING' doesn't make sense in our case. Tick all 3 checkboxes. On next page, pick the .pfx certificate file and give the password that was used to secure this pfx file, this is used for Remote Desktop Connection to your instances.

How to control Fourtytwo with T-SQL

Fri, 15 Feb 2013 13:50:00 GMT

300

When you successfully deployed Fourtytwo on Azure, you can use the following to fire T-SQL:

SQL Azure Management (in SilverLight) via Windows Azure Portal
SQL Server Management Studio

Fourtytwo uses a very small SQL Azure database (1GB Web edition) to manage files, tasks. In fact inside this database we only have 6 tables and 2 stored procedures, yet they are able to provide all the concurrent control, expiring and retry logic.

For our flexibility we didn't enforce 'Foreign Key' relationships on the 4 tables at right hand side, otherwise you shall see arrowed lines link them together as normal ER model does: TaskRun depends on TaskType and FileBlob, in turn TaskType and FileBlob depends on FileType. TaskLock and Workers are standalone entities.

Now we show the purpose of each table:

This is a non-essential housekeeping table, when deployment starts, each instance will insert one row here to answer the 'roll call' (kind of, more like shouting aloud I'm alive). You can keep on checking this after you started the deployment, once you saw rows here it means Worker Roles have no problem contacting SQL Azure database, otherwise it means serious trouble.

The only non-essential table
Safe to clean up at any time
Column Name is the WINS name of the instance, you can use it for PowerShell Remoting if you enabled Windows Azure Connect

If you have read the example recipe page, you will see this is the concrete thing to hold a line of Task Type definition.

Suggest you to leave IsEnabled = 0 when you are doing the Insert, only enable it when you are ready to run the task
Any row with IsEnabled = 1 will effect immediately, i.e. new run will spin up after a Insert, no run will spin up after a Delete
The core table you need to fire Insert, Update, Delete on

This is a new concept we didn' mention in the recipe page. A Task Run is a combination of a File and a Task Type. Each row 'potentially' means a run of the task on one of the instance (you can see which by column InstanceID).

If you see '9999' in the ExpireTime column, this means a run is 'registered' but didn't run because it was part of a multiple file run (i.e. many-to-one), only when all input files are ready will you see a run with a real time indicates a real run.
ExpireTime is StartTime + 2 hours, this is now hard coded, if your program doesn't end within 2 hours, it will be killed on the instance and could be spin up again on another instance. We might in the future allow you to define your own time span.
If you keeps on checking this table, you might notice ExpireTime got updated sometimes, this is due to the extra file upload and download (all takes time), we take this into account and leave a real 2 hours for the program.
You can check the whole command line got fired after all the variable replacements to see if you defined the Task Type correctly by looking at column Command. Well, if you got anything wrong you shall see it immediately, but if it managed to run at all, you'll only see it after it finished the run.
You can play with column ExpireTime to manually expire a run.
If you Delete a row from this table yet related TaskType and FileBlob still around, it will be spin up again.

This is for internal usage. For the situation that only one instance can do something special, who ever won the lock will run and others will wait.

Again you shall recognize this is how you define a File Type. This should be ready before running, but you can also Insert new row when you need it.

This is a loose mirror of files in the container. By 'loose' we mean that currently if you remove a row from this table the file in the container won't be deleted, and vice versa if you delete a file from a container if the row already exists it won't get deleted.

Only container defined in Task Type Container_in will be scaned, for matching File Type.
You can check if your File Type regular expressions work or not by looking at columns NameBase and NameOrder.

How to modify the ServiceConfiguration.Cloud.cscfg file

Fri, 08 Feb 2013 16:55:00 GMT

200

Before you can deploy Fourtytwo, you should modify ServiceConfiguration.Cloud.cscfg file. This is an XML file that is part of a deployment. As you will see this file contains many security information thus please keep it secure!

<?xml version="1.0" encoding="utf-8"?>
<ServiceConfiguration serviceName="Fourtytwo" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="3" osVersion="*" schemaVersion="2012-10.1.8">
  <Role name="WorkerRole">
    <Instances count="2" />
    <ConfigurationSettings>

Depends on how much work you need to process, change the Instance count accordingly, the minimum is 1, the maximum is your total core count of your Azure account e.g. the trial account is 20, then divide by 4 (For Large size VM).

      <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=xie;AccountKey=qp17XKZ2gXs/tYNfsrjDilC8w3GTZ2UqRjuTJwQYG6ZkAvA0KFltwL5kOg1zKxXYAZxUl9IrbAmdvu18P6E/xw==" />

Here you need to give your new Azure Storage account name and key, you can get the key string easily by using Azure Portal.

      <Setting name="SQLAzureConnection" value="Server=tcp:someserver.database.windows.net,1433;Database=Fourtytwo;User ID=someuser@someserver;Password=PassW0rd;Trusted_Connection=False;Encrypt=True;Connection Timeout=30;" />

Here we need the SQL Azure database connection string, you can get this from Azure Portal by finding your database and click 'Show connection strings' at Dashboard. Remember to give the real password here.

      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.Enabled" value="true" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountUsername" value="dong" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountEncryptedPassword" value="MIIBvgYJKoZIhvcNAQcDoIIBrzCCAasCAQAxggFvMIIBawIBADBTMD8xPTA7BgNVBAMMNFdpbmRvd3MoUikgQXp1cmUoVE0pIEVuY3J5cHRpb24gVG9vbCB2ZXJzaW9uIDEuNy4wLjACEErIo/BsDLG2T9DcxPm/7CwwDQYJKoZIhvcNAQEBBQAEggEAUG8EF0ePgYnQqOLGdsm7aV2Ek71XDRKP50BTcxJH/swj8YLeTAG9jzR4Tq0n+NB6FYIbF2MUR32mIjrxBlod26soJxJEp0WYeBXXc89nox1S3sCo01U7zjjl9vClFTWp+xxO2DGNOvsjl6ySyStIMOl/qiSrNUA10t7feWaM8w6PmyuUMrAqg/cbABMgSirJlRC+OPQmssPnU8pCkALaFsUhDftgkK0BHPyNvAbL7p5yssgwMOicsV3cFSjHw4qz/GuE6HqOAG8w+ggP+joWyo6eVpOGGZfimL7NGQ0od4tEGb6PlTpo6h6mWIpnqOkj/yQfcZMbXsHJpHoummJ/DTAzBgkqhkiG9w0BBwEwFAYIKoZIhvcNAwcECFoFt+W9B34lgBAigtTZdIR1c2U0SPuCoZbi" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteAccess.AccountExpiration" value="2013-09-12T23:59:59.0000000+01:00" />
      <Setting name="Microsoft.WindowsAzure.Plugins.RemoteForwarder.Enabled" value="true" />

Here we config the Remote Desktop, first give a user name, don't use 'administrator' though. For the encrypted password, this is a long string, you should have it after you completed How to set up Remote Desktop Connection step, note your text file might be in multiple lines, thus please modify it into a single line before you copy and paste to here. You can also give an expiration date for this user account.

      <Setting name="StorageConnection" value="DefaultEndpointsProtocol=https;AccountName=xie;AccountKey=qp17XKZ2gXs/tYNfsrjDilC8w3GTZ2UqRjuTJwQYG6ZkAvA0KFltwL5kOg1zKxXYAZxUl9IrbAmdvu18P6E/xw==" />
      <Setting name="StorageConnection_http" value="DefaultEndpointsProtocol=http;AccountName=xie;AccountKey=qp17XKZ2gXs/tYNfsrjDilC8w3GTZ2UqRjuTJwQYG6ZkAvA0KFltwL5kOg1zKxXYAZxUl9IrbAmdvu18P6E/xw==" />

Here Storage account again, do the same as in above steps.

      <Setting name="string.xdrive.name" value="Fourtytwo_xdrive.vhd" />
      <Setting name="StorageContainerLog" value="fourtytwolog" />
      <Setting name="StorageContainerVhd" value="fourtytwovhd" />

Here you can give a file name for your xdrive (Virtual Hard Drive) vhd file. Next is the container's name to hold the running log files, then the container's name as to where to find the xdrive vhd file. Those two containers must exist in your storage account, if not, please create them.

      <Setting name="internal.string.regex1" value="<(?![io]:)([^<>]+?)(?::(\d+))?>" />
      <Setting name="internal.string.regex2" value="<([io]):(?:([a-zA-Z0-9-]{3,63}):)?(.+?)>" />
      <Setting name="internal.int.largefileopts" value="15" />

Internal use for now.

      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.ActivationToken" value="524c3848-27be-4ed0-93d7-56ecaf9240fd" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Refresh" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.WaitForConnectivity" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Upgrade" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.EnableDomainJoin" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainFQDN" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainControllerFQDN" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainAccountName" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainPassword" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainOU" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.Administrators" value="" />
      <Setting name="Microsoft.WindowsAzure.Plugins.Connect.DomainSiteName" value="" />

Here for the Azure Connect, paste in the Activation Token here. The rest can be left as is.

    </ConfigurationSettings>
    <Certificates>
      <Certificate name="Microsoft.WindowsAzure.Plugins.RemoteAccess.PasswordEncryption" thumbprint="DF9913DA63103F0D62C7FC02E5B5A52732E65FA3" thumbprintAlgorithm="sha1" />
    </Certificates>
  </Role>
</ServiceConfiguration>

For Remote Desktop again, this is where you put the Thumbprint of your RDP certificate.

Please try your best not to make typos or missing brackets so this file will stay a valid XML file, otherwise your deployment might not work.