Using secure Subversion from TeamCity

I have installed TeamCity (5.0 EAP version) on a Windows 2003 server. Both the Tomcat web server and the build agent are started as windows services.

The Subversion server is installed on a Linux server with Apache Tomcat web server. The Subversion server is protected with HTTPS.

When I tried to connect to the SVN-server through TeamCity, I always received the authentication error:

svn: Authentication required for '<https://<server name>:443>'

And the strange thing is that it worked fine if I used the SvnKit command tool, with the same user. So it was no certificate problem.

Searching the Internet for solutions always ended up with suggestions to change the svnkit.http.methods parameter.
But it had no effect on My problem. I was sure that NTLM authentication should be used, and from version 4.0.2 of TeamCity, the NTLM protocol is used by default.

Finally, I found out that SvnKit includes two NTLM implementations, the default is pure Java. But its also possible to use the native NTLM through the JNA library.

I added the svnkit.http.ntlm=jna parameter and suddenly the SVN connection was successful!!!
So much pain for this small window :-)

 

JNA is included in the TeamCity Windows build agent package, so its not even necessary to install it on the server.

The SvnKit parameter must be defined in two places, for the build agent and for the web server:

1. The build agent properties file, i.e. <install path>TeamCity\buildAgent\launcher\conf\wrapper.conf:

# TeamCity agent JVM parameters
wrapper.app.parameter.2=-ea
wrapper.app.parameter.3=-Xmx512m
# The next line can be removed (and the rest of the lines renumbered) to prevent memory dumps on OutOfMemoryErrors
wrapper.app.parameter.4=-XX:+HeapDumpOnOutOfMemoryError
# Preventing process exiting on user log off
wrapper.app.parameter.5=-Xrs
# Uncomment the next line (insert the number instead of "N" and renumber the rest of the lines) to improve JVM performance
# wrapper.app.parameter.N=-server
wrapper.app.parameter.6=-Dlog4j.configuration=file:../conf/teamcity-agent-log4j.xml
wrapper.app.parameter.7=-Dsvnkit.http.ntlm=jna
wrapper.app.parameter.8=-Dteamcity_logs=../logs/
wrapper.app.parameter.9=jetbrains.buildServer.agent.AgentMain
# TeamCity agent parameters
wrapper.app.parameter.10=-file
wrapper.app.parameter.11=../conf/buildAgent.properties

2. Configure the Tomcat web server.

Open the configuration window with <install path>TeamCity\bin\tomcat6w.exe //ES//TeamCity.

Add the SvnKit parameter in the Java tab – Java Options:

Restart both services to get the new parameter initiated.

Executing PartCover from NAnt

I wanted to replace NCover with PartCover. PartCover is a new code coverage tool, and its still a freeware.

With NCover I had to run NUnit twice, first to get the unit test results, and then another run to get the code coverage. (At least with the last freeware version of NCover, 1.5.8. I have not tried the commercial versions)

With PartCover its possible to run the tests AND get the coverage at the same time.

Another advantage is that I can use newer versions of NUnit. The NCover 1.5.8 does not work with NUnit from version 2.5.

It seemed rather easy, but I had problems with quotes in the NAnt build file. The PartCover always terminated with an exception, whatever I tried; quotes, variables, expressions:

Invalid option '--target=C:\Program Files\NUnit 2.4.5\bin\nunit-console.exe'

One work-around is to use a NUnit or PartCover configuration file, but I wanted the NAnt build file to be independent.

The solution was to use HTML character entity references, i.e. a double quote (“) can be written as &quot;.

Example of NAnt target that executes PartCover which in turn produces both a coverage report and a unit test report in a specified folder:

  <target name="unitTest">

    <!-- Get all unit test assemblies -->
    <foreach item="File" property="filename">
      <in>
        <items basedir=".">
          <include name="${Build.Output}\bin\${MyProject}.Test.dll"></include>
          <include name="${Build.Output}\bin\${MyProject}.*.Test.dll"></include>
        </items>
      </in>
      <do>

        <echo message="Unittesting ${filename}"/>

        <exec program="${PartCoverHome}\Partcover.exe" failonerror="true">
          <arg line="--target &quot;${NUnitExePath}&quot;" />
          <arg line="--target-work-dir ${Build.Output}\bin"/>
          <arg line="--target-args &quot;${filename} /xml=${Build.Reports}\${path::get-file-name-without-extension(filename)}-UnitTest.xml&quot;" />
          <arg line="--include [${MyProject}.*]*" />
          <arg line="--exclude [${MyProject}.*Test*]*" />
          <arg line="--output ${Build.Reports}\${path::get-file-name-without-extension(filename)}-Coverage.xml" />
        </exec>
      </do>
    </foreach>
  </target>  

Register WCF COM proxies with WIX

This is how I get the registration of WCF COM Proxies to work in WIX installation packets.

The CLSID and APPID are regenerated by the framework each time the version of the assembly is changed by. Avoid that by adding a Guid attribute to the COM proxy interface file (that’s the one generated with svcutil.exe).
I have still not managed to define all guid’s in the WCF Service interface file which I would prefer, as the proxy interface file changes are destroyed each time its regenerated.

namespace MyComProxy
{
    [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
    [System.ServiceModel.ServiceContractAttribute(Namespace="http://myservices.com", ConfigurationName="MyService.IMyInterface")]
    [Guid("E055A238-F196-3EF1-ADE2-AB124C197A1F")]
    public interface IMyInterface
    {
        [System.ServiceModel.OperationContractAttribute(Action="http://myservices.com/IMyInterface/DoSomething", ReplyAction="http://myservices.com/IMyInterface/DoSomethingResponse")]
        string DoSomething(string newValue);
    }
    
    [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
    [Guid("D1902FA8-A2DC-39BD-8009-6047F340E1CC")]
    public interface IMyInterfaceChannel : MyService.IMyInterface, System.ServiceModel.IClientChannel
    {
    }
    
    [System.Diagnostics.DebuggerStepThroughAttribute()]
    [System.CodeDom.Compiler.GeneratedCodeAttribute("System.ServiceModel", "3.0.0.0")]
    [Guid("3BAA7AE8-70C7-3D37-92B9-39971872567D")]
    public partial class MyInterface : System.ServiceModel.ClientBase<MyService.IMyInterface>, MyService.IMyInterface
    {
...

Remember to add the Guid attributes again if the file has to be regenerated with svcutil.

Now its time to use heat.exe to generate the Registry keys that makes the proxy visible to COM clients.

Verify that the “Register for COM interop” is checked for the COM Proxy project. Visual Studio then creates a type library file during compilation.

First generate the keys for the type library (it’s not recommended to use the WIX Typelib element, that’s why I specify –scom to get the actual Registry keys):

heat file -scom MyService.tlb -out tlbtags.wxs

Copy the Registry tags from the output file and add them to the tlb-component in the wxs-file.

Then generate Registry keys for the assembly:

heat file MyService.dll -out dlltags.wxs

Copy those keys to the dll-component in the wxs-file (ignore the <Class> tags). The Codebase keys can be removed.

Empty Actions when mocking Java Web Service

I get the
“System.InvalidOperationException: The operations methodX and methodY have the same action ().  Every operation must have a unique action value”, when trying to mock a Java Web Service with Rhino Mocks in CSharp.

The WSDL-file that was downloaded from the Java Web Service declared an empty soap action for each exported operation.
For example:

<operation name="aMethod">
  <soap:operation soapAction="" /> 
  <input>
   <soap:body use="literal" /> 
  </input>
  <output>
   <soap:body use="literal" /> 
  </output>
  <fault name="MyWebServiceException">
   <soap:fault name="MyWebServiceException" use="literal" /> 
  </fault>
</operation>

WCF uses the action to dispatch an incoming message to the correct method, see Action Property. Each method must have a unique action value.

I solved it by downloading the WSDL-file and remove all soap action declarations:

<soap:operation soapAction="" />

I ran the svcutil.exe on the changed WSDL-file.
WCF then creates unique action values in the service interface file, and the service can be mocked (as I described in Mocking WCF service).

Creating disconnected ADO Recordset in C#

I had some problems to create a disconnected ADO Recordset in C#. Finally I get it right.

public static Recordset CreateDisconnectedRecordset()
{
    // Create new recordset
    var rs = new Recordset();

    // Add some updatable fields
    rs.Fields.Append("name", DataTypeEnum.adVarChar, 20, FieldAttributeEnum.adFldUpdatable, Missing.Value);
    rs.Fields.Append("country", DataTypeEnum.adVarChar, 20, FieldAttributeEnum.adFldUpdatable, Missing.Value);

    // Open recordset
    rs.Open(Missing.Value, Missing.Value, CursorTypeEnum.adOpenUnspecified, LockTypeEnum.adLockUnspecified, 0);

    // Add data
    rs.AddNew(Missing.Value, Missing.Value);
    rs.Fields["name"].Value = "Anders";
    rs.Fields["country"].Value = "Sweden";
    rs.Update(Missing.Value, Missing.Value);

    return rs;
}

Continuous Integration Server Configuration

There are a few simple steps to set up a build server, but I always forget how and where I found the answer on the few problems that always pops up.

Here are the steps for building .NET 3.5 web-projects on a Windows 2008 server (using NAnt and NUnit):

Install:

1. NAnt
2. NUnit
3. NCover
4. NCoverExplorer (for fancy unit test coverage reports)
5. .NET 3.5 Framework SDK
6. CruiseControl.NET (or TeamCity)

Some special handling after installation:

1. Open NAnt.exe.config and change the sdkInstallRoot value to a correct path.

   <readregistry
       property="sdkInstallRoot"
       key="SOFTWARE\Microsoft\Microsoft SDKs\Windows\v6.1\WinSDKNetFxTools\InstallationFolder"
       hive="LocalMachine"
       failonerror="false" />

2. Create the C:\Program Files\MSBuild\Microsoft\VisualStudio\v9.0\WebApplications folder and copy the Microsoft.WebApplication.targets file from a computer with Visual Studio installed. (Used by Web Application Projects)

Debug Classic ASP in Visual Studio 2008

It is a little bit tricky to debug classic ASP in Visual Studio 2008 so I decided to document it. There are many articles that describes it, but this is how I managed to get it to work.

Visual Studio 2008 Service Pack 1 must be installed, it adds debugging and IntelliSense functionality for classic ASP.

Then the IIS must be configured to allow debugging, on both client- and server-side. Remember also to set Send Errors To Browsers to True if the ASP is hosted in IIS7.

Now open Your ASP project in Visual Studio and set the local IIS as Web server:

Ok, then its time to debug. Open Your ASP-page in Windows Explorer. Then attach the VS2008 debugger to the IIS hosting process, i.e. select Debug->Attach to Process... and attach to the dllhost.exe if running on IIS6 or w3wp.exe if running on IIS7.
If there are multiple processes, pick the one with Script type.

The Attach to: should be set to Script code.

(The Show processes from all users must be checked to see the hosting processes)

Add some breakpoints, hit the F5 to reload the web-page and now should the debugger stop at your breakpoints.

Important: Visual Studio often crashes when ending a debug-session. If that happens, kill the dllhost.exe/w3wp.exe to avoid unpredictable errors.

Convert ADO Stream and Recordset to XML

When working with older applications, i.e. pre-.NET, its helpful with functions that converts ADO Streams and Recordset to XML and vice versa. Especially when older applications should be migrated or communicate with .NET applications and its prefered to rewrite as little as possible. Another approach would be to use JSON.

Anyway, here are a couple of converting functions that are useful.

Recordset <-> XML

/// <summary>
/// Convert XML to recordset
/// </summary>
/// <param name="sXML"></param>
/// <returns>Recordset</returns>
public static Recordset recordsetFromXML(string sXML)
{
    if (string.IsNullOrEmpty(sXML))
    {
        // Nothing to convert
        return null;
    }

    // Open an ADO Stream
    var oStream = new Stream();
    oStream.Open(Missing.Value, ConnectModeEnum.adModeUnknown,
                 StreamOpenOptionsEnum.adOpenStreamUnspecified,
                 "", "");

    // Load the XML string into stream
    oStream.WriteText(sXML, StreamWriteEnum.adWriteChar);
    oStream.Position = 0;

    // Create empty recordset
    var oRecordset = new Recordset();

    // Read the XML stream
    oRecordset.Open(oStream, Missing.Value,
                    CursorTypeEnum.adOpenUnspecified,
                    LockTypeEnum.adLockUnspecified, 0);
    oStream.Close();

    //Return the recordset
    return oRecordset;
}

/// <summary>
/// Convert recordset to XML
/// </summary>
/// <param name="oRecordset"></param>
/// <returns>String</returns>
public static string recordsetToXML(Recordset oRecordset)
{
    string xmlString = "";

    if (oRecordset != null)
    {
        // Load recordset into stream
        var oStream = new Stream();
        oRecordset.Save(oStream, PersistFormatEnum.adPersistXML);

        // Get the XML
        xmlString = oStream.ReadText(oStream.Size);
    }

    return xmlString;
}

Stream <-> XML

/// <summary>
/// Convert XML to stream
/// </summary>
/// <param name="sXML"></param>
/// <returns>Stream</returns>
public static Stream streamFromXML(string sXML)
{
    // Load XML into XmlDocument
    var oXML = new XmlDocument();
    oXML.InnerXml = sXML;

    // Get the STREAM element
    var aNode = oXML.GetElementsByTagName("STREAM")[0];

    // Create a binary stream
    var oStream = new Stream();
    oStream.Type = StreamTypeEnum.adTypeBinary;
    oStream.Open(Missing.Value, ConnectModeEnum.adModeUnknown,
                 StreamOpenOptionsEnum.adOpenStreamUnspecified,
                 "", "");

    // Load XML
    oStream.Write(Convert.FromBase64String(aNode.InnerXml));
    oStream.Position = 0;

    return oStream;
}

/// <summary>
/// Convert stream to XML
/// </summary>
/// <param name="oStream"></param>
/// <returns>String</returns>
public static string streamToXML(Stream oStream)
{
    // Create XmlDocument
    var oXML = new XmlDocument();
    oXML.AppendChild(oXML.CreateProcessingInstruction("xml", "version='1.0'"));

    // Add STREAM element to hold the binary data
    XmlElement oElem = oXML.CreateElement("STREAM");

    // Define type of value
    XmlAttribute dt = oXML.CreateAttribute("dt", "dt", "urn:schemas-microsoft-com:datatypes");
    dt.Value = "bin.base64";
    oElem.SetAttributeNode(dt);

    // Convert stream data to string
    oElem.InnerXml = Convert.ToBase64String((byte[])oStream.Read(-1));
    oXML.AppendChild(oElem);

    // Return Xml
    return oXML.InnerXml;
}

Mocking WCF service

We are using continuous integration (with help of CruiseControl.NET) in My current project. Some of the components connects to WCF-services. The continuous integration includes automated unit tests with NUnit.
We needed to mock the services as we don't want to install and run any services on the build server.
I have used Rhino Mocks in previous projects so I decided to give it a try. As always, I started to google for any experience out there, and found this blog which describes exactly what I wanted to do! http://kashfarooq.wordpress.com/2008/11/29/mocking-wcf-services-with-rhinomocks/
Use the WCF self hosting to host a dummy class that expose the service interface. The methods does not need to be implemented as they are mocked with Rhino Mocks.

Consuming WCF Service From Classic ASP

Consuming WCF services from classic ASP is described on several places, for instance http://msdn.microsoft.com/en-us/library/bb735856.aspx#_Toc156817372.

I ran into the "The maximum string content length quota (8192) has been exceeded"-exception, i.e. the message I try to send is larger than 8192 bytes.

Ok, just to add a binding configuration to allow larger messages. But I consume the WCF service from VB Script with a moniker, so where to define the binding configuration?

It took a while before I realised that, as classic ASP does not have a web.config, the binding configuration has to be in the machine.config.

Not so dynamic but an acceptable solution.