race condition?

We've got ncover running as one of the later steps in what is becoming an increasingly large build process. The problem we're having is that ncover now hangs pretty regularly when run as part of the build, but runs fine from the same tree when you run that step alone.

When the hang occurs, NCover sits there "Waiting for profiled application to connect..."

I've "solved" the problem for now, by adding a 5 second sleep to the build process, just before the ncover portion begins. I don't know if this will solve it completely or not, and I don't know if the fix will be needed, or work at all when our new (much much faster) build server arrives in the coming weeks.

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Thomas,

Oh dear, that doesnt look too good. When you say it is reproducible - in a way that you can send me or Peter a repro case or is it just on your own codebase that you cannot share?

I can see why that exception would get thrown in the code but not what would cause it - Ive been running 1.5.7 on half a dozen projects over the last few weeks without any issues. My guess would be some sort of anonymous method type code that in one "pass" by NCover gets profiled and in the next doesnt - causing a mismatch in the array of methods or classes that the merge code then falls over with. Purely conjecture though.

You running NCover using TestDriven.Net or command line?

Re: NCover 1.5.7. - Index was outside the bounds of the array

Do you have a small reproducible case that you can send us?

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter, Hi Grant,

this is not as easy. I've executed my unit and integration tests (approx. 1.000) on my product assemblies.

I've created a dumpfile of NCover when the problem occured. Please let me know if this dumpfile might help you. Grant has my mail address.

I'll try do break down the tests and try to locate the root cause, which leads to the index exeception.

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter, Hi Grant,

I was able to locate the exact test case. Here two AppDomains are created and the same assembly is loaded into each of them to execute a side by side versioning test.

I'll continue breaking it down to a simple testcase.

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter, Hi Grant,

I've broke down an "simple" testcase.

1. Compile the following code to an testassembly "Foo_uTest.dll":

using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.Reflection;

using NUnit.Framework;

namespace Test
{
    /// <summary>
    /// Tester class which is created inside another AppDomain
    /// </summary>
    [Serializable]
    public class Foo : MarshalByRefObject
    {
        public void Write(string name)
        {
            System.Diagnostics.Trace.Write(string.Format("Hello {0}", name));
        }
    }

    [TestFixture]
    public class SideBySide
    {
        string myuTestAssemblyName = "Foo_uTest";

        [Test]
        public void ExecuteInTwoAppDomains()
        {
            // Create our own appBase Path to force loading of the specified assemblies again into each AppDomain
            string appBasePath = typeof(SideBySide).Module.FullyQualifiedName;
            appBasePath = Path.GetDirectoryName(appBasePath);

            AppDomain domv1 = AppDomain.CreateDomain("PlatformV1", AppDomain.CurrentDomain.Evidence, appBasePath, "", true);
            Foo foo1 = (Foo)domv1.CreateInstanceAndUnwrap(myuTestAssemblyName, "Test.Foo");
            foo1.Write("app1");

            AppDomain domv2 = AppDomain.CreateDomain("PlatformV2", AppDomain.CurrentDomain.Evidence, appBasePath, "", true);
            Foo foo2 = (Foo)domv2.CreateInstanceAndUnwrap(myuTestAssemblyName, "Test.Foo");
            foo2.Write("app2");

            // Finalizers should be called during AppDomain Unload
            AppDomain.Unload(domv1);
            AppDomain.Unload(domv2);
        }
    }
}

2. Load this assembly to the NUnit-GUI with NCover control and execute the test.

3. When the test has been finished close the NUnit windows.

4. After that NCover will result in the index out of bounds exception.

Please let me know if and when you may provide a fix for this issue. Currently this blocks me to upgrade to the latest NCover version.

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

Great. Thanks! I'll see if I can reproduce it here and fix it.

Re: NCover 1.5.7. - Index was outside the bounds of the array

I hate to tell you this, but I took your code and was not able to make it fail (either under nunit-console or nunit gui). Can you send me the dump file? peter@waldschmidt.com

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi,

Just adding some information here for you as I'm experencing the same problem.

I have just installed NCover & NCover Explorer for the first time. I was using it to help me explore some code I had downloaded from http://sourceforge.net/projects/csharpoptparse/. I had created a small console app that simply called various methods in the library so I could follow how they worked. Here is the code from the test app that made the call:

private static void TestUsageBuilder(OptionDefinition[] oda)

{

CommandLine.OptParse.UsageBuilder bldr = new UsageBuilder();

bldr.GroupOptionsByCategory = true;

bldr.BeginSection("Header Text");

bldr.AddOptions(oda);

bldr.EndSection();

System.IO.StringWriter sw = new System.IO.StringWriter();

//System.IO.StreamWriter sw = new System.IO.StreamWriter(System.Console);

bldr.ToText(sw, OptStyle.Unix, true);

Console.WriteLine(sw.ToString());

sw.Close();

}

In the TextTransformHelper class of the library is a method called FormatText(text, indent, hangingindent). Towards the end of the method is 1 call to string.LastIndexOfAny(). Whenever it hits this line it throws the same exception. The line runs fine inside VSIDE when debugging. I've added a try/catch around that line and added some debug info to watch what was happening. When in VSIDE all the variables are set properly. When using NCover all the local int variables are either 0 or -1 when it hits this line. Of course adding a -1 to 0 will throw the starting index out of bounds. When run under VSIDE 2005 this call never makes it to this line, but because the values are different under NCover, it gets hit every time. Here is a look at the variables involved when run under NCover:

//line.LastIndexOfAny('- ', 0 + -1 - 1, -1)

//line: 'Header Text'

//cols = -1

//text: Header Text

//indent: 0

//hangingIndent: 0

//first: True

//firstColumns: -1

//maxColumns: 0

//subseqColumns: -1

I'm running XP, VS2005, NCoverExplorer 1.3.6.26, NCover.Console.exe (1.5.7)

Hope this information helps you.

Re: NCover 1.5.7. - Index was outside the bounds of the array

I think there are two different problems here. In the first case, NCover itself is throwing an exception when reporting coverage.

In this second case, it sounds like you are getting different behavior when running inside NCover than you are when running outside NCover.

Re: NCover 1.5.7. - Index was outside the bounds of the array

ThomasD wrote:

Hi Peter, Hi Grant,

I've broke down an "simple" testcase.

1. Compile the following code to an testassembly "Foo_uTest.dll":

We have fixed some problems like this in version 1.5.7. It might be worthwhile for you to check to make sure the the correct version of NCover is installed (and running). Uninstall it fully and check your disk for any rogue copies of CoverLib.dll. Then reinstall the latest version and see if this fixes the problem. I am sometimes hesitant to suggest such a simple resolution, but it has gotten me more than once and a lot of other people too.

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter,

I don't understand what is case one and what is case two for you.

Case one, the NCover exception is reproducible, but the real root cause is unknown. Can you be robust and handle the observed problem?

Case two, possibly the repro does not work correctly, or it's the wrong one. I need to execute all my tests and run into the execption continiously. It's quite hard to break it down.

Did you investigate the dump file I've sent you? I don't use the installation of NCover at all, I do xcopy deployment, then calling regsvr32 to register CoverLib.dll.

I'll try to reproduce it on a different computer tomorrow.

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

Peter,

I've just tried Thomas's example (under Win2K, with nunit 2.4 and VS2005) and replicated the issue with 1.5.7, both running nunit.exe and nunit-console.exe.

Interestingly I also managed to get the "Unable to read beyond the end of the stream" error using that 1.5.8 alpha build, by closing the nunit gui and then pressing ctrl-c rather than waiting for ncover to eventually fail with the above error.

Give me a yell tonight if you like if you need me to replicate this all again - I will try it all under XP when I get home.

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter, Hi Grant,

today I did some further tests on different computers. The problem seems not to occur on a hyperthreading computer, but occurs on single core cpus.

It seems to occur with the "old" NUnit but not with the latest version.

The repro is not always reproducible. Within NUnit I get the error, in a simple console application not.

As a conclusion this sounds like a race condition which depends strongly on the runtime behaviour. Might there by any threading/synchonization issues, in order that some data structures get out of sync?

Grant, I don't use a symbol server in this use case and I'm using the //a filter options.

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

Hi Peter,

I did a retest with the 1.5.8 and the problem is still there?

Do you have already any idea what might cause this exception and how to handle it?

Ciao
Thomas

Re: NCover 1.5.7. - Index was outside the bounds of the array

I am seeing this problem in 1.5.8 also. Any ideas on how to address it?

Steve

Re: NCover 1.5.7. - Index was outside the bounds of the array

we are seeing this problem also. It occurs when we try to run ncover against nunit 2.4.1 which is running against two assemblies. If we run the assemblies independently there is no problem. Any news on a fix?

Re: NCover 1.5.7. - Index was outside the bounds of the array

further to this, I just installed 1.5.5 and it works correctly.

Re: NCover 1.5.7. - Index was outside the bounds of the array

It looks like NCover.Framework.CoverageData.MergeModule() is comparing the lengths of two arrays in a loop without checking they are the same length.

Some more WinDBG/SOS give the following account.

Let's look at the CoverageData that is being used:

0:000> !dumpobj 0x01279b6c
Name: NCover.Framework.CoverageData
MethodTable: 009154e4
EEClass: 00cb30d4
Size: 36(0x24) bytes
(C:\Tools\NCover.Framework.dll)
Fields:
      MT    Field   Offset                 Type VT     Attr    Value Name
79124228 4000001        4      System.Object[] 0 instance 01279c68 Modules
79124228 4000002        8      System.Object[] 0 instance 01f61bb4 Documents
79105ef0 4000003       10      System.DateTime 1 instance 01279b7c StartTime
79105ef0 4000004       18      System.DateTime 1 instance 01279b84 MeasureTime
790fa3e0 4000005        c        System.String 0 instance 01f63f2c ProfilerVersion
0:000> !dumparray 0x01279c68
Name: NCover.Framework.ModuleData[]
MethodTable: 79124228
EEClass: 7912479c
Size: 104(0x68) bytes
Array: Rank 1, Number of elements 22, Type CLASS
Element Methodtable: 009156ac
[0] 01279cd0
[1] 015b1634
[2] 015b18f4
[3] 015b1c38
[4] 015b1f04
[5] 015b2254
[6] 0183c908
[7] 01894134
[8] 01a2c590
[9] 01a2c914
[10] 01a2cc94
[11] 01a2cfe4
[12] 01a2d2b0
[13] 01a2d57c
[14] 01a2d8cc
[15] 01cb7f74
[16] 01d0f7a0
[17] 01ea7bfc
[18] 01ea7f4c
[19] 01eadeec
[20] 01eae410
[21] 01eae8bc

It looks like CoverageData.MergeMultiplyLoadedModules produces a unique list out of this array, merging the module data as it goes through it. Each element of the Modules array is a ModuleData instance. Each ModuleData instance has a Classes array which holds ClassData elements. The MergeModule method assumes these arrays always have the same length per ModuleData value (keyed on AssemblyIdentity, it appears), but it doesn't hold up. Entry [0] and entry [8] are both ModuleData instances for 'mscorlib.dll', so they get passed into MergeModule.

Here is the dump of Modules[0].Classes:

0:000> !do 01279f2c
Name: System.Object[]
MethodTable: 79124228
EEClass: 7912479c
Size: 9296(0x2450) bytes
Array: Rank 1, Number of elements 2320, Type CLASS
Element Type: NCover.Framework.ClassData
Fields:
None

and Modules[8].Classes:

0:000> !do 01a2c904
Name: System.Object[]
MethodTable: 79124228
EEClass: 7912479c
Size: 16(0x10) bytes
Array: Rank 1, Number of elements 0, Type CLASS
Element Type: NCover.Framework.ClassData
Fields:
None

Notice the difference in size. This isn't checked, and a loop is started on the length of the first ModuleData instance's Classes array. This results in the IndexOutOfBoundsException.

Re: NCover 1.5.7. - Index was outside the bounds of the array

I can also confirm that 1.5.5 is free of this defect.

Re: NCover 1.5.7. - Index was outside the bounds of the array

i am seeing this problem as well. i am trying generate reports on two assemblies, but i can reproduce the problem consistently with just one assembly.

please let me know if there is anything i can do to help.

Re: NCover 1.5.7. - Index was outside the bounds of the array

I'm seeing this issue as well. It looks like reverting from 1.5.8 and 1.5.7 to 1.5.5, it all works ok.

Nick

Re: NCover 1.5.7. - Index was outside the bounds of the array

Peter -

Is there anything more I can do to help you out with this issue?

Re: NCover 1.5.7. - Index was outside the bounds of the array

We are also seeing this issue with NCover 1.5.8
Is there anything we can do to help tracking this problem down?

Regards,
Anthony

RE: race condition?

I am seeing this problem also. I am running NCover 1.5.8 from TestDriven. How can I configure TestDriven so that it executes a different version of ncover? Thanks

RE: race condition?

pogorman,

Thanks for writing in! I recently wrote to the creator of TestDriven, Jamie Cansdale, and asked him this very question. He told me that TestDriven passes a call to NCover.Console.exe to the command line from the TestDriven GUI. That means that whichever version of NCover you have installed should get called. So if your path goes to (usually) C:\Program Files\Ncover, then the command line will execute whichever version of NCover it finds there. If you have the latest version--3.4.12 at www.ncover.com/download--installed there, that should be the version that is called.