Performance Measurement

Is it possible to enhance NCover to do some performance analysis?

To do this, for example, you could determine if a sequence point represents an entry or exit in a method. (So long as .Net gives this data?) And use a high resolution timer.

The typical agorithm is to log time for a "method only" and "total time". Total time is the simplest, as it is just time between entry and exit of a method. "Method only" time is harder, as you need to suspend counting time for one method each time a new sub-method is called, and remember to resume counting time when returning into a method (or maybe subtract the total time in called methods from the total time for "this method").

I know this must be a fair amount of work, but the big question is: Is there enough information in the sequence points to do this at all?

If this can be done, and NCover explorer can be updated to show the results, then a tool chain of NUnit, NCover. NCover Explorer would be very powerful.

Re: Performance Measurement

Yes, there is. I have planned to do this for a long time. I haven't had time to pull it off though.

Re: Performance Measurement

Great.

Have you considered teaming up with, or reviving one of the other open source profilers?

prof-it for example http://dotnet.jku.at/projects/Prof-It/Default.aspx

or maybe NProf: http://www.mertner.com/confluence/display/NProf/Home

Prof-it seems very dead, amd NProf seems to have some recent life. They don't work with .Net 2.0. In fact I don't know of any open source tool which is useful with .Net 2.0.

It would seem sane not to do everything from scratch, but: I'm not sure if either of these uses sequence points.

Trev

Re: Performance Measurement

Update:

NProf now works with .Net 2.0.

I posted a working build as an attachment to http://www.mertner.com/confluence/display/NProf/Home

The source is available in subversion at

http://www.mertner.com/svn/nprof/

Note: In the zip, Only the "Hook" came from the latest sources. All the rest is the last posted alpha.

Maybe you can make use of some of hook code?

Trev

Re: NCover and Xml Serialization

I've recorded it as a bug to investigate.

Re: Compiling

Hi,

The Boost library is available for download at: http://boost.org/

- Kim

Re: Compiling

Thanks ! I got it up and running now.
Downloaded Boost library (exact version differed from the one used, but that should be fine)
Corrected the paths used to the paths on my machine, and everything compiled fine.

Thanks for the tip.

Re: NCoverExplorer.Console.exe weak documentation

Hi mate,

Sorry to hear it caused you some grief. I admit the examples are kind of spread around the place in terms of my blog etc rather than being handy in the "/?".

I did start experimenting with the Screwturn wiki software a couple of weeks ago with writing some proper help pages which would have all the examples etc anyone could want. However before I get too carried away with that I need to find a hosting provider for it all and thats not a 5 minute exercise. If anyone has any recommendations for an inexpensive provider supporting ASP.Net 2.0 I'm all ears...

Re: NCoverExplorer.Console.exe weak documentation

I have added examples to the console output which will be part of the 1.3.6 release. I'm still wanting to do the wiki documentation thing I mention above but still havent found time to look into the hosting... one day...

Re: ncoverexplorer nant task

Hi mate,

Yes you need to update your NCoverExplorer version at the same time (you are using 1.3.4). You can download the latest version as per the links at the top of this thread:

http://ncover.org/SITE/forums/thread/485.aspx

I am very close to officially releasing this as 1.3.5 - I've just been waiting for any additional bug reports and to get a better idea on when Jamie will be releasing his RTM version of TestDriven.Net to see if we can synchronise our releases like we usually try to.

Re: ncoverexplorer nant task

I've come across this issue also.

If you wanted to fix it before the guys get around to it, modify the NCoverExplorerTask.cs file (Line 548) and specify a format for the ToString function...

programArguments.AppendFormat("/r:{0} ", reportType.ToString("D"));

This got me going again.

Re: ncoverexplorer nant task

Sorry Grant, I didn't see your reply. I should have viewed the complete thread before posting!

I actually had a problem with 1.3.5, the isRegex attribute went missing from the CoverageExclusion class. I was planning to use this, so opted back a version.

Re: ncoverexplorer nant task

Hi mate,

Yeah I've been continuing to add features to the 1.3.5 release and it keeps meaning updates to the NAnt/MSBuild tasks as well. I got lazy about maintaining an "old" version compatible with 1.3.4 and it's hit people who have downloaded recently, apologies.

The isRegex is from new stuff I added on the weekend. If you download the very latest NCoverExplorer 1.3.5 beta AND the latest NCoverExplorer.Extras.zip file (using the links in thread above) then you should be away laughing. Let me know if you have any problems with that combo.

I have been holding off releasing to wait for Jamie to be ready with his TestDriven.Net RTM build. That may still be a few weeks away so think I will go ahead with my NCoverExplorer 1.3.5 "official" release in the next day or so. It will mean that people will have to manually overwrite their NCoverExplorer folders if they are using TestDriven.Net until the new release is available but at least the option will be there if they want it.

Re: Merging NCover reports which contain data from multiple asse

After I've looked closer at ncover output I've noticed some sequence points on line 16707566 at column position 0 (this line of course doesn't exist in our code). NCoverExplorer probably ignores these phantoms (I wonder why ncover adds them at all).

Thus the number of sequence points (both total and unvisited) in Module Class report looks correct.

The second type of report - Module Class Function probably counts sequence points as methods/properties/constructors of the class (Tanimoto has 1 property with a getter, 1 constructor, and 1 method (Similarity), total 3, only the constructr is tested).

I guess, this way of counting results in not comparable results, I would expect real sequence points listed by each "function" (i.e. ".ctor" - 3 total, 3 visited, "get_IsNormalized" - 3/0, "Similarity" - 28/0).

In principle the resulting output meets our needs, I would be grateful if anyone would confirm

a) the clever workaround of NCoverExplorer with the line 16707566

b) that NCoverExplorer really sums up (merges) the sequence points from different ncover reports (like described in the first post)

c) that the output of Module Class Function report (other sense of sequence points) is by design, not a bug.

BTW, great tool this NCoverExplorer! :)

Re: Merging NCover reports which contain data from multiple asse

Ok, (a) seems to be a common ncover feature :)

http://ncover.org/SITE/forums/rss.aspx?ForumID=11&PostID=247

Re: Merging NCover reports which contain data from multiple asse

Hi there,

In answer to (a)/(b) - NCoverExplorer does an intelligent merge of the coverage.xml files, taking methods identified by their module, line number etc and adding together their statistics.

As you discovered NCover in versions up to and including 1.5.6 had a bad habit of including the null instruction sequence points in the output in some circumstances. Peter started trying to rectify this in 1.5.5 but did not do the job completely which created a nastier problem. In 1.5.7 (due shortly) this has been resolved once and for all.

So the totals should be correct for the standard reports. As for the function coverage report I will have to take a closer look - Im snowed under with my day job for the next week or so unfortunately but will take a look after that.

Re: Merging NCover reports which contain data from multiple asse

Hi Grant,

thanks a lot for clearing this out for me. And thank you for the great tool