Static Code Analysis for XNA Applications Part 4: Automatic Build Checking

Hidden in the code that I uploaded to the CodePlex in our last post was a technique for doing automatic checking on a build. I wanted to share a little about that technique.

Thus far we have been using the GUI version of FxCop. While a great tool for browsing the results, it is not appropriate for an automatic build. Thankfully, Microsoft provides a second FxCop executable: a command line version FxCopCmd.exe.

This command line FxCop obviously takes all of its input on the command line, it automatically performs the analysis, and outputs the results as text, again on the command line. The text output may either as plain text, or XML with an XSLT transformation applied.

For the automatic checking, I used the following command line as a post build step:


if "$(ConfigurationName)" NEQ "Ship" "C:\Program Files\Microsoft FxCop 1.35\FxCopCmd.exe" /console /project:"$(ProjectDir)$(SolutionName)_$(ConfigurationName).FxCop"


The first part of the command line is a check for which project configuration we are building. You’ll recall in Part 3, we added a third configuration to the two standard configurations of Debug and Release: Ship. The Ship configuration is set up just like Release, but without any code verification support. Therefore there is no sense in running FxCop on it.

This additional configuration was necessary so that we could do analysis on code both in #if DEBUG blocks and not. As a matter of principle, we should not add any code that would only be compiled into only the Ship build. It will not be analyzed by FxCop at all!

The next part of the command line, after the actual invocation of FxCop, we identify that we want plain text output with the /console switch, and finally we specify the project file we want to process with the /project switch. The FxCop project name also incorporates the configuration name. This allows us to have separate FxCop projects for both the Debug and Release configurations.

Now, because we are focused on XNA, we must use Visual C# Express Edition, which only has a fairly limited build system. We could add a FxCop command to every configuration of every project, but that poses several problems. The first is that FxCop is not exactly fast. Each FxCop invocation is generally slower than the rest of the build process combined. Secondly, we’d like to have a single CustomDictionary.xml, so all of the FxCop project files should live in the same directory as that file, which means our FxCop project files cannot be added to their corresponding Visual C# 2005 projects. (All project files have a root directory, and the files in the project are relative to that root, with no “..” support.)

To rectify the situation, I added another Visual C# project. This project has a single dummy source file that is built, but does not contain any code that is used. This project is also dependant on all of the projects for the modules of our game, so it builds last, and it contains the FxCop command line above as it’s post-build step. This Visual C# Express Edition project also has both of the FxCop project files (again, on for Debug and one for Release) in it as well. This solves both of the problems. We can keep our single CustomDictionary.xml (also in the Visual C# Express Edition project) and we run FxCop last, and only once. We can also keep all of our custom settings in each of the FxCop project files. That’s our automatic FxCop checking on build.

Now, as a matter of process, I prefer using the GUI FxCop to track down all of the initial issues. The automatic checking is useful to insure that once you’re clean, you stay clean.

That covers the basics of FxCop. Now, if I can only get my desktop fixed and run some XNA code, I’d like to cover some other topics.

OE

OvidEast @ CodePlex

I’ve set up my source code repository at CodePlex. The source code I’ve written so far is a version of an XNA webblog staple: the frame rate counter component. I’m going to go about it a little differently. Starting with TehOne’s frame rate component, (and nifty activation handler base classes,) I cooked up something new.

The frame rate counter component itself is not drawable. Instead, it is exposed as a performance counter. This counter can be exposed via the Windows Performance Monitor, the Remote Performance Monitor for Xbox 360, or the other component I’ve written: a performance counter monitor.

The monitor is a drawable component that can either display an arbitrary number of arbitrary performance counters on-screen over the game. It can also send those counter’s data to a logging facility as it’s obtained. Both the text drawing and logging facilities are abstracted by interface, allowing the component to interact easily with existing infrastructures.

Notice I said arbitrary performance counters: in addition to the frame rate counter I provide, Windows itself provides many useful counters: information about the managed heap, CPU usage, and others. All of which can be monitored.

Now, the code in CodePlex is by no means final, it’s only been checked in to protect it. The unit tests haven’t even been started, nor have I even run this code yet, as I no longer have access to a system that meets the XNA minimum bar. (I situation that I’m rectifying as soon as possible. It is FxCop clean, of couse. ) I’d certainly appreciate any feedback, and watch this space or the CodePlex for the final versions, which should be coming soon.

OE

Static Code Analysis for XNA Applications Part 3: Suppressing Messages

In the previous post, we examined how spelling related FxCop messages could be handled by using a custom dictionary file. In this post, we’ll look at the techniques for dealing with other messages that we do not intend to change code to resolve.

In working with some code recently, I was working my way through the FxCop messages as always, when I came upon an AvoidNamespacesWithFewTypes message. The Info property for the message reads: “A namespace should generally have more than five types.” And the Resolution: “Consider merging the types defined in ‘OvidEast.Xna.Framework’ with another namespace.”

Now the code I’m writing is meant to mirror the XNA namespaces as much as possible. Thus, in this case the code in OvidEast.Xna.Framework both corresponds with Microsoft.Xna.Framework, it is also shared amongst the sub-namespaces. In other words, while in general I agree that namespaces should probably have more than just a few types in them, in this case, we believe our design is a valid exception to the rule. We can suppress the message.

There are a variety of different ways to suppress a message. The first and simplest way is to simply turn off checking for the rule that generated the message. Clicking on the “Rules” tab, Expanding the “Design Rules” tree node, and deselecting the “Avoid namespaces with a few types” check box will do the trick. In some cases, this is the best way to go, especially if the rule in question directly conflicts with a design guideline of your own. Information about which rules are enabled and disabled may be saved into the FxCop project file.

The second method is to exclude the message itself in the FxCop project file. The project file will always contain a list of all of the messages from the last run, and you can exclude a message from future runs by right clicking the message, selecting Exclude, and saving the project. In later runs, the message will appear in the “Excluded In Project” tab in the GUI.

The third method is to add an annotation to the source code itself to exclude the message. This takes the form of an attribute. You can start by right clicking on the message you wish to exclude, go to Copy As, and select “SuppressMessage.” The text that is copied to the clipboard can be pasted almost anywhere in your code. Some people prefer to keep all message suppressions together as much as possible, others keep the suppression as close to the offending code as possible.

The text looks like:

[module: SuppressMessage("Microsoft.Design", "CA1020:AvoidNamespacesWithFewTypes", Scope="namespace", Target="OvidEast.Xna.Framework")]

The attribute identifies the scope the attribute exists for (the module it is pasted in,) the message to be suppressed, the scope in which the message is to be suppressed (not the same as the scope of the attribute!) and finally a target identifying that scope.

There are other useful properties of the SuppressMessage attribute. The most useful is the “Justification” property. This allows you to add a string to the attribute that explains in human readable terms why the message was suppressed. This is very useful to maintainers to understand why potentially unsafe code was not caught by FxCop.

After adding a Justification string, you’ll find your brand new attribute does not compile. You’ll need to add a reference to the System.Diagnostics.CodeAnalysis namespace to get the attribute. After recompiling again, you’ll find the attribute does not suppress the message. It is also necessary to add a #define (or use the /define compiler switch) for the macro CODE_ANAYSIS.

In the final design, I added CODE_ANLYSIS to my Release and Debug configurations’ /define list. I added a third configuration, cloned from Release, called Ship. Ship does not have CODE_ANLYSIS set. I added this to my references section in each source file:

using System.Diagnostics.CodeAnalysis;

Finally, I added the attribute to the offending source file.

[module: SuppressMessage("Microsoft.Design", "CA1020:AvoidNamespacesWithFewTypes", Scope="namespace", Target="OvidEast.Xna.Framework", Justification="The OvidEast namespaces mirror the Microsoft layout. These classes belong in the Framework namespace, despite there are only 2 of them.")]

OE

Links of interest:

• MSDN documentation on the SuppressMessageAttribute.
• An FxCop team blog post on why the CODE_ANALYSIS define is necessary.

Free .NET Performance Guide

Microsoft has recently released “Improving .NET Application Performance and Scalability.” The guide covers many of the .NET applications, such as ASP.Net, ADO.Net, XML, and others. But it also covers many topics relevant to XNA programming. Best of all, you can read it online or download it as a pdf.

In my first read, I found the first few sections, which cover general engineering practices regarding performance, good, but possibly a review for more experienced developers.

The good stuff starts in Chapter 5 “Improving Managed Code Performance.” Chapter 13, “Code Review: .NET Application Performance” is useful, and finally chapters 15-17 on measuring, testing and tuning, respectively contain good information.

So, while you can skip about 50% of the book, I do pretty much guarantee that you’ll learn something for the money you didn’t spend.

OE

Links:

• Read the guide online.
• Or, download it as a pdf.

Static Code Analysis for XNA Applications Part 2: Spelling

In the first post in this series, we introduced FxCop a .NET static code analysis tool. We ran the GUI version of FxCop, and applied it to a simple XNA application. Now that we’ve got the basics, this tutorial will show you how to start up the spell checker for your source code.

We’re going to start with a much more complicated XNA application for this tutorial. I’ve chosen to use Rocket Commander XNA by Benjamin Nitschke. It’s got plenty of code and plays great. Go ahead and download the source code from Benjamin’s site. Once you’ve got it unzipped, go ahead and open up the RocketCommanderXna.sln file and run the FxCop tool, and analyze the code.

Holy Cow! 686 messages! That’s a lot of messages, and the sheer volume of messages brings up a few interesting points. The first point is that if you are going to make heavy use of static code analysis (and I certainly recommend it,) it really pays to start your analysis runs early in the development cycle and fix the messages as you go along. Otherwise you could be staring down 686 messages or more, which can be really tedious to deal with, especially when you are looking at code that you have not written, or have not looked at in a long time. (Small update: there are a different number of messages for Release builds of Rocket Commander than Debug. That is because there is code in #if DEBUG blocks.)

There is an inherent danger in changing code, which is going to be a re-occurring theme in all of my code quality posts. Each time you change something you run a risk of introducing a build break, or worse a regression bug. So attempting to make 686 code changes to Benjamin’s code might not be the right thing to do (unless you are, in fact, Benjamin, or you’ve got plenty of time to test for regressions.)

The second interesting point is related to something I mentioned in the last post, but wanted to reinforce here: FxCop is not perfect. Scrolling through the message list, you can see that many of the messages are obviously not bugs, and some messages might not even be worth fixing (remember: this is not our code base, so we’re facing that regression risk I mentioned.) For example:

"Correct the spelling of the unrecognized token ‘Shader’ in member name ‘BaseGame.glowShader’."

Now, obviously Shader is a word that might appear quite frequently in an XNA application’s source code. There are 140 spelling related messages and 43 casing related messages in the log. At this point, we probably need to look into CustomDictionary.xml.

As a quick aside, I mentioned in the previous FxCop post, FxCop utilizes the spell checking engine from Microsoft Office to check spelling. It should be noted that the latest version of FxCop (1.35 at the time of this post,) is not compatible with Office 2007. I had to roll back to Office 2003 in order to get the spelling checker activated. That little limitation held back this post a few days.

CustomDictionary.xml identifies words and acronyms that are permissible in your code to FxCop. You may also specify words that might appear in the Office dictionary, but should not appear in your code (go watch “Pulp Fiction,” or use your imagination as to what those words might be.) A beginning of an XNA appropriate CustomDictionary.xml is shown below:

<?xml version="1.0" encoding="utf-8" ?><Dictionary>
 <Words>
  <Unrecognized />
  <Recognized>
   <Word>xna</Word>
   <Word>fxcop</Word>
   <Word>shader</Word>
   <Word>shaders</Word>
   <Word>specular</Word>
   <Word>multi</Word>
   <Word>fullscreen</Word>
  </Recognized>
  <Deprecated />
 </Words>
 <Acronyms>
  <CasingExceptions>
   <Acronym>XNA</Acronym>
  </CasingExceptions>
 </Acronyms>
</Dictionary>

You may download this file here.

<Recognized> contains a list of <Word>s that will not be in the dictionary, but are OK in the code. <Unrecognized> would also contain a list of <Word>s that exist in the dictionary, but are not permitted in the code. <Deprecated>, instead of <Word> elements, contains <Term> elements. An example <Term> is:

<Term PreferredAlternate=”EnterpriseServices”>ComPlus</Term>

Notice that <Term> calls out an alternative.

Finally, the <CasingExceptions> contains a list of <Acronym>s that are exempt from the usual .NET framework guideline casing rules.

You may simply drop CustomDictionary.xml into you FxCop install directory, or you can keep it with what’s called an “FxCop Project File.” Just like Visual C# 2005 Express keeps all of the various setting related to your project together, FxCop can do the same. If we are building multiple assemblies, we can add all of our assemblies to the FxCop project file so that they are all analyzed together.  There are many other settings that can be saved on a per-project scope. The feature we are concerned with now is that if we save our CustomDictionary.xml in the same directory as our project file, then FxCop will load it. Then, both files can be saved to source control, or backed up, or whatever.

If you do decide to go with project files, you’ll probably want to update the custom Tool menu item in Visual C# 2005 Express from the last post. Previously we had “$(TargetPath)” in the Arguments text box. Instead we might want to use a combination of $(ProjectDir), $(SolutionDir), $(TargetName), or simply hard-coding a file name such as $(SolutionDir)FxCop\Project.FxCop. You may also want to add both the Debug and Release targets to the project file. Just make sure you’re working with the right configuration!

Once we have set up our CustomDictionary.xml, we can re-run the analysis on Rocket Commander. (Down to 649!) In the results we see there are still many spelling related messages. Messages like this are very common:

"In method SpaceCamera.GetRotatedMovementVector(Vector3):Vector3, correct the spelling of the unrecognized token ‘rel’ in parameter name ‘relVector’ or strip it entirely if it represents any sort of Hungarian notation."

As well as this:

"In method ColorHelper.FromArgb(Byte, Byte, Byte):Color, consider providing a more meaningful name than the one-letter parameter name ‘g’."

In the first case, I believe the ‘rel’ is just a short-hand ‘relative’ and clearly in the second case, the ‘g’ means ‘green.’ Now, if I were actually working on the code, I might prefer the slightly longer but slightly more readable names for maintainability. The Visual C# 2005 Express “Refactor” feature is great for this sort of work, incidentally. It may not amortize all of the risk of changing method parameter names, but it certainly eliminates most of the risk.

Well, originally I was going to try to handle a bunch of more content in this post. I’m thinking this one is long enough. In the next couple posts in the FxCop saga, we’ll look at automatic build integration, the various other ways of handling messages without touching your code (much,) and maybe I’ll even see if we can cough up a rule to flag the foreach construct that XNA developers seem to want to avoid like the plague.

OE

Useful links:

• FxCop help page for the IdentifiersShouldBeSpelledCorrectly rule.
• FxCop help page for the UsePreferredTerms rule.
• A blog entry with a little of the frustration you may face when trying to fix spelling in your code for FxCop.

Setbacks Galore

The second post on the ole blog was supposed to be all about FxCop’s spell checking facility. It was also supposed to be posted last week. Why the delay? I was doublely foo-bar’d:

• My notebook’s ancient ATI Radeon Mobility 7500C doesn’t meet the XNA minimum bar. Thus, I can only run XNA apps on my desktop box, which I was away from all weekend. (Someone wrote an app to check the minimum specs on a system for XNA. I remember seeing it in a blog post somewhere. Can I get a pointer to that? Thanks.)
• Even though I couldn’t run XNA apps on my laptop, I could analyze them with FxCop. Much to my dismay, I found that my desktop machine was generating different messages than my laptop. I’ve uploaded the report xml files from both machines here. I’ve also started a thread on the MSDN forums looking for some answers.

On the plus side, I am writing some code. After I get this FxCop stuff sorted out, I think I’m going to switch gears for a little XNA style “Hello World.”

OE

Static Code Analysis for XNA Applications

This tutorial assumes familiarity with the .NET Framework, C#, Visual C# 2005 Express Edition, and XNA.

There are many tools for making better software. One of the more recent innovations in software development tools is the idea of static code analysis. A program examines the source or object code of another program looking for potential problems. These potential problems are not syntax errors; compilers can catch those. Instead, static code analysis tools look for patterns of usage, defined by rules. Once a code pattern matching a rule has been located, it is flagged. At that point the software developer has a chance to evaluate the potential problem and fix it or leave a special mark in the source code that identifies to the reader (as well as the static analysis tool) that the potential problem is not a concern.

Notice that I explicitly stated that static code analysis is used to find “potential problems.” In truth, the tool only matches patterns of code to rules. The rules are formulated by experts, and while they can help identify problems from security issues to code style issues, they are not perfect.

For .NET programming, a team at Microsoft has developed and released, for free, a static code analysis tool called FxCop. FxCop’s rule set is based on Microsoft’s .NET Class Library Design Guidelines. These guidelines are available in the book “Framework Design Guidelines: Conventions, Idioms, and Patterns for Reusable .NET Libraries” by Krzysztof Cwalina and Brad Abrams.” There is also an excerpt of the book available online at MSDN.

The FxCop team maintains a website, where you can download the tool, as well as read the documentation, post in a user’s forum, and read the developer’s weblog. Note: if you are reading this after July of 2007, it is probable that the FxCop team website has been moved. Hopefully I have been back to update the link, but if not a quick search should dig it up. I would anticipate it’s on MSDN somewhere.

Now, you should download and install the software. Also note that some of the rules that FxCop is checking are for spelling. In order to enforce those rules, FxCop will need to have a spell checking component installed from Microsoft Office. It is probably sufficient to only install Microsoft Word. It will be necessary on all computers that are going to run FxCop, including build machines.

Now that we have FxCop installed, let’s put it through its paces. For now, we are going to start with the simplest XNA application possible.

Start Visual C# 2005 Express Edition. Go to the “Tools” menu and select “External Tools…” This brings up the following dialog box.

You should click “Add” and then fill out the text boxes as I have. The clipped out portion of the Command text box is “fxcop.exe” Unfortunately, we have to hard-code the path to the tool. Once you’re done, click “OK”.

Now, we need a program to analyze. Create a new Project (I called mine FxCop Test,) and select Windows Game as its template. Now, build the program. FxCop examines compiled assemblies code using the metadata stored in the assembly. It does not matter if you build “Debug” or “Release.” FxCop works for both. Once the build is done, go to the “Tools” menu and start FxCop. Once you do, you will see a window like this.

You will see in the “Targets” tab your assembly’s exe file name. From this tree view, you can browse through the members of the assembly. At each point, clicking on the tree view node will bring up a list of the properties of that member in the properties window. As many FxCop messages deal with the assembly metadata, it is convenient to be able to easily browse it.

Go ahead and click “Analyze.” You will need to help FxCop resolve the reference to Microsoft.Xna.Framework.Game. Just browse to the References directory in your XNA install to resolve it. Once you do, you will see some messages generated. Even this most simple of XNA applications seems to have some potential problems! Here are the messages.

You’ll see columns in the messages panel identify which rule has been broken, as well as “Certainty,” “Fix Category,” and “Level.” These can all be used to try to make decisions about which rules violations should be looked at first, or if a violation should be examined at all. For now, because there are only 4 messages, we’ll fix them all.

The bottom message is the most benign, and it’s also the easiest to fix. It’s simply reminding you that there is a parameter to a method that is not being used. In this case, it’s the auto-generated “args” parameter to Main. Fixing this message is as simple as just removing the parameter from the method.

The top message is perplexing. If you click on the message, in the properties window, you’ll see some information about the rule that has been violated. The “Info” and “Help” properties are especially useful. In FxCop 1.35, the default links in the Help URLs are broken, but are redirected to the proper documents on MSDN.

Reading more, we learn that Microsoft recommends that assemblies be given a “strong name.” We can resolve this problem through Visual C#. Go to the property page for your project, and select the “Signing” tab. Check the “Sign the assembly” check box, and in the “Choose a strong name key file:” combo box, select “…” In the resulting dialog box, type in a name for the new key pair file you will be creating. Usually, these files have a .pfx extension; Personal inFormation eXchange. Make sure you choose a password as well to protect the key pair file. Click OK, and a new file will be created and added to your project. Now, re-build. Your assembly is now strongly named. If you return to the FxCop GUI and click “Analyze” again, you will see only one message left.

Now, what exactly did we just do? .NET security is a very complicated topic, and strong naming alone is worth an entire blog post. However, in summary, we have created a public key encryption key pair. That pair is stored in the pfx file. Because a private key is stored in the file as well as the public key, it should be kept as secret as possible. That is why we have added password protection to the key file. (Don’t forget the password!) The assembly is signed as part of the build process, which is the naming. One of the benefits of this code signing is your assemblies are now more resilient to tampering. There are other benefits, and a few challenges. I’ll cover all of this in a later post. For now, I recommend using strong names. I do believe that you should not re-distribute the pfx file, however, if you are going to share your game’s code.

Our final message that we need to deal with is “Mark assemblies with CLSCompliant.” This message is referring to CLS, which is the Common Language Specification. A subset of Microsoft’s .NET has been submitted to various standards bodies. Part of the standard is the idea that code can be written and shared amongst multiple languages (the IL is shared, not the source!) The CLS enforces the rules necessary for that sharing to occur. Unfortunately, one of the CLS rules is that CLS compliant classes must only derive from other CLS compliant classes. As Microsoft.Xna.Framework.Game is not CLS compliant, then our game cannot be compliant either. You can indicate this to the complier by adding the following attribute to Program.cs.


using System;
[assembly: CLSCompliant(false)]
namespace FxCopTest


Originally, I was planning on adding this attribute to AssemblyInfo.cs (it seemed the logical choice.) However, it appears the CLSCompliant attribute must be attached to namespaces (despite no reference to namespaces in the FxCop message.) If anyone can explain this, let me know.

At this point, if we rebuild and re-analyze the code, we will see no messages. We are FxCop clean. By introducing FxCop right from the get-go in our development process, we can handle the messages as they come. In my follow up post, we will see that in a source code base of only moderate size, the messages can pile up fast! We’ll also take a look at the mechanism for marking code as safe without fixing the message, as well as the command line FxCop and integration with Visual C# 2005 Express Edition’s build process.

OE

Other links of interest:

• FxCop, Your .NET Cop Another FxCop basic intro, this article covers the GUI in more details, and includes some helpful information on the command line tool.
• FxCop In Depth A transcript of a talk given by Michael Murray and Jeffery Van Gogh, FxCop developers. It includes some insider background on how FxCop came to be, how it works, and how the rules work.
• Strong-Named Assemblies One of the several partial treatments of Strong-Named Assemblies on MSDN.
• Common Language Specification CLS documentation on MSDN.