29 November 2008

aspnet_compiler and missing .compiled files

This week, I've been facing a rather annoying issue. A developer told me that some global.asax events no longer fired after he precompiled his ASP.NET application using aspnet_compiler on his computer. He noticed that this behavior was due to a missing App_global_asax.dll.compiled file in the output bin directory of the precompiled application. Strangely, when he just published the web site using the Visual Studio publish web site command, everything went well. When aspnet_compiler was invoked through our NAnt build script, .compiled files were missing.

I spent a few hours on this one…I discovered that this behavior was reproducible on my computer too, but not consistently. Starting aspnet_compiler directly from a command prompt, or invoking MSBuild on the solution file containing the web site did sometimes produce the required .compiled files, but rarely. This was driving me crazy. How could a compiler not be consistent, "forget" about some files, without even producing any kind of errors? I monitored the entire compilation process using Process Monitor, and I did not notice any suspect. When everything went well, aspnet_compiler just created and copied the .compiled files, otherwise, it seemed to simply not even attempt to do it. No access denied whatsoever, not a single file system error…After a few hours, a colleague and I had a stupid idea (well, it appeared to us that it was not so stupid): we killed all the antivirus processes using task manager. And guess what? All of a sudden, aspnet_compiler systematically created the required .compiled files, at every build and with whatever method we used to launch it (VS publish, MSBuild, NAnt, command-line)…We did the same test on the developer's computer (which of course ran the same antivirus), and it worked like a charm.

Lesson learned: antivirus programs are not the developer's friends.

We already experienced some annoying AppDomain restarts while debugging ASP.NET applications, due to the antivirus touching web.config and other monitored files. And now, it interferes even with a compiler, and this without causing any actual build failure! I admit that I do not understand what really happens at the filesystem level: why do I not see anything in Process Monitor (including no antivirus activity on .compiled files)? Do these antivirus programs sit even lower in the OS architecture than Process Monitor so that they can "swallow" events? Maybe…Anyway, whenever I'll experience some strange behavior of my development tools in the future, one of the first thing I'll do is turn off the antivirus. And, by the way, I'm very happy that no AV is running on our build server, otherwise…

Last thing: our machines were running the CA eTrust antivirus. On my second laptop, where I have the free edition of AVG, I never encountered any issue of this kind.