29 November 2008

aspnet_compiler and missing .compiled files


This week, I've been facing a rather annoying issue. A developer told me that some global.asax events no longer fired after he precompiled his ASP.NET application using aspnet_compiler on his computer. He noticed that this behavior was due to a missing App_global_asax.dll.compiled file in the output bin directory of the precompiled application. Strangely, when he just published the web site using the Visual Studio publish web site command, everything went well. When aspnet_compiler was invoked through our NAnt build script, .compiled files were missing.

I spent a few hours on this one…I discovered that this behavior was reproducible on my computer too, but not consistently. Starting aspnet_compiler directly from a command prompt, or invoking MSBuild on the solution file containing the web site did sometimes produce the required .compiled files, but rarely. This was driving me crazy. How could a compiler not be consistent, "forget" about some files, without even producing any kind of errors? I monitored the entire compilation process using Process Monitor, and I did not notice any suspect. When everything went well, aspnet_compiler just created and copied the .compiled files, otherwise, it seemed to simply not even attempt to do it. No access denied whatsoever, not a single file system error…After a few hours, a colleague and I had a stupid idea (well, it appeared to us that it was not so stupid): we killed all the antivirus processes using task manager. And guess what? All of a sudden, aspnet_compiler systematically created the required .compiled files, at every build and with whatever method we used to launch it (VS publish, MSBuild, NAnt, command-line)…We did the same test on the developer's computer (which of course ran the same antivirus), and it worked like a charm.

Lesson learned: antivirus programs are not the developer's friends.

We already experienced some annoying AppDomain restarts while debugging ASP.NET applications, due to the antivirus touching web.config and other monitored files. And now, it interferes even with a compiler, and this without causing any actual build failure! I admit that I do not understand what really happens at the filesystem level: why do I not see anything in Process Monitor (including no antivirus activity on .compiled files)? Do these antivirus programs sit even lower in the OS architecture than Process Monitor so that they can "swallow" events? Maybe…Anyway, whenever I'll experience some strange behavior of my development tools in the future, one of the first thing I'll do is turn off the antivirus. And, by the way, I'm very happy that no AV is running on our build server, otherwise…

Last thing: our machines were running the CA eTrust antivirus. On my second laptop, where I have the free edition of AVG, I never encountered any issue of this kind.

11 comments:

Erik Ropez said...

I had same issue but with different reason - file system compression switched on.

RobD said...

I would never have found this without your blog post. Thank you so very much!!! I am eternally indebted.

Anonymous said...

I have had this problem for months on my portable development computer. Last night it happened on my workstation. I have spent 10 hours trying to resolve the issue. I found your post, un-installed McAfee and it works again. Thank you so much for the post.

Anonymous said...

Thanks for this page! I spent two days on this issue and on the second day noticed the missing .precompiled files. Searched for it and found your article and voila! You're a life saver! :)

Anonymous said...

Thank you. Thank you. Thank you. We just had the same exact issue and it was McAfee. Had our systems group disable McAfee as a quick test and everything worked great. We're currently trying to find which folders are needed as exceptions.

Anonymous said...

Had the issue of global.asax not getting compiled to App_global.asax.compiled/App_global.asax.dll and the cause was as for Erik Ropez, disk compression turned on.

Anonymous said...

I too have been fighting this problem. My disk is not compressed, and my company does not allow any changes to the virus scan settings (McAfee) so I'm not sure how I'm going to proceed, but thanks so much for giving me some hope, I was at wit's end.

Anonymous said...

Thanks Ydie you saved my a real headache. McAfee On-Access kept screwing with my precompilation task preventing creation of any .compiled files (on occasion it generated .deleted files).
Since my admin wasnt too happy about disabling AV on the build server I got him to add the (1) build folder and (2) "C:\Windows\Microsoft.NET\Framework" to scan exclusion and voila, it works.

Anonymous said...

Had the same issue as above. This blog saved me a lot of time. McAfee scanned all our files constantly while trying to pre-compile with aspnet_compiler. In the end we added 2 exclusions to our virus scan policies; 1. the build folder, 2. "C:\Windows\Microsoft.NET\Framework". The downside of this solution is that the build folder must abide to the policy. So if you have 5 devs all doing some pre-complication on their machines then all of them should have the same build (output) folder. Then again you should maybe consider build services then ;D Thanx Ydie and Anyonymous (the one above this post)

Anonymous said...

This post saved us too!! Thank you soo much. I spent hours looking why the .compiled files will not produce and the app domain has been unloaded but I could not find any information and found this post.

Anonymous said...

This post solved my issue on an Azure On Prem Build Agent running Kaspersky Antivirus! Thank you soo much!!!