DataSets in SOA

Last November, I engaged Udi Dahan (the Software Simplist) on using DataSets in OLTP situations. It turns out that Udi uses a question I posed that kind of preceded this dialogue for his latest podcast at Dr. Dobbs Journal titled DataSets and Web Services. Yes, I am the Jacob he mentions.

You can tell that he's not a great fan of DataSets in general, though he takes pains to treat them fairly. Which puts him ahead of most of the developers I track, really.

Row State

One of the things that Udi doesn't like about DataSets is that they track row state. Unfortunately, he seems to be wrapping together two different consequences of doing so and it's a little bit confusing. His point actually makes more sense if you break it apart a bit.

The first thing he brings up is that because a DataSet tracks row state, many developers build that tracking into the web service as an unstated expectation. I've seen this, too, and it is hideous. In the most extreme cases, you'll get a web service for each of the CRUD operations for your objects—essentially making your web service a thin proxy for your data layer.

The real problem with using row state, however, is that it muddies the contract for your services. This is where Udi is talking about "passing along the intent." Services should have an explicit intent for the data that they receive (or return). Without an explicit intent, it can be hard to discern what any specific service is supposed to do and therefore what business logic that service will enforce. The narrower you can define your intent the better. If your service only knows that a customer record has changed, that service doesn't know if it's supposed to change the address, the name, the marital status, or what all. Which means that the service will not only have to handle all business logic associated with a customer record every time, it also has to try to reconstruct what has happened if it wants to have any kind of concurrency at all.

Incidentally (and more importantly, I think), this also means that those using that service don't know what all it is supposed to do or what criteria it uses to do it. Developers don't deal well with uncertainty. Before you know it, you'll see business logic creep into the calling code "just to be sure." Or worse, you'll see new services pop up duplicating pieces of the larger service.

The problem with row state isn't deterministically bad—just because it's easier to screw up by relying on that additional information doesn't mean you have to screw it up. In other words, just because DataSets track row state doesn't mean you have to give row state significance in your services.

Too Much Information

Row State is really just a specific example of a wider problem, though—namely that DataSets encode a lot of details beyond the data that your service needs to do its stuff. These details make DataSets unsuited for including as a part of an SOA contract because alterations in those implementation details will break a service contract even when the data structures remain the same.

Udi's conclusion is that DataSets should never be used in SOA. I'm forced to agree with him for this last reason. Versioning objects across an enterprise is a tricky business. Minimizing the number of things that break downstream stuff when changed is important to good architecture. The more that you can isolate, the happier you will be.

So the key question for me becomes: are you actually implementing SOA or just using web services? If you're simply using web services, DataSets have limitations to watch for but can be useful. If you're going SOA, keep DataSets to yourself.

I can live with that.

28. March 2007 20:36 by Jacob | Comments (0) | Permalink

Programmer Personality Types

Well, it turns out that, like Simone, I am a DHSB programmer. I can't say that I'm surprised. The only part I'm a little leery of is that S. It just sounds so unfriendly to be a Solo programmer. I've worked in some excellent teams in the past, but coordinating with others is something I had to consciously learn to be good at.



Here's the breakdown:

  • Doer
  • High Level
  • Solo
  • liBeral
Technorati tags: , ,
24. March 2007 04:50 by Jacob | Comments (0) | Permalink

Arguing Data

People have a lot of different reasons for posting blog entries. These reasons vary from financial, to personal, to professional, to I'm afraid to know more. For me, one reason I take the time when I could be doing something else is that I like to put my ideas out there to be tested. I don't really care if a majority of people agree with me so much as I want to see what other people have to say for or against certain things. The downside to this is that I'll sometimes find that an idea isn't as good as I had originally thought it was. The upside is the opportunity to refine something to be better or to discard an idea that turns out simply to be bad.

Which is why I'm glad to see Karl Seguin's response to a post I had made about DataSets. Karl's a bright guy and he has a good background in the problem domain associated with DataSet objects. He displays class, too, even when he feels I've been a bit rough in a point or two.

The School of Hard Knocks

I empathize with his experience where DataSet misuse caused much pain and suffering. I've been in similar situations and it's no fun. In a full-blown business transaction environment, DataSets have some liabilities that make them ill-suited for business-layer usage. The thing is, the opposite problem exists as well, and it's one that is more serious than people want to give it credit for: a layer of specialized, hand-crafted business objects that don't actually do anything.

I'm currently working at a place that has an extreme case of this problem. We have four entirely separate ASP.Net applications for our internal invoice processing. All four of these applications have their own set of substantially similar custom objects that are completely unique for that application. Each object doesn't do anything more than contain a group of properties that are populated from a database and write changes back to it.

I shudder to think how many hours were wasted on this travesty. It's over-complex, can't leverage any type of automated binding, doesn't track row state, and testing and debugging changes is an unmitigated pain. It's like someone attended an n-tier lecture somewhere and never bothered understanding what the point of having one actually was. Frankly, I'd prefer if the previous developers had simply put all the data access right in each individual page--at least that'd be easier to fix when something blew up.

Learning Your Craft

The thing is, my experience no more proves custom business objects wrong than Karl's experience proves DataSets wrong. That's the trouble with anecdotal experience: it feels more important than it is (it doesn't help that pain is such an efficient teacher).

The trick of learning a craft is in gaining experience that is both specific and broad. This can be tricky in a field that is as immense as software development. You really have no choice but to specialize at some point. Even narrowing it down to ".Net Framework" isn't nearly enough to constitute adequate focus for competence.

Unfortunately, Karl's point that there are a lot of lazy programmers out there is true. Anyone who has had to hire or manage programmers will confirm this. Too many developers don't bother learning enough of their craft to be considered actually competent. Faced with the need to specialize carefully, many simply give up and learn only enough to get by (and sometimes not even that much). They're content to learn the bare minimum needed to get hired. They'll learn enough of the "how" to create a program without ever bothering to learn any of the "why".

Teaching Others

I have a minor problem with Karl's explanation, though. He says, "I advocate against the use of DataSets as a counterbalance to people who blindly use them." While I understand this position, I'm not sure I can be said to appreciate it. It smacks a little of the "for your own good" school of learning; which works well enough in a parent-child or even teacher-student relationship. I'm not sure it works so well in public or general discourse.

It is hard to correct bad habits, particularly habits as widespread as DataSet misuse seems to be. As one who often has the bad habits to be corrected, though, I think that I'd prefer having the problem explained and given the context so I can understand the trade-offs being made. That would give me the opportunity to know why something is wrong, not just that something is wrong.

That'd require discussing DataSets in specific instead of general terms. I'm not sure if Karl would really want to do that, though. I mean, his specialty at CodeBetter is really ASP.Net. Expecting him to tackle ADO.Net is not just unrealistic, it could have the effect of diluting his blog posts and alienating his regular readers or getting him embroiled in things he's less interested in.

I would like to see someone respectable and wider-read than I am take on Strongly-typed DataSets in a more complete fashion, though.

Professor Microsoft

Which is why I have to agree with Karl that the blame for DataSet misuse lies squarely in Microsoft's court. I stopped counting how many official articles and examples from Microsoft included egregious misuse or abuse of DataSets. And I have yet to see any that describe how to do it right or what kinds of things to look for in determining the trade-offs between a Strongly-typed DataSet and a more formal OR/M solution, let alone ameliorating factors for each. The only articles about DataSets that I can remember that don't actually teach bad habits are articles about how bad they are. Which isn't helpful. It'd be nice to have something, somewhere that talks about using them wisely and what their strengths actually are. Maybe that should be a future blog post here...

26. February 2007 18:33 by Jacob | Comments (0) | Permalink

Out-Cleverring Yourself

Have you ever hacked a product to do something it wasn't intended to do in order to "simplify" things for your users and have that blow up in your face? This is an account of my experiences doing just that with MS Reporting Services.

If you've used Reporting Services at all, you'll know that there are two virtual directories that are created on IIS when you first install it to a server: ReportServer actually serves up the reports by passing the requested data to external applications via whatever protocol you have configured and Reports (aka ReportManager) which serves as a user interface for reports on the server.

ReportManager is, by far, the most visible of the two. Using ReportManager, you can organize your reports and data sources and set permissions on who can view and change them. Often, this is enough of a user interface that it is a viable deployment mechanism to simply point your users to the Url for a report under the ReportManager directory.

Ingenious people, like myself, will often designate a new Web instance on IIS just for Reporting Services, generally naming it something clever like "reports". When you do this, it is very tempting to point the root directory of that web instance to the ReportManager directory. This means that you can point your users to "http://reports/Invoice" instead of "http://reports/Reports/Invoice". You can see why you'd want to make this change (assuming you are similarly obsessive).

There is, however, an unintended side-effect to this change. Once you do this, your ReportServer will begin throwing errors if you ever decide you want to use direct Url access to display reports. Not a lot of people do this unless they're using a Reporting Services "Integration" that uses this functionality for showing reports. Personally, I ran into this situation when trying to tie our Reporting Services forms into Great Plains. Since Great Plains integrations (both native and third party) expect to use ReportServer for the report display, I was shocked and dismayed to find myself staring at this error:

The type Microsoft.ReportingServices.UI.WebControlConnection, ReportingServicesWebUserInterface, Version=, Culture=neutral, PublicKeyToken=89845dcd8080cc91 does not implement IReportServerConnection or could not be found

Hitting Google for this error is more than a little bit frustrating because the vast majority of issues where it shows up is in mis-configuring a ReportViewer object in an ASP.Net web application. This is manifestly not the case here.

This was a dome-scratcher for a lot longer than I like to admit. All of the advice pieces that I could find for this error are simply inapplicable. Fortunately, I ran across a Usenet post by Dave Green from last July. Dave reported that he could fix ReportServer if he removed a couple of settings from the <appSettings> section of the ReportManager. This is most odd because the two simply shouldn't be linked--at least, in that direction. If they were linked, I'd expect that link to go the other way (i.e. something in ReportServer breaking ReportManager). After all, ReportManager is essentially a UI on top of ReportServer.

I tested his findings and sure enough, removing the appSettings fixed ReportServer for me. Since removing those settings broke ReportManager, this was an unappetizing solution.

It was then that I remembered that IIS does an interesting little trick with web.config files and subdirectories. You see, each subdirectory on a website inherits the configuration from parent directories (also machine.config, but that's not relevant to our story). It doesn't matter how those directories are physically arranged, what is important is that IIS uses the Url to determine inheritance. So in a situation where "http://reports" is pointing at ReportManager and "http://reports/ReportServer" is pointing to ReportServer, that means that ReportServer is inheriting the web.config settings from ReportManager. Thus, the appSettings for ReportManager are being read by ReportServer and misapplied (I've no idea why the presence of an appSettings entry would break ReportServer)

Fortunately, the fix for this issue is pretty simple. The <appSettings> element includes a spec for a clear directive that will wipe out all inherited values. Adding an appropriate entry to the web.config in ReportServer cleared the problem right up. Here's the section for the curious (and/or lazy):

    <clear />

So there you go. I hoisted myself by my own petard. Fortunately, I was able to figure out how to get myself unhoisted. I now have both ReportManager and ReportServer humming along nicely without having to undo my clever directory mapping. Since I haven't been able to find this solution anywhere, I figured I'd post this for those in similar situations.

21. February 2007 21:47 by Jacob | Comments (0) | Permalink

Minor Updates

Okay, things seem to have settled down with the blog app. I love when that happens. It should stay stable until I decide to hack around with it again. One of the more interesting changes I made is that I'm now using Bloglines' public API to display my blogroll. This means that the blogs listed on the right, there, are the same as the ones I monitor on a regular basis. When I add a feed, it'll show up (and conversely when I remove one...)

In other news, two of my customizations to the SubText engine have been approved for the standard code base. This is kind of cool to me because I'm such a geek. Not that I did anything great. My first was to change the sort on the referrers list in Admin. My second was a change that allows me to list a content category as a list of entry links. I needed that second for the "About Me" section on the side. Small things, yes, but still gratifying.

Technorati tags: , ,
26. January 2007 12:19 by Jacob | Comments (0) | Permalink

Ways to Skin a Blog

The folks who visit the actual blog (as opposed to using an RSS feed) have probably noticed that I re-skinned the site. Now that I'm more comfortable with SubText's skin structure, I figured this would be the next step. Personally, I think that it looks tons better now, but you can let me know in the comments if I'm just being delusional. I still need to tackle that calandar, though.

I can't take much credit for it if you like it. I'm perfectly aware of my (lack of) design sense. I can generally do okay evaluating existing designs, but I suck very, very much in creating my own. I found a design that I liked on Open Source Web Design and picked it apart enough to apply it to my SubText skin. Sweet that folks'll share like that.

Technorati tags: , , ,
12. January 2007 21:15 by Jacob | Comments (0) | Permalink

Building Custom SubText Controls

In my response to Jeff Atwood, I mentioned the Technorati widget control that he is now using in lieu of Trackbacks. The Technorati widget is fairly simple to implement, but you have to be able to feed it the permalink for your post for it to be able to work. Since I already had a go at creating custom controls for use in SubText, I decided that now was a good time to see if I couldn't come up with a way to create one that has access to blog entry attributes like the permalink. For your edification, here's what I found.

First Things First

Before we walk this primrose path, lets make sure that we aren't setting ourselves up for hard times later. Create a custom Skin by copying the files from your favorite built-in Skin into a new folder in the Skins directory.

Once you do this, you'll need to let SubText know about your new skin. This is done in the Admin directory by creating a "Skins.User.config" file. The easiest way to do that is to copy the existing Skins.config file and rename your copy to "Skins.User.config". They have the same structure, but having the extra config for user skins allows the SubText team to maintain the built-in skins without messing up any customizations you might be using. Make sure that you delete all the entries for skins (except the one you copied from above). Then you can tweak the template Name and and TemplateFolder attributes (highlighted below).

Once this is done and uploaded to your SubText site, you'll see your skin listed at the bottom of the skin selection drop-down in the Admin pages (that dropdown is not sorted, so the user skins will be at the bottom). Now you can modify to your heart's content and not worry about a future upgrade nailing your customization. Also, if you really mess things up, just hit Admin and set your skin back to your old starting point.

Where Things Are

The controls you'll use to customize SubText are all in the Controls folder in your custom Skin directory.

The most daunting thing about creating custom controls is figuring out how SubText handles things. You can skip the rest of this section if you don't really care and just want to get on with creating your control.

There are a couple of quirks of SubText that can help you isolate any problems that you might encounter.

  1. SubText transfers some of the controls programmatically by grabbing subcontrols of one object and copying them to the controls collection of the parent object. This is most notable for the subcontrol brought in by the PageTemplate for the "contentregion". If you pick apart your PageTemplate.ascx, you'll see a control with an ID of "MPMain". That's where this happens. The reason for this is that SubText replaces MPMain with a different control depending on what kind of page it is trying to display. Which leads to the following problem.
  2. It isn't immediately clear which control is being pulled in by MPMain for different sections of your blog. The main page (default.aspx), for example, pulls the HomePage.ascx control. It took me a while to figure this out, but now that I have, I found that the control that functions as "MPMain" is determined by a group of <HttpHandler> elements in the web.config file (technically, MPMain can be replaced with multiple controls as you can see if you actually dig out your web.config). The HttpHandler elements in web.config describe a RegEx "pattern" attribute to apply to incoming URL requests and then a "controls" attribute that describes the control(s) used. I could have saved a lot of time if I had known this up front (say by asking members of the dev team who are actually quite helpful and would probably have responded pretty quickly, really)
  3. The classes that implement many of these controls use the specific ID of some controls to populate certain attributes--mostly Text. For example, Day.ascx can have a Literal control with an ID of "PostDesc" that is populated with the descriptive elements of a post at run time (e.g. when it was posted, if there are comments, that kind of thing). Some of these are checked for existence (like "PostDesc"), but some aren't (like "PostText"). Which are checked and which aren't makes pretty good sense, though, so I don't see that as a problem. I mean, I want to see an error if my blog is unable to display the PostText of a post.

Creating a New Control

The key feature that you need in order to leverage SubText entry information is a control class defined in SubText called CurrentEntryControl. This nifty little guy has no implementation in any of the default skins, so he's easy to overlook. The key feature he possesses is that if referenced from a control that has a blog entry associated to it, it'll let you use databinding syntax within that control to access the properties of an entry. I'm not going to say how long it took me to figure this out. Let's just be happy that I have...

The Technorati embed needs your fully qualified URL (i.e. the full-on web address to reach your post--the one you'd paste into a browser's address bar to reach just that post), so the best databind expression that I found for my purposes is:

<%# UrlEncode (Entry.FullyQualifiedUrl) %>

Believe it or not, this actually works. UrlEncode is defined in the CurrentEntryControl object along with the Entry property. Again, don't ask how long it took me to figure all this out. Just for kicks, here's the Entry class properties:

Since I'm going to want the Technorati embed to function in a couple of different page contexts, I think the easiest way to implement it is to put the script in its own control. This will be pretty simple for just the Technorati embed script:

<%@ Control Language="c#" Inherits="Subtext.Web.UI.Controls.CurrentEntryControl" %>
<script src="" type="text/javascript"></script>
<a class="tr-linkcount" href="<%# UrlEncode (Entry.FullyQualifiedUrl) %>">View blog reactions</a>

That should be generic enough to function in any blog. I've put it together for you to download here if you want to save yourself some copy and pasting.

Placing The New Control

Now that we have the control we're going to use for Technorati's embed, you have to decide where you're going to use it. For me, I decided to use it both for posts on the blog's home page and for the individual posts themselves. After digging around, I decided that the best candidates for the new control were Day.ascx and ViewPost.ascx. Day is eventually referenced from the home page and ViewPost is referenced when displaying a post from archive (i.e. permalinked pages).

For the control to work, you'll need two things: a Register tag and the control tag. The Register tag lets the parent control know where to find your custom control. The control tag lets it know where to display it. My new ViewPost.ascx looks like this (my changes are highlighted):

Save those changes, upload to your server, and you're done. Just because I can, I included both of the modified controls in the download I referenced above. Pop all three into the Controls folder of your custom Skin and you should be good to go. Note that I pulled both of those controls out of the "Piyo" built-in skin, so it's very possible that they may differ from the controls in other skins.

22. December 2006 05:30 by Jacob | Comments (2) | Permalink

Trackbacks Are Dead


Jeff Atwood has a recent post on why he finally gave up and disabled Trackbacks on his blog. My blog is the tiniest fraction of his and I had to disable trackbacks just for sheer spam volume back in October (inspiring an anti-spam rant of my own).

Jeff lays the blame for Trackbacks' demise on Six Apart--the outfit that created the standard in 2002. Ah, those heady glory days when you still had to explain to people what a blog was. Trackbacks were a great idea. They still are a great idea. But Jeff is right, the simplicity of the standard has left it wide open to abuse and that abuse has killed them dead.

So my question is (to Jeff or anyone else), how would you alter the design to make the standard more robust?

My initial take on it was to alter the standard to incorporate a public key exchange and a signature. But then I realized that, hey, spammers can create asymmetric keys as well as anyone else can. In other words, the problem isn't being able to authenticate the link--it's being able to evaluate the linked post.

Jeff's current stop-gap of finding links to his posts through Technorati seems like a reasonable short-term solution. Introducing a third party is problematic, though, because it leads to inevitable issues in finding a trustworthy third party that will carry the authentication burden for you (as well as traffic and processing costs as people ping them for link verification). Indeed, Akismet (a popular stab at trackback filtering) has those same third-party screening issues and isn't substantively different from Jeff's use of Technorati.

I suspect that the problem might not even be in bad design by Six Apart, however. The thing that makes Trackbacks so popular and led them to be so widely adopted is that it allows the creation of inter-post linkages from unaffiliated sources with very little effort. I'm afraid that any solution to the trackback problem is going to necessarily involve increasing the effort of unaffiliated linking to a point where it becomes much less attractive.

A Stab in the Dark

That said, here's two thoughts about potentially rewarding avenues for solving the problem picked up from spam solutions in other realms.

First from email spam, and recognizing that I'm pretty thoroughly ignorant of the underlying mechanisms involved in Bayesian content analysis, I wonder if there might be some useful application for content analysis here. Spammers are increasingly sophisticated in overcoming content analysis, though. Trackbacks may be easier to analyze, though, because they have an easily available comparison text (the originating post). It may be easier to compare your post with that of the linker and come up with a tougher analysis than you can in, say, a lone email. I don't know about that.

Second, the key to the success of spamming is that they have such a very low cost per "signal" (email, comment, trackback, what have you). Their only incremental cost is bandwidth to find blog posts and to send trackback signals. Raising those costs can have a significant effect on spam. This is essentially the key to Captcha's success in curbing comment spam. If a trackback request prompted a user-interactive Captcha-like query, that alone may well be enough to stem the vast majority of trackback spam. Perhaps a design amendment that included a short interaction on a trackback ping would be successful in cutting spam back to manageable levels.

In looking at those two ideas, it occurs to me that the main problem with a Bayesian solution is that it places the burden (both in implementing the Bayesian algorithms and in processing the incoming links) squarely on the target of the spam. This can lead to an unwanted side-effect by leaving your blog much more open to another Internet dirty trick--denial of service attacks. Frankly, you don't even have to deny service to affect a lot of private bloggers--attacks that increase their bandwidth usage would be as unwelcome to many as a full-on denial might be. After all, it doesn't cost you extra hosting fees when your blog goes down.

So maybe that means I only really have one thought/solution/suggestion. Bayesian analysis would be cool for the AI geeks, but not terribly practical in the constrained environment confronting most bloggers. I wonder what it'd take to create a Captcha mechanism in trackback notification?

21. December 2006 11:59 by Jacob | Comments (2) | Permalink

Hacking SubText

Okay, I've had SubText up and running for a week and some now, so naturally it is time to tinker. In poking around, I'm not as happy with SubText's integration model as I was with DasBlog's. DasBlog exposed a number of "macros" that you could use to insert certain internal values into your own stuff. I kind of liked their model because once you registered your own code, you could reference your macros from DasBlog stuff as well (bear in mind that I didn't actually implement DasBlog, so my impression could be off).

That said, as long as you don't need to access things like entry links or other internal blog workings, it's pretty easy to modify a "Skin" in SubText to add new stuff. SubText reads a main ASP.Net Web User Control called PageTemplate.ascx that holds essentially the format for the other Web User Controls that a SubText site uses. Which subcontrols are used are pretty much determined by this PageTemplate control and the controls it references. This structure made it pretty easy for me to add a new embed control I wanted to play with.

The Technorati Embed

There are a number of sites that offer badges, embeds, and other tools intended for their users to simultaneously spruce-up their own sites and provide free advertising for the service involved. The Technorati search feature on the right is an example and was my first foray into modifying my blog template. With Technorati, the tag I need to use to expose a Technorati search link is this:
<script src="" type="text/javascript"></script>

To pop it on there, I just added that code right in the PageTemplate.ascx file where I thought it'd do some good (using a text editor). Straightforward, but inelegant.

The Great Games Experiment Embed

 For my next trick, I wanted to see how SubText would handle it if I mimicked their own structure to create a new control. To do this, I created a couple of things that are way more complicated than they needed to be. I mean, technically, all I needed to do to pop the user embed onto my site is copy the badge tag from their site in the same place I put the Technorati stuff. The tag isn't all that complicated:
<iframe src="" height="140" width="204" scrolling="no" frameBorder="0" />

Doing so would produce this handy "badge":

But I'm a geek and I wanted to test some stuff out, so instead of simply pasting a bunch of html, I created a .Net project and two custom web user controls. One of the controls creates a user badge. The other control creates a game badge. The result of this project is that you can drag one of these controls onto an ASP.Net page and set the "UserName" property (or "Game" property for a game badge) and it'll take care of the rest. The active html would look like this:
<cc1:GreatGamesUserEmbed ID="GreatGamesUserEmbed1" runat="server" UserName="Jacob" />

Somehow, I managed to get that to work. As long as you've registered my assembly for the right tag prefix, you're golden.

But that wasn't quite enough for me, oh, no. Simply adding that control to PageTemplate would be kind of hokey and out of place, so I went one further. Following the pattern of the other controls, I created a GreatGames.ascx file under the Controls directory and referenced that instead. It's a simple file, but then, most of those Control files are. It creates a section that fits in much better on the right side bar and groups the Great Games badges together more naturally.

And wonder of wonders, it worked first time. Love when that happens.


SubText doesn't have any documentation that I could find detailing these steps, so making these modifications required mucking about in the code to follow their flow during page delivery. If I get the gumption later, I'll re-write this as a more generic how-to in creating very basic SubText add-ins because the community needs something like this. Also, I saw a couple of interesting objects that look like you can extend them for more complicated things (like controls that access the SubText data, I'll bet). I'm sure that Phil Haack and others could point those out. In fact, they're nice enough folk on their dev email list I'd bet they'd be happy to answer questions if I took the time to ask.

Also, if you want those Great Games controls, download the source code. It's nothing fancy, but does the job. Or, if you just want the binary, download that instead. The source code project includes my GreatGames.ascx file as well so you can see an example of implementing them. Let me know if you do anything interesting with them.

14. December 2006 18:34 by Jacob | Comments (1) | Permalink

Creating a Domain Publisher Cert for a Small Internal Software Shop

The trend towards increasing security introduces a number of intricacies for medium-sized business software shops using Active Directory Domains. An internal domain with more than a dozen workstations can introduce issues that are old hat for larger shops, but way beyond anything a small business will have to deal with. I ran into one such issue recently when I decided it'd be a cool thing for one of my apps to actually run from the network.

The Problem

The first sign I had a problem was when a module that worked fine locally threw a "System.Security.SecurityException" when run from a network share. It told me that I had a problem at "System.Security.CodeAccessPermission.Demand()" when requesting "System.Security.Permissions.EnvironmentPermission". Since it worked fine while local, I figured I had a code trust problem and that I could probably get around it in the .Net Framework Configuration settings and push a domain policy that would update everyone.

I knew this because I had run into something similar once before (deploying a VSTO solution on the network).

Here's where it pays to be a real (i.e. lazy) developer: since I've run into this before, wouldn't it be nice to come up with a solution that will make it easier when I run into this in future? There are four ways to do this, I figure (well, that I could think of, there are probably more).

  1. Create some kind of scripting solution for deploying future projects that automatically creates policies (and propagates them) for each new assembly.
  2. Create a standard directory on the network that can be marked as "trusted" and deploy any trusted code into that directory.
  3. Use a "developer" certificate as your trusted publisher.
  4. Figure out how to get a publisher cert to use to sign your code and then propagate a rule certifying that publisher as trusted.

Some developers would go with number 1. Which makes me shudder. Anyone using the first option isn't someone I want to code with or after (barring some quirky deployment requirement that makes it more attractive, of course). Number 2 would probably be the most common solution because it's pretty simple and most medium-sized businesses are used to security compromises that use "special knowledge" and a lack of being an attractive target for security trade-offs. Number 3 would be a little more "upper-crust", mainly from people who had tried 4 and run into difficulties. And frankly, for most cases Number 3 is likely adequate. The problem is that using number 4 has a couple of not insignificant hurdles.

The Issues with Certificates

There are a couple of obstacles in your way if you want to produce a valid publisher certificate for use in signing code.

  • For a smaller internal shop, going the "official" route of contacting one of the major certificate stores (Thawte, Verisign, et. al.) is overkill with a price tag.
  • Setting up a private Certificate Authority isn't that hard, but unless you're running Windows 2003, Enterprise Edition, you cannot customize certificate templates.
  • The settings on the Code Signing template marks the private key as "unexportable".

That last is the most significant problem. You see, if you cannot export your private key, you cannot export to a "pfx" file (aka "PKCS #12"). You could export a .cer file (public key only) and then convert that to an spc using cert2spc.exe but that leaves you with a file that pretty much anyone can use to sign code. There's a reason Visual Studio Help warns that cert2spc.exe is for testing only.

If I lost you in all the security acronyms, don't worry about it. The important thing to note is that a) non-pfx files don't need a password to use in signing assemblies and b) there's no easy way to create a non-developer created pfx file signed by your organization's CA.

How to Get Your CA to Issue an Exportable Certificate

There is, however, a loophole you can exploit to con your CA into giving you a Code Signing certificate that you can export into a valid .pfx file. I'll skip the setup stuff on the CA. It is important to make sure that your CA makes the Code Signing template available (it isn't by default). Making it available is pretty straightforward, so I won't go into that here.

The first thing you'll need to do is use makecert.exe to create both a private and public key. A basic commandline to do so would be:

makecert -n "CN=Text" -pe -b 12/01/2006 -e 12/01/2012 -sv c:\test.pvk c:\test.cer

You can hit the help file for other fields you might want to set (or use the -? and -! switches to get two lists of available options). This command will pop up a GUI prompt for your private key password. Note that I typoed "CN=Text". While I meant to make that "Test", it turns out to be a good way to illustrate what that value is so I decided to keep it in the following examples. Also note that "-pe" is what makes the private key exportable. After running this command, you'll have two files in your root directory. The pvk is the private key file and the .cer is the public key.

Next you use a Platform SDK tool called pvk2pfx.exe. This wasn't in my regular path so I had to do a search to find it. I'm guessing that most development machines will have it already. If not, it's available from Microsoft. Here's the command I needed:

"C:\Program Files\Microsoft SDKs\Windows\v6.0\Bin\pvk2pfx.exe" -pvk c:\test.pvk -spc c:\test.cer -pfx c:\test.pfx

Like makecert, this command will give you a password dialog for the private key. Note that even though the command switch is "spc", it'll accept a .cer file just fine. Now, you might think that we're done because we have a valid pfx file. The problem is that this pfx file is derived from a CA of "Root Agency". In order to get this into your internal CA, you're going to need to use your certificate manager. You'll likely need to Run certmgr.msc to get to it. Once there, head to the Personal|Certificates node. This will let you play with certificates on your current workstation.

Right-clicking on "Personal" gives you an "Import" option. Follow the prompts to pull your certificate in. It'll prompt you for the private key password. Once you do this, you'll see your new private key and probably an auto-imported "Root Agency".

Here's where we find the handy loophole. While the default value for allowing private key exporting on the Code Signing template is false, you can use your handy new certificate to request a duplicate. Right-click that key and select "Request Certificate with Same Key". You can also use "Renew Certificate with Same Key". The functional difference seems to me to be that Renew keeps your password while Request provides an empty one (which is nervous-making, but rectifiable using a number of different tools including Visual Studio once the certificate is exported).

In the Wizard that follows, make sure you select the Code Signing template. What you'll receive back is a certificate from your CA for code signing that includes a private key that is marked exportable. At this point, I delete both the "Root Agency" and "Text" certs in order to avoid future confusion.

Use the Right-click|Export command to export this certificate to a pfx file. The pfx file has everything you need to be able to create a .Net Framework code policy using "publisher" as the distinguishing characteristic to mark your code trusted. Once that policy is propagated to all the domain workstations, you're good to go. You'll need to use the resulting pfx file to sign the assemblies (once they're ready for release), but you knew that already :).

A Final Note

After I had a valid certificate for signing, I actually ended up using .Net's ClickOnce technology to deploy the project. I still needed a certificate to create a strong-named assembly, but a weaker or temp certificate would have been adequate for internal deployment. The more robust certificate will let me eliminate a security prompt the first time a user runs the application, though. Since that prompt has a big red exclamation point in it, I'm just as happy to eliminate it.

4. December 2006 22:33 by Jacob | Comments (1) | Permalink


<<  September 2017  >>

View posts in large calendar