Here's a small HTML page that illustrates the problem:
<html>
<body style="background: blue;">
<fieldset style="width: 500px;">
<form>
<legend>This is my form</legend>
<p>Enter some text: <input type="text"></p>
<p><input type="submit" /></p>
</form>
</body>
</html>
Here's how the page renders in IE 10. Note the wide, misaligned right border.
The border looks normal, however, in the other browsers I tested: Chrome 30.0.1599.69 m and Firefox 24.0.
I found a simple workaround, which is to explicitly set the border style for the fieldset tag:
<fieldset style="width: 500px; border: 1px solid white;">
This cleans up the border like so:
And it continues to look fine in Chrome and Firefox.
Of course, you could do the same thing in a CSS file if you prefer:
fieldset {
border: 1px solid white;
}