FIX: Supporting MBCS in Positional Flat Files (312616)



The information in this article applies to:

  • Microsoft BizTalk Server 2002
  • Microsoft BizTalk Adapter for SAP 1.0

This article was previously published under Q312616

SYMPTOMS

If you interchange a multi-byte character set (MBCS) flat file between BizTalk and other applications such as SAP, the document may be corrupted.

CAUSE

BizTalk Server 2002 does not support MBCS in positional flat files. MBCS refers to those character sets with a mixed one-byte and two-byte representation of characters.

BizTalk counts position in character units, but legacy applications usually use byte units to count position from the positional layout of a flat file. These two counts are different for an MBCS file.

RESOLUTION

A supported hotfix is now available from Microsoft, but it is only intended to correct the problem that is described in this article. Only apply it to systems that are experiencing this specific problem. This hotfix may receive additional testing. Therefore, if you are not severely affected by this problem, we recommend that you wait for the next service pack that contains this hotfix.

To resolve this problem immediately, contact Microsoft Product Support Services to obtain the fix. For a complete list of Microsoft Product Support Services phone numbers and information about support costs, visit the following Microsoft Web site:NOTE: In special cases, charges that are ordinarily incurred for support calls may be canceled if a Microsoft Support Professional determines that a specific update will resolve your problem. The typical support costs will apply to additional support questions and issues that do not qualify for the specific update in question.

The English version of this fix should have the following file attributes or later:
   Date          Time       Version           Size   File name
   ----------------------------------------------------------------------
   07-Feb-2002   12:12:03   3.0.1159.1     188,688   Cispositionalmbcsparser.dll
   07-Feb-2002   12:12:20   3.0.1159.1     180,496   Cispositionalmbcsserializer.dll
   07-Feb-2002   12:11:27   3.0.1159.1     172,304   Cismsg.dll
   30-Jan-2002   20:35:22   2.0.0.0      1,405,200   Mapedit.ocx
   30-Jan-2002   20:35:22   2.0.0.0        274,704   Treedatasvr.dll
				

STATUS

Microsoft has confirmed that this is a bug in the Microsoft products that are listed at the beginning of this article.

MORE INFORMATION

To apply the fix, follow these steps:
  1. Copy Cispositionalmbcsparser.dll to the \Program Files\Microsoft BizTalk Server folder.
  2. Copy Cispositionalmbcsserializer.dll to the \Program Files\Microsoft BizTalk Server folder.
  3. Copy Cismsg.dll to the \Program Files\Microsoft BizTalk Server folder to replace the existing version of this file.
  4. Copy Mapedit.ocx to the \Program Files\Microsoft BizTalk Server\XML Tools folder to replace the existing version of this file.
  5. Copy Treedatasvr.dll to the \Program Files\Microsoft BizTalk Server\XML Tools folder to replace the existing version of this file.
  6. For each of these five files, go to their corresponding folders and use the regsvr32 command to register these files.
  7. In the BizTalk Server Administration, click Group, click Properties, and then click the Parsers tab. Click Refresh, and then move BizTalk.ParserPositionalMBCS.1 to the top of the list.
For serialization, the Messaging Manager allows you to select the new MBCS serializer when you configure ports because it contains the implementation category for the serializer. To do this, follow these steps:
  1. In the Messaging Manager, open envelopes that you want to use on the MBCS serializer.
  2. Change the envelope format from FLATFILE to CUSTOM and save the envelope with a different name.
  3. Open ports that use these envelopes and click Next until you reach the Envelope Information page.
  4. Select the modified envelope and confirm that its envelope format is CUSTOM.
  5. Open the channels associated with the ports, and then click Next until you reach the Advanced Configuration page.
  6. Click Advanced, and then click the Envelope tab in the dialog box.
  7. Select BizTalk SerializerPostionalMBCS as the Serializer component.
This fix adds a positional MBCS parser and a positional MBCS serializer to the BizTalk Server 2002 system. They are designed to handle positional MBCS flat files. This fix also adds the Count Positions In Bytes property to the Parse tab of the BizTalk Editor. You need to set this property to Yes for MBCS flat files. If you do not set this property to Yes, the default flat file parser is used. The MBCS parser always has to be listed before the default flat file parser in the parser list.

The MBCS parser is not designed to work with Unicode encoding. If the property is set to Yes and the codepage was set to Unicode, the MBCS parser will suspend the flat file and log an error. To remedy this problem, change the codepage under the Parse tab of the Editor to a non-Unicode codepage value for the language you are using.

When you use the new MBCS parser and serializer, consider the following points:
  • Positional parsing derives from start and end positions only.
  • The start and end positions for all fields in positional records will be in bytes.
  • Because the validation done on the minimum and maximum length properties (which is counted in characters, not in bytes) cannot be disabled, if the user doesn't know in advance the exact byte length of the field (in other words, whether the field will be a mixture of 1-byte and 2-byte characters or 2-byte characters only), the minimum and maximum length properties must be set to blank in the BizTalk Editor to avoid any validation errors. However, the following functionality will be lost: instance validation, autocalculation, and native instance generation.
  • The serializer should not output leading bytes of 2-byte characters or partial bytes of MBCS characters at the end of the field. You can specify a pad character (for instance, a single-byte space) as a field property in the Parse tab of the BizTalk Editor. This pad character will be used to complete the field if needed.
  • Original Flat File parser requirements:
    • If a document is submitted as a file to BizTalk Server, the encoding of the data is specified in the schema.
    • If data is submitted as a Unicode stream (BSTR) to BizTalk Server, the encoding is always Unicode.
  • MBCS parser requirements:
    • When data gets to the MBCS parser, it is either a Unicode stream (BSTR) or an MBCS stream.
    • Unicode streams (BSTR) can come from HTTP, MSMQ, and Submit calls.
    • MBCS streams can come from File, HTTP, MSMQ, and Submit calls.
    • The MBCS parser only handles byte positioning, and can only do so on the MBCS stream.
    • If the incoming data is a BSTR and the codepage is not UNICODE, the MBCS parser converts it to an MBCS stream and parses it.
    • Unicode surrogate pairs (4 bytes to represent 1 character) are not supported in the original flat files and are also not supported in this MBCS parser.
The MBCS parser addresses the following four situations, assuming that Count Positions in Bytes is set to Yes:
  • The MBCS parser parses data that you submit as a file if the codepage in the schema is not UNICODE.
  • The MBCS parser logs an error if data is submitted as a file and the codepage in the schema is UNICODE. If this happens, either use the original flat file parser or change the name of the codepage.
  • If you submit the data as a BSTR, and the codepage in the schema is not UNICODE, the MBCS parser converts the data to an MBCS stream and parses the data.
  • The MBCS parser logs an error if you submit the data as a BSTR and the codepage in the schema is UNICODE. If this happens, use the original flat file parser.

Modification Type:MinorLast Reviewed:10/12/2005
Keywords:kbHotfixServer kbQFE kbbug kbfix kbQFE KB312616