Fixed length varies with new download client

  • 1
  • Problem
  • Updated 1 year ago
  • In Progress
  • (Edited)

Since upgrading to the new download client, we are getting download records that are not fixed length even though our format file defines fixed length as the output format. Needless to say, this makes it next to impossible to parse. Has anyone else seen this problem, and if so, how did you rectify it. Will there be a fix coming to the download client? thanks. [edit: this happens both with and without the stripdiacritics feature].
Photo of severa

severa, Champion

  • 101 Posts
  • 10 Reply Likes

Posted 1 year ago

  • 1
Photo of Kasey Crosby

Kasey Crosby, Official Rep

  • 66 Posts
  • 2 Reply Likes
Hello,

I've created a support ticket so we can better troubleshoot your issue. Please respond there with which fields are not downloading correctly. It would also be helpful if you would include your format file for us to take a look at as well.

Thank you,

kc
Photo of severa

severa, Champion

  • 101 Posts
  • 10 Reply Likes

Its hard to know exactly which fields are downloading incorrectly because its not easy to parse it. In one case i do know that it was the address field. Basically there was a 4 byte Unicode blank space value inserted between the words in the address field, but i think it can be any field. Others i will have to parse by hand. My guess is its all utf-8 stuff though. I saw this bug a few years ago with some Chinese characters and decided not to use stripdiacritics for that reason. Now it seems to happen whether we use that or not. In that old situation, the output length was not truncating any extra characters created by conversion to utf-8 (where a single 'character' is represented by multiple bytes). So my guess is the code just needs to truncate to the length specified in the format file (instead of the length specified by the field value).
Alternatively, not allowing utf-8 to go into the output might be another approach.