encoding challenge
Locked
 
|
A core part of communications, whether online or to persistence, is
encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? Let's show them how to code when given the space in the environment. Tnks! :) In class ASN1UTCTimeType.. encodeValue: anObject withDERStream: derStream | yy mo dd hh mm ss utcDateTime | yy := anObject year asString copyFrom: 3 to: 4. mo := anObject month asString padded: #left to: 2 with: $0. dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. hh := anObject hour asString padded: #left to: 2 with: $0. mm := anObject minute asString padded: #left to: 2 with: $0. ss := anObject seconds asString padded: #left to: 2 with: $0. utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. derStream nextPutAll: utcDateTime. decodeValueWithDERStream: derStream length: length | aUTCDateTime | aUTCDateTime := (derStream next: length) asByteArray asString. ^ (DateAndTime readFromString: ( ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), (aUTCDateTime copyFrom: 1 to: 2), '-', (aUTCDateTime copyFrom: 3 to: 4), '-', (aUTCDateTime copyFrom: 5 to: 6), 'T', (aUTCDateTime copyFrom: 7 to: 8), ':', (aUTCDateTime copyFrom: 9 to: 10), ':', (aUTCDateTime copyFrom: 11 to: 12))) offset: 0 -- . .. .. ^,^ robert |
|
|
It turns out my problem was elsewhere. These convert just fine.
My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. I will look into Asn1Der encoding of the control messages, however. cheers,, robert On 12/19/2015 01:09 AM, Robert Withers wrote: > A core part of communications, whether online or to persistence, is > encoding, or marshalling. Let's do an example! Thank you for your > thoughtful, earnest participation, as God wills it. > > Here are 2 methods that encode and decode a DateAndTime object, but > rather poorly. I've been going without much sleep for a bit and I > can't figure it out. Here is the code in hopes that some will find > this an interesting exercise in team and community building. For that > intention, as a beginning. What's the best solution? > > Let's show them how to code when given the space in the environment. > Tnks! :) > > In class ASN1UTCTimeType.. > > encodeValue: anObject withDERStream: derStream > > | yy mo dd hh mm ss utcDateTime | > yy := anObject year asString copyFrom: 3 to: 4. > mo := anObject month asString padded: #left to: 2 with: $0. > dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. > hh := anObject hour asString padded: #left to: 2 with: $0. > mm := anObject minute asString padded: #left to: 2 with: $0. > ss := anObject seconds asString padded: #left to: 2 with: $0. > utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. > derStream nextPutAll: utcDateTime. > > > > decodeValueWithDERStream: derStream length: length > > | aUTCDateTime | > aUTCDateTime := (derStream next: length) asByteArray asString. > ^ (DateAndTime readFromString: ( > ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: > ['19'] ifFalse: ['20']), > (aUTCDateTime copyFrom: 1 to: 2), '-', > (aUTCDateTime copyFrom: 3 to: 4), '-', > (aUTCDateTime copyFrom: 5 to: 6), 'T', > (aUTCDateTime copyFrom: 7 to: 8), ':', > (aUTCDateTime copyFrom: 9 to: 10), ':', > (aUTCDateTime copyFrom: 11 to: 12))) > offset: 0 > > -- . .. .. ^,^ robert |
Re: [Pharo-users] encoding challenge
|
|
Hi,
If you are interested in ASN.1, there is an implementation of that as part of the SS7 project: CI Server is here: https://ci.hartl.name/job/ASN.1/ This implements a ASN.1 parser. This means you do not have to do the encoding yourself but you can use the ASN.1 as published. Very useful when implementing protocols that are specified using ASN.1, e.g. all the telephony stuff for mobile networks. If it makes sense to use in this context I am not sure. Marcus > On 19 Dec 2015, at 09:22, Robert Withers <[hidden email]> wrote: > > It turns out my problem was elsewhere. These convert just fine. > > My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. > > I will look into Asn1Der encoding of the control messages, however. > > cheers,, > robert > > On 12/19/2015 01:09 AM, Robert Withers wrote: >> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >> >> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >> >> Let's show them how to code when given the space in the environment. Tnks! :) >> >> In class ASN1UTCTimeType.. >> >> encodeValue: anObject withDERStream: derStream >> >> | yy mo dd hh mm ss utcDateTime | >> yy := anObject year asString copyFrom: 3 to: 4. >> mo := anObject month asString padded: #left to: 2 with: $0. >> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >> hh := anObject hour asString padded: #left to: 2 with: $0. >> mm := anObject minute asString padded: #left to: 2 with: $0. >> ss := anObject seconds asString padded: #left to: 2 with: $0. >> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >> derStream nextPutAll: utcDateTime. >> >> >> >> decodeValueWithDERStream: derStream length: length >> >> | aUTCDateTime | >> aUTCDateTime := (derStream next: length) asByteArray asString. >> ^ (DateAndTime readFromString: ( >> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >> (aUTCDateTime copyFrom: 1 to: 2), '-', >> (aUTCDateTime copyFrom: 3 to: 4), '-', >> (aUTCDateTime copyFrom: 5 to: 6), 'T', >> (aUTCDateTime copyFrom: 7 to: 8), ':', >> (aUTCDateTime copyFrom: 9 to: 10), ':', >> (aUTCDateTime copyFrom: 11 to: 12))) >> offset: 0 >> >> > > -- > . .. .. ^,^ robert > |
Re: [Pharo-users] encoding challenge
|
|
Hi Marcus,
That is what I am using, an ASN.1 parser I wrote years ago. It is decently robust, but I defer marshalling to the objects, which define their schemas as well. The schemas rely on getter/setters. I do not recall if I started it or picked up where others left off. For sure, Ron and others helped build it adn for that I am grateful. If you want to see the inside of structure and mapped type marshalling, please load version 61 of SecureSession and run the Connect test and look at the stack in the debugger, from the error. I have had to refamiliarize myself with it. This is not to say the other parser you mentioned isn't a candidate, yet I have to manage my focus. If you are familiar with that parser or ASN.1 in genral, I would really welcome some feedback on this implementation. best, robert On 12/19/2015 03:36 AM, Marcus Denker wrote: > Hi, > > If you are interested in ASN.1, there is an implementation of that as part of the SS7 project: > CI Server is here: > > https://ci.hartl.name/job/ASN.1/ > > This implements a ASN.1 parser. This means you do not have to do the encoding > yourself but you can use the ASN.1 as published. Very useful when implementing protocols that > are specified using ASN.1, e.g. all the telephony stuff for mobile networks. > > If it makes sense to use in this context I am not sure. > > Marcus > >> On 19 Dec 2015, at 09:22, Robert Withers <[hidden email]> wrote: >> >> It turns out my problem was elsewhere. These convert just fine. >> >> My issue is the combination of flipping the whole stack to frames and adding headers and header method calls down at the Transport layer. The session layer is actually 2 sub-layers, data on top of control. In this case, we need a data frame but no control frame, so the transport is frameless. It is blowing up on me because control messages have an artificial frame. >> >> I will look into Asn1Der encoding of the control messages, however. >> >> cheers,, >> robert >> >> On 12/19/2015 01:09 AM, Robert Withers wrote: >>> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >>> >>> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >>> >>> Let's show them how to code when given the space in the environment. Tnks! :) >>> >>> In class ASN1UTCTimeType.. >>> >>> encodeValue: anObject withDERStream: derStream >>> >>> | yy mo dd hh mm ss utcDateTime | >>> yy := anObject year asString copyFrom: 3 to: 4. >>> mo := anObject month asString padded: #left to: 2 with: $0. >>> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >>> hh := anObject hour asString padded: #left to: 2 with: $0. >>> mm := anObject minute asString padded: #left to: 2 with: $0. >>> ss := anObject seconds asString padded: #left to: 2 with: $0. >>> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >>> derStream nextPutAll: utcDateTime. >>> >>> >>> >>> decodeValueWithDERStream: derStream length: length >>> >>> | aUTCDateTime | >>> aUTCDateTime := (derStream next: length) asByteArray asString. >>> ^ (DateAndTime readFromString: ( >>> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >>> (aUTCDateTime copyFrom: 1 to: 2), '-', >>> (aUTCDateTime copyFrom: 3 to: 4), '-', >>> (aUTCDateTime copyFrom: 5 to: 6), 'T', >>> (aUTCDateTime copyFrom: 7 to: 8), ':', >>> (aUTCDateTime copyFrom: 9 to: 10), ':', >>> (aUTCDateTime copyFrom: 11 to: 12))) >>> offset: 0 >>> >>> >> -- >> . .. .. ^,^ robert >> > -- . .. .. ^,^ robert |
Re: [Pharo-users] [squeak-dev] encoding challenge
|
|
In reply to this post by Robert Withers
Thank you, this is good. I was modeling DateAndTime as UTCTime, which is
a specified format for two digit years: http://www.obj-sys.com/asn1tutorial/node15.html. I noticed this other time representation while lookin... GeneralizedTime: http://www.obj-sys.com/asn1tutorial/node14.html. To meet that format with padding, I decided to convert to strings. Not pretty but there is a test! :) Unfortunately the ASN.1 DER endoding for time is specific format. I coudl just throws milliseconds in an integer slot, but I wanted to meet the spec as well for a full ASN.1 tool. I am undecided whether to use ASN.1 for header encoding in SecureSession, or custom binary specification - I use the second option at this time. Yet I am in the middle of redefining that message protocol, so now would be the time to decide. Fuel and I have a date coming up, for sure. thank you, Robert On 12/21/2015 03:34 PM, Sven Van Caekenberghe wrote: > Robert, > >> On 19 Dec 2015, at 07:09, Robert Withers <[hidden email]> wrote: >> >> A core part of communications, whether online or to persistence, is encoding, or marshalling. Let's do an example! Thank you for your thoughtful, earnest participation, as God wills it. >> >> Here are 2 methods that encode and decode a DateAndTime object, but rather poorly. I've been going without much sleep for a bit and I can't figure it out. Here is the code in hopes that some will find this an interesting exercise in team and community building. For that intention, as a beginning. What's the best solution? >> >> Let's show them how to code when given the space in the environment. Tnks! :) >> >> In class ASN1UTCTimeType.. >> >> encodeValue: anObject withDERStream: derStream >> >> | yy mo dd hh mm ss utcDateTime | >> yy := anObject year asString copyFrom: 3 to: 4. >> mo := anObject month asString padded: #left to: 2 with: $0. >> dd := anObject dayOfMonth asString padded: #left to: 2 with: $0. >> hh := anObject hour asString padded: #left to: 2 with: $0. >> mm := anObject minute asString padded: #left to: 2 with: $0. >> ss := anObject seconds asString padded: #left to: 2 with: $0. >> utcDateTime := (yy, mo, dd, hh, mm, ss, 'Z') asByteArray. >> derStream nextPutAll: utcDateTime. >> >> >> >> decodeValueWithDERStream: derStream length: length >> >> | aUTCDateTime | >> aUTCDateTime := (derStream next: length) asByteArray asString. >> ^ (DateAndTime readFromString: ( >> ((aUTCDateTime copyFrom: 1 to: 2) asInteger > 50 ifTrue: ['19'] ifFalse: ['20']), >> (aUTCDateTime copyFrom: 1 to: 2), '-', >> (aUTCDateTime copyFrom: 3 to: 4), '-', >> (aUTCDateTime copyFrom: 5 to: 6), 'T', >> (aUTCDateTime copyFrom: 7 to: 8), ':', >> (aUTCDateTime copyFrom: 9 to: 10), ':', >> (aUTCDateTime copyFrom: 11 to: 12))) >> offset: 0 >> >> >> -- >> . .. .. ^,^ robert > Like you say, this is *not* good, because it is pretty inefficient. You want to encode binary, yet you create a string representation. But even that is done badly ;-) Any you are introducing 2YK all over again. > > Anyway, the solution is to encode the timestamp using one or more numbers, directly. From your code I assume you are only interested in second precision and the UTC/GMT/Z timezone. In that case either one rather large number (seconds since X, like unix time) or two smaller numbers (julian day number and seconds since midnight) are way more efficient. > > For inspiration, have a look at Fuel's DateAndTime>>#serializeOn: (that also encodes nanoseconds and the timezone). > > HTH, > > Sven > > > -- . .. ... ^,^ robert Go Panthers! |
«
Return to Squeak - Dev
|
1 view|%1 views
| Free forum by Nabble | Edit this page |