Zinc JSON parsing error handling

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Zinc JSON parsing error handling

Ben Coman

With... 
  Object subclass: #BittrexResponse
instanceVariableNames: 'success message result'
classVariableNames: ''
package: 'Bittrex'

  Object subclass: #BittrexMarketSummary
instanceVariableNames: 'MarketName High Low Volume Last 
                BaseVolume TimeStamp Bid Ask OpenBuyOrders 
                OpenSellOrders PrevDay Created DisplayMarketName'
classVariableNames: ''
package: 'Bittrex'

this code works great when the response holds good data...
  ZnClient new 
enforceHttpSuccess: true;
accept: ZnMimeType applicationJson;
contentReader: [ :entity | |reader|
reader := (NeoJSONReader on: entity readStream).
reader for: BittrexResponse do: [:m| 
m mapInstVar: #success.
m mapInstVar: #message.
(m mapInstVar: #result) valueSchema: #ResultArray].
reader for: #ResultArray customDo: [ :mapping | 
                       mapping listOfElementSchema: BittrexMarketSummary ].
reader mapInstVarsFor: BittrexMarketSummary. 
      reader nextAs: BittrexResponse ];
   get.

i.e. a raw response looking like this....
(ZnClient new 
get) inspect.
==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"


But for bad response looking like this...
==> {"success":false,"message":"INVALID_MARKET","result":null}

the JSON handling code fails deep in the call stack with an error
     "NeoJSONParseError: [ expected" 
which is not so friendly for users of the Bittrex library.  

What are the different/recommended approaches with Zinc 
for catching JSON errors such that I can pass "message" 
as a higher level Error up the stack to the Bittrex user.
 
 cheers -ben
Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

Sven Van Caekenberghe-2
BTW, this is no about Zinc, but about NeoJSON.

This is what I meant with variability that is hard to capture with a simple static type schema.

I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.

> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>
>
> With...
>   Object subclass: #BittrexResponse
> instanceVariableNames: 'success message result'
> classVariableNames: ''
> package: 'Bittrex'
>
>   Object subclass: #BittrexMarketSummary
> instanceVariableNames: 'MarketName High Low Volume Last
>                 BaseVolume TimeStamp Bid Ask OpenBuyOrders
>                 OpenSellOrders PrevDay Created DisplayMarketName'
> classVariableNames: ''
> package: 'Bittrex'
>
> this code works great when the response holds good data...
>   ZnClient new
> url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
> enforceHttpSuccess: true;
> accept: ZnMimeType applicationJson;
> contentReader: [ :entity | |reader|
> reader := (NeoJSONReader on: entity readStream).
> reader for: BittrexResponse do: [:m|
> m mapInstVar: #success.
> m mapInstVar: #message.
> (m mapInstVar: #result) valueSchema: #ResultArray].
> reader for: #ResultArray customDo: [ :mapping |
>                        mapping listOfElementSchema: BittrexMarketSummary ].
> reader mapInstVarsFor: BittrexMarketSummary.
>       reader nextAs: BittrexResponse ];
>    get.
>
> i.e. a raw response looking like this....
> (ZnClient new
> url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
> get) inspect.
> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>
>
> But for bad response looking like this...
> (ZnClient new
> url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
> get) inspect.
> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>
> the JSON handling code fails deep in the call stack with an error
>      "NeoJSONParseError: [ expected"
> which is not so friendly for users of the Bittrex library.  
>
> What are the different/recommended approaches with Zinc
> for catching JSON errors such that I can pass "message"
> as a higher level Error up the stack to the Bittrex user.
>  
>  cheers -ben


Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

Ben Coman
hi Sven,

> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>
>
> With...
>   Object subclass: #BittrexResponse
>       instanceVariableNames: 'success message result'
>       classVariableNames: ''
>       package: 'Bittrex'
>
>   Object subclass: #BittrexMarketSummary
>       instanceVariableNames: 'MarketName High Low Volume Last
>                 BaseVolume TimeStamp Bid Ask OpenBuyOrders
>                 OpenSellOrders PrevDay Created DisplayMarketName'
>       classVariableNames: ''
>       package: 'Bittrex'
>
> this code works great when the response holds good data...
>   ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>       enforceHttpSuccess: true;
>       accept: ZnMimeType applicationJson;
>       contentReader: [ :entity | |reader|
>               reader := (NeoJSONReader on: entity readStream).
>               reader for: BittrexResponse do: [:m|
>                       m mapInstVar: #success.
>                       m mapInstVar: #message.
>                       (m mapInstVar: #result) valueSchema: #ResultArray].
>               reader for: #ResultArray customDo: [ :mapping |
>                        mapping listOfElementSchema: BittrexMarketSummary ].
>               reader mapInstVarsFor: BittrexMarketSummary.
>       reader nextAs: BittrexResponse ];
>    get.
>
> i.e. a raw response looking like this....
> (ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>       get) inspect.
> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>
>
> But for bad response looking like this...
> (ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>       get) inspect.
> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>
> the JSON handling code fails deep in the call stack with an error
>      "NeoJSONParseError: [ expected"
> which is not so friendly for users of the Bittrex library.
>
> What are the different/recommended approaches with Zinc
> for catching JSON errors such that I can pass "message"
> as a higher level Error up the stack to the Bittrex user.
>
>  cheers -ben


On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
BTW, this is no about Zinc, but about NeoJSON.

This is what I meant with variability that is hard to capture with a simple static type schema.

I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.

 
okay.  So I played around tracing this through the debugger and for anyone else
with a similar need later, I ended up with the following... 

response := (ZnClient new 
enforceHttpSuccess: true;
accept: ZnMimeType applicationJson;
contentReader: 
[ :entity | 
[ |reader|
reader := (NeoJSONReader on: entity readStream).
reader for: BittrexResponse do: 
[:m| 
m mapInstVar: #success.
m mapInstVar: #message.
(m mapInstVar: #result) valueSchema: BittrexTicker
].
reader mapInstVarsFor: BittrexTicker.
reader nextAs: BittrexResponse
] on: Error do: 
[ :err | |json| 
json := NeoJSONReader fromString: entity contents. 
(json at: 'success') 
ifFalse: [ self error: (json at: 'message') ]
ifTrue: [ self error: 'UKNOWN ERROR' ]

]) get. 


for which I get a nice pre-debug window titled "Error: INVALID_MARKET"

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

NorbertHartl
To make your core a little bit more reliable you should not catch the general error but a NeoJSONParseError. And you need to guard the dictionary access. There is no guarantee you get something back that inlcudes key success and message.

Norbert

Am 13.12.2017 um 09:54 schrieb Ben Coman <[hidden email]>:

hi Sven,

> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>
>
> With...
>   Object subclass: #BittrexResponse
>       instanceVariableNames: 'success message result'
>       classVariableNames: ''
>       package: 'Bittrex'
>
>   Object subclass: #BittrexMarketSummary
>       instanceVariableNames: 'MarketName High Low Volume Last
>                 BaseVolume TimeStamp Bid Ask OpenBuyOrders
>                 OpenSellOrders PrevDay Created DisplayMarketName'
>       classVariableNames: ''
>       package: 'Bittrex'
>
> this code works great when the response holds good data...
>   ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>       enforceHttpSuccess: true;
>       accept: ZnMimeType applicationJson;
>       contentReader: [ :entity | |reader|
>               reader := (NeoJSONReader on: entity readStream).
>               reader for: BittrexResponse do: [:m|
>                       m mapInstVar: #success.
>                       m mapInstVar: #message.
>                       (m mapInstVar: #result) valueSchema: #ResultArray].
>               reader for: #ResultArray customDo: [ :mapping |
>                        mapping listOfElementSchema: BittrexMarketSummary ].
>               reader mapInstVarsFor: BittrexMarketSummary.
>       reader nextAs: BittrexResponse ];
>    get.
>
> i.e. a raw response looking like this....
> (ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>       get) inspect.
> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>
>
> But for bad response looking like this...
> (ZnClient new
>       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>       get) inspect.
> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>
> the JSON handling code fails deep in the call stack with an error
>      "NeoJSONParseError: [ expected"
> which is not so friendly for users of the Bittrex library.
>
> What are the different/recommended approaches with Zinc
> for catching JSON errors such that I can pass "message"
> as a higher level Error up the stack to the Bittrex user.
>
>  cheers -ben


On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
BTW, this is no about Zinc, but about NeoJSON.

This is what I meant with variability that is hard to capture with a simple static type schema.

I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.

 
okay.  So I played around tracing this through the debugger and for anyone else
with a similar need later, I ended up with the following... 

response := (ZnClient new 
enforceHttpSuccess: true;
accept: ZnMimeType applicationJson;
contentReader: 
[ :entity | 
[ |reader|
reader := (NeoJSONReader on: entity readStream).
reader for: BittrexResponse do: 
[:m| 
m mapInstVar: #success.
m mapInstVar: #message.
(m mapInstVar: #result) valueSchema: BittrexTicker
].
reader mapInstVarsFor: BittrexTicker.
reader nextAs: BittrexResponse
] on: Error do: 
[ :err | |json| 
json := NeoJSONReader fromString: entity contents. 
(json at: 'success') 
ifFalse: [ self error: (json at: 'message') ]
ifTrue: [ self error: 'UKNOWN ERROR' ]

]) get. 


for which I get a nice pre-debug window titled "Error: INVALID_MARKET"

cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

Sven Van Caekenberghe-2


> On 13 Dec 2017, at 10:14, Norbert Hartl <[hidden email]> wrote:
>
> To make your core a little bit more reliable you should not catch the general error but a NeoJSONParseError. And you need to guard the dictionary access. There is no guarantee you get something back that inlcudes key success and message.

Yes, indeed.

Also, a better REST API (not something you can change) would not return 200 OK with an error in the payload, but would return a 404 Not Found with a proper error object as payload.

FWIW, there are a couple of more sophisticated examples, like in NeoJSONMappingTests and NeoJSONExamplesTests, with comments that do various special cases.

> Norbert
>
> Am 13.12.2017 um 09:54 schrieb Ben Coman <[hidden email]>:
>
>> hi Sven,
>>
>> > On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>> >
>> >
>> > With...
>> >   Object subclass: #BittrexResponse
>> >       instanceVariableNames: 'success message result'
>> >       classVariableNames: ''
>> >       package: 'Bittrex'
>> >
>> >   Object subclass: #BittrexMarketSummary
>> >       instanceVariableNames: 'MarketName High Low Volume Last
>> >                 BaseVolume TimeStamp Bid Ask OpenBuyOrders
>> >                 OpenSellOrders PrevDay Created DisplayMarketName'
>> >       classVariableNames: ''
>> >       package: 'Bittrex'
>> >
>> > this code works great when the response holds good data...
>> >   ZnClient new
>> >       url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>> >       enforceHttpSuccess: true;
>> >       accept: ZnMimeType applicationJson;
>> >       contentReader: [ :entity | |reader|
>> >               reader := (NeoJSONReader on: entity readStream).
>> >               reader for: BittrexResponse do: [:m|
>> >                       m mapInstVar: #success.
>> >                       m mapInstVar: #message.
>> >                       (m mapInstVar: #result) valueSchema: #ResultArray].
>> >               reader for: #ResultArray customDo: [ :mapping |
>> >                        mapping listOfElementSchema: BittrexMarketSummary ].
>> >               reader mapInstVarsFor: BittrexMarketSummary.
>> >       reader nextAs: BittrexResponse ];
>> >    get.
>> >
>> > i.e. a raw response looking like this....
>> > (ZnClient new
>> >       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>> >       get) inspect.
>> > ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>> >
>> >
>> > But for bad response looking like this...
>> > (ZnClient new
>> >       url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>> >       get) inspect.
>> > ==> {"success":false,"message":"INVALID_MARKET","result":null}
>> >
>> > the JSON handling code fails deep in the call stack with an error
>> >      "NeoJSONParseError: [ expected"
>> > which is not so friendly for users of the Bittrex library.
>> >
>> > What are the different/recommended approaches with Zinc
>> > for catching JSON errors such that I can pass "message"
>> > as a higher level Error up the stack to the Bittrex user.
>> >
>> >  cheers -ben
>>
>>
>> On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
>> BTW, this is no about Zinc, but about NeoJSON.
>>
>> This is what I meant with variability that is hard to capture with a simple static type schema.
>>
>> I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.
>>
>>  
>> okay.  So I played around tracing this through the debugger and for anyone else
>> with a similar need later, I ended up with the following...
>>
>> response := (ZnClient new
>> url: 'https://bittrex.com/api/v1.1/public/getticker?market=INVALID';
>> enforceHttpSuccess: true;
>> accept: ZnMimeType applicationJson;
>> contentReader:
>> [ :entity |
>> [ |reader|
>> reader := (NeoJSONReader on: entity readStream).
>> reader for: BittrexResponse do:
>> [:m|
>> m mapInstVar: #success.
>> m mapInstVar: #message.
>> (m mapInstVar: #result) valueSchema: BittrexTicker
>> ].
>> reader mapInstVarsFor: BittrexTicker.
>> reader nextAs: BittrexResponse
>> ] on: Error do:
>> [ :err | |json|
>> json := NeoJSONReader fromString: entity contents.
>> (json at: 'success')
>> ifFalse: [ self error: (json at: 'message') ]
>> ifTrue: [ self error: 'UKNOWN ERROR' ]
>> ]
>> ]) get.
>>
>>
>> for which I get a nice pre-debug window titled "Error: INVALID_MARKET"
>>
>> cheers -ben
>>


Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

NorbertHartl

> Am 13.12.2017 um 10:18 schrieb Sven Van Caekenberghe <[hidden email]>:
>
>
>
>> On 13 Dec 2017, at 10:14, Norbert Hartl <[hidden email]> wrote:
>>
>> To make your core a little bit more reliable you should not catch the general error but a NeoJSONParseError. And you need to guard the dictionary access. There is no guarantee you get something back that inlcudes key success and message.
>
> Yes, indeed.
>
> Also, a better REST API (not something you can change) would not return 200 OK with an error in the payload, but would return a 404 Not Found with a proper error object as payload.
>
I must say that the latter is not a good advize. All HTTP status codes refer to the resource that is specified in the uri. REST is resource based but GET and POST are exceptions beacuse they can address processing entities. It means that the result of the operation is not the entity in the uri. Returning a 404 would mean that the

https://bittrex.com/api/v1.1/public/getticker

resource is not available but that is not true and can lead to unwanted behaviour, e.g. that a http stack caches the error and does not try to access that resource again. So the proper status code depends on the definition of the interface. If it is considered to always return an object then the status code needs to be 400 because the specification of the client was invalid. On the other hand you can just return a 200 with an empty result if the search result is optional.

my 2 cents,

Norbert

> FWIW, there are a couple of more sophisticated examples, like in NeoJSONMappingTests and NeoJSONExamplesTests, with comments that do various special cases.
>
>> Norbert
>>
>> Am 13.12.2017 um 09:54 schrieb Ben Coman <[hidden email]>:
>>
>>> hi Sven,
>>>
>>>> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>>>>
>>>>
>>>> With...
>>>>  Object subclass: #BittrexResponse
>>>>      instanceVariableNames: 'success message result'
>>>>      classVariableNames: ''
>>>>      package: 'Bittrex'
>>>>
>>>>  Object subclass: #BittrexMarketSummary
>>>>      instanceVariableNames: 'MarketName High Low Volume Last
>>>>                BaseVolume TimeStamp Bid Ask OpenBuyOrders
>>>>                OpenSellOrders PrevDay Created DisplayMarketName'
>>>>      classVariableNames: ''
>>>>      package: 'Bittrex'
>>>>
>>>> this code works great when the response holds good data...
>>>>  ZnClient new
>>>>      url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>>>>      enforceHttpSuccess: true;
>>>>      accept: ZnMimeType applicationJson;
>>>>      contentReader: [ :entity | |reader|
>>>>              reader := (NeoJSONReader on: entity readStream).
>>>>              reader for: BittrexResponse do: [:m|
>>>>                      m mapInstVar: #success.
>>>>                      m mapInstVar: #message.
>>>>                      (m mapInstVar: #result) valueSchema: #ResultArray].
>>>>              reader for: #ResultArray customDo: [ :mapping |
>>>>                       mapping listOfElementSchema: BittrexMarketSummary ].
>>>>              reader mapInstVarsFor: BittrexMarketSummary.
>>>>      reader nextAs: BittrexResponse ];
>>>>   get.
>>>>
>>>> i.e. a raw response looking like this....
>>>> (ZnClient new
>>>>      url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>>>>      get) inspect.
>>>> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>>>>
>>>>
>>>> But for bad response looking like this...
>>>> (ZnClient new
>>>>      url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>>>>      get) inspect.
>>>> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>>>>
>>>> the JSON handling code fails deep in the call stack with an error
>>>>     "NeoJSONParseError: [ expected"
>>>> which is not so friendly for users of the Bittrex library.
>>>>
>>>> What are the different/recommended approaches with Zinc
>>>> for catching JSON errors such that I can pass "message"
>>>> as a higher level Error up the stack to the Bittrex user.
>>>>
>>>> cheers -ben
>>>
>>>
>>> On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
>>> BTW, this is no about Zinc, but about NeoJSON.
>>>
>>> This is what I meant with variability that is hard to capture with a simple static type schema.
>>>
>>> I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.
>>>
>>>
>>> okay.  So I played around tracing this through the debugger and for anyone else
>>> with a similar need later, I ended up with the following...
>>>
>>> response := (ZnClient new
>>> url: 'https://bittrex.com/api/v1.1/public/getticker?market=INVALID';
>>> enforceHttpSuccess: true;
>>> accept: ZnMimeType applicationJson;
>>> contentReader:
>>> [ :entity |
>>> [ |reader|
>>> reader := (NeoJSONReader on: entity readStream).
>>> reader for: BittrexResponse do:
>>> [:m|
>>> m mapInstVar: #success.
>>> m mapInstVar: #message.
>>> (m mapInstVar: #result) valueSchema: BittrexTicker
>>> ].
>>> reader mapInstVarsFor: BittrexTicker.
>>> reader nextAs: BittrexResponse
>>> ] on: Error do:
>>> [ :err | |json|
>>> json := NeoJSONReader fromString: entity contents.
>>> (json at: 'success')
>>> ifFalse: [ self error: (json at: 'message') ]
>>> ifTrue: [ self error: 'UKNOWN ERROR' ]
>>> ]
>>> ]) get.
>>>
>>>
>>> for which I get a nice pre-debug window titled "Error: INVALID_MARKET"
>>>
>>> cheers -ben
>>>

Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

Sven Van Caekenberghe-2


> On 13 Dec 2017, at 10:54, Norbert Hartl <[hidden email]> wrote:
>
>>
>> Am 13.12.2017 um 10:18 schrieb Sven Van Caekenberghe <[hidden email]>:
>>
>>
>>
>>> On 13 Dec 2017, at 10:14, Norbert Hartl <[hidden email]> wrote:
>>>
>>> To make your core a little bit more reliable you should not catch the general error but a NeoJSONParseError. And you need to guard the dictionary access. There is no guarantee you get something back that inlcudes key success and message.
>>
>> Yes, indeed.
>>
>> Also, a better REST API (not something you can change) would not return 200 OK with an error in the payload, but would return a 404 Not Found with a proper error object as payload.
>>
> I must say that the latter is not a good advize. All HTTP status codes refer to the resource that is specified in the uri. REST is resource based but GET and POST are exceptions beacuse they can address processing entities. It means that the result of the operation is not the entity in the uri. Returning a 404 would mean that the
>
> https://bittrex.com/api/v1.1/public/getticker
>
> resource is not available but that is not true and can lead to unwanted behaviour, e.g. that a http stack caches the error and does not try to access that resource again. So the proper status code depends on the definition of the interface. If it is considered to always return an object then the status code needs to be 400 because the specification of the client was invalid. On the other hand you can just return a 200 with an empty result if the search result is optional.
>
> my 2 cents,

Well, yes, you're right. I was too quick and assumed the resource was properly named.

That was another remark. The interface does not properly refer to a resource, it is still too procedural, IMHO.

I would write it as

 https://bittrex.com/api/v1.1/public/tickers/LTC

A process interface should only be used for, duh, processes that can be identified as such, like a trade that then gets its own unique URI that can be queried for state and so on.

REST interface design is a subject we can discuss endlessly, no doubt.

> Norbert
>
>> FWIW, there are a couple of more sophisticated examples, like in NeoJSONMappingTests and NeoJSONExamplesTests, with comments that do various special cases.
>>
>>> Norbert
>>>
>>> Am 13.12.2017 um 09:54 schrieb Ben Coman <[hidden email]>:
>>>
>>>> hi Sven,
>>>>
>>>>> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>>>>>
>>>>>
>>>>> With...
>>>>> Object subclass: #BittrexResponse
>>>>>     instanceVariableNames: 'success message result'
>>>>>     classVariableNames: ''
>>>>>     package: 'Bittrex'
>>>>>
>>>>> Object subclass: #BittrexMarketSummary
>>>>>     instanceVariableNames: 'MarketName High Low Volume Last
>>>>>               BaseVolume TimeStamp Bid Ask OpenBuyOrders
>>>>>               OpenSellOrders PrevDay Created DisplayMarketName'
>>>>>     classVariableNames: ''
>>>>>     package: 'Bittrex'
>>>>>
>>>>> this code works great when the response holds good data...
>>>>> ZnClient new
>>>>>     url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>>>>>     enforceHttpSuccess: true;
>>>>>     accept: ZnMimeType applicationJson;
>>>>>     contentReader: [ :entity | |reader|
>>>>>             reader := (NeoJSONReader on: entity readStream).
>>>>>             reader for: BittrexResponse do: [:m|
>>>>>                     m mapInstVar: #success.
>>>>>                     m mapInstVar: #message.
>>>>>                     (m mapInstVar: #result) valueSchema: #ResultArray].
>>>>>             reader for: #ResultArray customDo: [ :mapping |
>>>>>                      mapping listOfElementSchema: BittrexMarketSummary ].
>>>>>             reader mapInstVarsFor: BittrexMarketSummary.
>>>>>     reader nextAs: BittrexResponse ];
>>>>>  get.
>>>>>
>>>>> i.e. a raw response looking like this....
>>>>> (ZnClient new
>>>>>     url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>>>>>     get) inspect.
>>>>> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>>>>>
>>>>>
>>>>> But for bad response looking like this...
>>>>> (ZnClient new
>>>>>     url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>>>>>     get) inspect.
>>>>> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>>>>>
>>>>> the JSON handling code fails deep in the call stack with an error
>>>>>    "NeoJSONParseError: [ expected"
>>>>> which is not so friendly for users of the Bittrex library.
>>>>>
>>>>> What are the different/recommended approaches with Zinc
>>>>> for catching JSON errors such that I can pass "message"
>>>>> as a higher level Error up the stack to the Bittrex user.
>>>>>
>>>>> cheers -ben
>>>>
>>>>
>>>> On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
>>>> BTW, this is no about Zinc, but about NeoJSON.
>>>>
>>>> This is what I meant with variability that is hard to capture with a simple static type schema.
>>>>
>>>> I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.
>>>>
>>>>
>>>> okay.  So I played around tracing this through the debugger and for anyone else
>>>> with a similar need later, I ended up with the following...
>>>>
>>>> response := (ZnClient new
>>>> url: 'https://bittrex.com/api/v1.1/public/getticker?market=INVALID';
>>>> enforceHttpSuccess: true;
>>>> accept: ZnMimeType applicationJson;
>>>> contentReader:
>>>> [ :entity |
>>>> [ |reader|
>>>> reader := (NeoJSONReader on: entity readStream).
>>>> reader for: BittrexResponse do:
>>>> [:m|
>>>> m mapInstVar: #success.
>>>> m mapInstVar: #message.
>>>> (m mapInstVar: #result) valueSchema: BittrexTicker
>>>> ].
>>>> reader mapInstVarsFor: BittrexTicker.
>>>> reader nextAs: BittrexResponse
>>>> ] on: Error do:
>>>> [ :err | |json|
>>>> json := NeoJSONReader fromString: entity contents.
>>>> (json at: 'success')
>>>> ifFalse: [ self error: (json at: 'message') ]
>>>> ifTrue: [ self error: 'UKNOWN ERROR' ]
>>>> ]
>>>> ]) get.
>>>>
>>>>
>>>> for which I get a nice pre-debug window titled "Error: INVALID_MARKET"
>>>>
>>>> cheers -ben


Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

NorbertHartl


> Am 13.12.2017 um 11:04 schrieb Sven Van Caekenberghe <[hidden email]>:
>
>
>
>>> On 13 Dec 2017, at 10:54, Norbert Hartl <[hidden email]> wrote:
>>>
>>>
>>> Am 13.12.2017 um 10:18 schrieb Sven Van Caekenberghe <[hidden email]>:
>>>
>>>
>>>
>>>> On 13 Dec 2017, at 10:14, Norbert Hartl <[hidden email]> wrote:
>>>>
>>>> To make your core a little bit more reliable you should not catch the general error but a NeoJSONParseError. And you need to guard the dictionary access. There is no guarantee you get something back that inlcudes key success and message.
>>>
>>> Yes, indeed.
>>>
>>> Also, a better REST API (not something you can change) would not return 200 OK with an error in the payload, but would return a 404 Not Found with a proper error object as payload.
>>>
>> I must say that the latter is not a good advize. All HTTP status codes refer to the resource that is specified in the uri. REST is resource based but GET and POST are exceptions beacuse they can address processing entities. It means that the result of the operation is not the entity in the uri. Returning a 404 would mean that the
>>
>> https://bittrex.com/api/v1.1/public/getticker
>>
>> resource is not available but that is not true and can lead to unwanted behaviour, e.g. that a http stack caches the error and does not try to access that resource again. So the proper status code depends on the definition of the interface. If it is considered to always return an object then the status code needs to be 400 because the specification of the client was invalid. On the other hand you can just return a 200 with an empty result if the search result is optional.
>>
>> my 2 cents,
>
> Well, yes, you're right. I was too quick and assumed the resource was properly named.
>
> That was another remark. The interface does not properly refer to a resource, it is still too procedural, IMHO.
>
> I would write it as
>
> https://bittrex.com/api/v1.1/public/tickers/LTC
>
> A process interface should only be used for, duh, processes that can be identified as such, like a trade that then gets its own unique URI that can be queried for state and so on.
>
> REST interface design is a subject we can discuss endlessly, no doubt.
>
No doubt! But with you I might enjoy it 😉
Nevertheless I think we don‘t want to live without RPC and that is hard to map on REST.

Norbert

>> Norbert
>>
>>> FWIW, there are a couple of more sophisticated examples, like in NeoJSONMappingTests and NeoJSONExamplesTests, with comments that do various special cases.
>>>
>>>> Norbert
>>>>
>>>>> Am 13.12.2017 um 09:54 schrieb Ben Coman <[hidden email]>:
>>>>>
>>>>> hi Sven,
>>>>>
>>>>>> On 13 Dec 2017, at 07:59, Ben Coman <[hidden email]> wrote:
>>>>>>
>>>>>>
>>>>>> With...
>>>>>> Object subclass: #BittrexResponse
>>>>>>    instanceVariableNames: 'success message result'
>>>>>>    classVariableNames: ''
>>>>>>    package: 'Bittrex'
>>>>>>
>>>>>> Object subclass: #BittrexMarketSummary
>>>>>>    instanceVariableNames: 'MarketName High Low Volume Last
>>>>>>              BaseVolume TimeStamp Bid Ask OpenBuyOrders
>>>>>>              OpenSellOrders PrevDay Created DisplayMarketName'
>>>>>>    classVariableNames: ''
>>>>>>    package: 'Bittrex'
>>>>>>
>>>>>> this code works great when the response holds good data...
>>>>>> ZnClient new
>>>>>>    url: 'https://bittrex.com/api/v1.1/public/getmarketSummary?market=BTC-LTC';
>>>>>>    enforceHttpSuccess: true;
>>>>>>    accept: ZnMimeType applicationJson;
>>>>>>    contentReader: [ :entity | |reader|
>>>>>>            reader := (NeoJSONReader on: entity readStream).
>>>>>>            reader for: BittrexResponse do: [:m|
>>>>>>                    m mapInstVar: #success.
>>>>>>                    m mapInstVar: #message.
>>>>>>                    (m mapInstVar: #result) valueSchema: #ResultArray].
>>>>>>            reader for: #ResultArray customDo: [ :mapping |
>>>>>>                     mapping listOfElementSchema: BittrexMarketSummary ].
>>>>>>            reader mapInstVarsFor: BittrexMarketSummary.
>>>>>>    reader nextAs: BittrexResponse ];
>>>>>> get.
>>>>>>
>>>>>> i.e. a raw response looking like this....
>>>>>> (ZnClient new
>>>>>>    url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=BTC-LTC';
>>>>>>    get) inspect.
>>>>>> ==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"
>>>>>>
>>>>>>
>>>>>> But for bad response looking like this...
>>>>>> (ZnClient new
>>>>>>    url: 'https://bittrex.com/api/v1.1/public/getmarketsummary?market=INVALID';
>>>>>>    get) inspect.
>>>>>> ==> {"success":false,"message":"INVALID_MARKET","result":null}
>>>>>>
>>>>>> the JSON handling code fails deep in the call stack with an error
>>>>>>   "NeoJSONParseError: [ expected"
>>>>>> which is not so friendly for users of the Bittrex library.
>>>>>>
>>>>>> What are the different/recommended approaches with Zinc
>>>>>> for catching JSON errors such that I can pass "message"
>>>>>> as a higher level Error up the stack to the Bittrex user.
>>>>>>
>>>>>> cheers -ben
>>>>>
>>>>>
>>>>> On 13 December 2017 at 15:37, Sven Van Caekenberghe <[hidden email]> wrote:
>>>>> BTW, this is no about Zinc, but about NeoJSON.
>>>>>
>>>>> This is what I meant with variability that is hard to capture with a simple static type schema.
>>>>>
>>>>> I have no time to try myself right now, but a custom mapping for ivar result (as in a block) might be able to see the difference and act accordingly.
>>>>>
>>>>>
>>>>> okay.  So I played around tracing this through the debugger and for anyone else
>>>>> with a similar need later, I ended up with the following...
>>>>>
>>>>> response := (ZnClient new
>>>>>    url: 'https://bittrex.com/api/v1.1/public/getticker?market=INVALID';
>>>>>    enforceHttpSuccess: true;
>>>>>    accept: ZnMimeType applicationJson;
>>>>>    contentReader:
>>>>>    [    :entity |
>>>>>        [    |reader|
>>>>>            reader := (NeoJSONReader on: entity readStream).
>>>>>            reader for: BittrexResponse do:
>>>>>            [:m|
>>>>>                m mapInstVar: #success.
>>>>>                m mapInstVar: #message.
>>>>>                (m mapInstVar: #result) valueSchema: BittrexTicker
>>>>>            ].
>>>>>            reader mapInstVarsFor: BittrexTicker.
>>>>>            reader nextAs: BittrexResponse
>>>>>        ] on: Error do:
>>>>>            [    :err | |json|
>>>>>                json := NeoJSONReader fromString: entity contents.
>>>>>                (json at: 'success')
>>>>>                    ifFalse: [ self error: (json at: 'message') ]
>>>>>                    ifTrue: [ self error: 'UKNOWN ERROR' ]
>>>>>            ]
>>>>>    ]) get.
>>>>>
>>>>>
>>>>> for which I get a nice pre-debug window titled "Error: INVALID_MARKET"
>>>>>
>>>>> cheers -ben

Reply | Threaded
Open this post in threaded view
|

Re: Zinc JSON parsing error handling

Esteban A. Maringolo
In reply to this post by Ben Coman
Is this Bittrex client available somewhere to play with?

Regards!

Esteban A. Maringolo

2017-12-13 3:59 GMT-03:00 Ben Coman <[hidden email]>:

With... 
  Object subclass: #BittrexResponse
instanceVariableNames: 'success message result'
classVariableNames: ''
package: 'Bittrex'

  Object subclass: #BittrexMarketSummary
instanceVariableNames: 'MarketName High Low Volume Last 
                BaseVolume TimeStamp Bid Ask OpenBuyOrders 
                OpenSellOrders PrevDay Created DisplayMarketName'
classVariableNames: ''
package: 'Bittrex'

this code works great when the response holds good data...
  ZnClient new 
enforceHttpSuccess: true;
accept: ZnMimeType applicationJson;
contentReader: [ :entity | |reader|
reader := (NeoJSONReader on: entity readStream).
reader for: BittrexResponse do: [:m| 
m mapInstVar: #success.
m mapInstVar: #message.
(m mapInstVar: #result) valueSchema: #ResultArray].
reader for: #ResultArray customDo: [ :mapping | 
                       mapping listOfElementSchema: BittrexMarketSummary ].
reader mapInstVarsFor: BittrexMarketSummary. 
      reader nextAs: BittrexResponse ];
   get.

i.e. a raw response looking like this....
(ZnClient new 
get) inspect.
==> "'{""success"":true,""message"":"",""result"":[{""MarketName"":""BTC-LTC"",""High"":0.01982450,""Low"":0.01285257,""Volume"":1436429.81313360,""Last"":0.01842000,""BaseVolume"":24841.17217724,""TimeStamp"":""2017-12-13T05:56:25.937"",""Bid"":0.01840001,""Ask"":0.01842000,""OpenBuyOrders"":10140,""OpenSellOrders"":6306,""PrevDay"":0.01439800,""Created"":""2014-02-13T00:00:00""}]}'"


But for bad response looking like this...
==> {"success":false,"message":"INVALID_MARKET","result":null}

the JSON handling code fails deep in the call stack with an error
     "NeoJSONParseError: [ expected" 
which is not so friendly for users of the Bittrex library.  

What are the different/recommended approaches with Zinc 
for catching JSON errors such that I can pass "message" 
as a higher level Error up the stack to the Bittrex user.
 
 cheers -ben