Subscribing to streaming input with ZnClient

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Subscribing to streaming input with ZnClient

Jimmie Houchin-5

Hello,

I am wanting to subscribe to a perpetual stream of JSON data in my app. I have failed to find an example which helps me understand how to do so. It may be there but I have not discovered it.

How do I do something like that? A curl example from the website.

curl \
  -H "Authorization: abcdefg?????" \
  "https://stream-mystreamingsite.com/stream?query=whatIwant"
Response Headers
HTTP/1.1 200 OK
Access-Control-Allow-Headers: Authorization, Content-Type, Accept-Datetime-Format
Access-Control-Allow-Methods: PUT, PATCH, POST, GET, OPTIONS, DELETE
Access-Control-Allow-Origin: *
Content-Type: application/octet-stream


After subscribing to the stream I will need to get and access the JSON data so that my app can process it.

Any help and wisdom greatly appreciated.

Thanks.

Jimmie


  



	
	
	
	
Reply | Threaded
Open this post in threaded view
|

Re: Subscribing to streaming input with ZnClient

Sven Van Caekenberghe-2
Hi Kimmy,

> On 27 Dec 2018, at 15:15, Jimmie Houchin <[hidden email]> wrote:
>
> Hello,
>
> I am wanting to subscribe to a perpetual stream of JSON data in my app. I have failed to find an example which helps me understand       how to do so. It may be there but I have not discovered it.
>
> How do I do something like that? A curl example from the website.
>
> curl \
>   -H "Authorization: abcdefg?????" \
>  
> "https://stream-mystreamingsite.com/stream?query=whatIwant"
> Response Headers
> HTTP/1.1 200 OK
> Access-Control-Allow-Headers: Authorization, Content-Type, Accept-Datetime-Format
> Access-Control-Allow-Methods: PUT, PATCH, POST, GET, OPTIONS, DELETE
> Access-Control-Allow-Origin: *
> Content-Type: application/octet-stream
>
>
> After subscribing to the stream I will need to get and access the JSON data so that my app can process it.
>
> Any help and wisdom greatly appreciated.
>
> Thanks.
>
> Jimmie

The client and server part of the WebSockets extension to Zinc HTTP Components already supports similar functionality, so this is certainly doable.

Unless you can give me a public end point to test, I can only write some untested pseudo code.

| client reader |
client := ZnClient new.
client url: 'https://stream-mystreamingsite.com/stream?query=whatIwant'.
client streaming: true.
client headerAt: 'Authorization' put: 'xxx'.
client get.
reader := NeoJSONReader on: (ZnCharacterReadStream on: client contents).
[ reader atEnd] whileFalse: [ reader next ].
reader close.
client close.

Be sure to tell us how you got it working.

HTH,

Sven


Reply | Threaded
Open this post in threaded view
|

Re: Subscribing to streaming input with ZnClient

Jimmie Houchin-5
On 12/27/18 9:00 AM, Sven Van Caekenberghe wrote:

> Hi Jimmie,
>
>> On 27 Dec 2018, at 15:15, Jimmie Houchin <[hidden email]> wrote:
>>
>> Hello,
>>
>> I am wanting to subscribe to a perpetual stream of JSON data in my app. I have failed to find an example which helps me understand       how to do so. It may be there but I have not discovered it.
>>
>> How do I do something like that? A curl example from the website.
>>
>> curl \
>>    -H "Authorization: abcdefg?????" \
>>    
>> "https://stream-mystreamingsite.com/stream?query=whatIwant"
>> Response Headers
>> HTTP/1.1 200 OK
>> Access-Control-Allow-Headers: Authorization, Content-Type, Accept-Datetime-Format
>> Access-Control-Allow-Methods: PUT, PATCH, POST, GET, OPTIONS, DELETE
>> Access-Control-Allow-Origin: *
>> Content-Type: application/octet-stream
>>
>>
>> After subscribing to the stream I will need to get and access the JSON data so that my app can process it.
>>
>> Any help and wisdom greatly appreciated.
>>
>> Thanks.
>>
>> Jimmie
> The client and server part of the WebSockets extension to Zinc HTTP Components already supports similar functionality, so this is certainly doable.
>
> Unless you can give me a public end point to test, I can only write some untested pseudo code.
>
> | client reader |
> client := ZnClient new.
> client url: 'https://stream-mystreamingsite.com/stream?query=whatIwant'.
> client streaming: true.
> client headerAt: 'Authorization' put: 'xxx'.
> client get.
> reader := NeoJSONReader on: (ZnCharacterReadStream on: client contents).
> [ reader atEnd] whileFalse: [ reader next ].
> reader close.
> client close.
>
> Be sure to tell us how you got it working.
>
> HTH,
>
> Sven

Hello Sven,

Thanks for your help.

The server I am connecting to requires an account to access. But you
provided sufficient information to get me going. Thanks.

This is for a Trading Application. I am a ways off before ready for
releasing. I will open source when it is usable and cleaned up enough
from personal code.

Currently I have named it SmallTrader, hence the ST prefix for classes.

I have an STClient class which subclasses ZnClient and sets it up with
the necessary account authorization.

Below is the STPriceStream#mainLoop method.

Basically it is populating some OrderedCollections in the STPriceStream
which are available to the app's mainLoop.

It is working perfectly. I currently have it printing out data to stdout
so that I can watch the data it collects to visually understand what is
happening.

The connection below is the instance of the class which connects to this
specific broker. Since connections are specific to a broker I am putting
all of the parsing into ST objects such as STPrice and STCandle in the
broker specific class as it understands what the broker returns. This
will allow my app to be able to use different brokers simply by changing
which connection class we call. Currently I am writing for one specific
one where my account is located. But I want to be able to add brokers in
time.

Thanks again for your help.

mainLoop
     | delay pricesStream reader |
     delay := (Duration milliSeconds: 100) asDelay.
     minuteDateTime := STCandle datetimeMinute: (DateAndTime now asUTC).
     pricesStream := connection newPriceStreamConnectionForInstruments:
instruments.
     pricesStream get.
     reader := STONReader on: (ZnCharacterReadStream on: pricesStream
contents).
     [ reader atEnd] whileFalse: [
         stop
             ifFalse: [ | price |
                 price := connection pricesStreamNextPrice: (reader next).
                 price ifNotNil: [
                     prices add: price.
                     price datetimeMinute = minuteDateTime
                         ifTrue: [ minutePrices add: price ]
                         ifFalse: [ self processMinutePrices: price ]]]
             ifTrue: [
                 reader close
                 ^ self ].
         delay wait ].


Jimmie







Reply | Threaded
Open this post in threaded view
|

Re: Subscribing to streaming input with ZnClient

Sven Van Caekenberghe-2
Very nice, and thank you for the feedback.

> On 28 Dec 2018, at 23:16, Jimmie Houchin <[hidden email]> wrote:
>
> On 12/27/18 9:00 AM, Sven Van Caekenberghe wrote:
>> Hi Jimmie,
>>
>>> On 27 Dec 2018, at 15:15, Jimmie Houchin <[hidden email]> wrote:
>>>
>>> Hello,
>>>
>>> I am wanting to subscribe to a perpetual stream of JSON data in my app. I have failed to find an example which helps me understand       how to do so. It may be there but I have not discovered it.
>>>
>>> How do I do something like that? A curl example from the website.
>>>
>>> curl \
>>>   -H "Authorization: abcdefg?????" \
>>>   "https://stream-mystreamingsite.com/stream?query=whatIwant"
>>> Response Headers
>>> HTTP/1.1 200 OK
>>> Access-Control-Allow-Headers: Authorization, Content-Type, Accept-Datetime-Format
>>> Access-Control-Allow-Methods: PUT, PATCH, POST, GET, OPTIONS, DELETE
>>> Access-Control-Allow-Origin: *
>>> Content-Type: application/octet-stream
>>>
>>>
>>> After subscribing to the stream I will need to get and access the JSON data so that my app can process it.
>>>
>>> Any help and wisdom greatly appreciated.
>>>
>>> Thanks.
>>>
>>> Jimmie
>> The client and server part of the WebSockets extension to Zinc HTTP Components already supports similar functionality, so this is certainly doable.
>>
>> Unless you can give me a public end point to test, I can only write some untested pseudo code.
>>
>> | client reader |
>> client := ZnClient new.
>> client url: 'https://stream-mystreamingsite.com/stream?query=whatIwant'.
>> client streaming: true.
>> client headerAt: 'Authorization' put: 'xxx'.
>> client get.
>> reader := NeoJSONReader on: (ZnCharacterReadStream on: client contents).
>> [ reader atEnd] whileFalse: [ reader next ].
>> reader close.
>> client close.
>>
>> Be sure to tell us how you got it working.
>>
>> HTH,
>>
>> Sven
>
> Hello Sven,
>
> Thanks for your help.
>
> The server I am connecting to requires an account to access. But you provided sufficient information to get me going. Thanks.
>
> This is for a Trading Application. I am a ways off before ready for releasing. I will open source when it is usable and cleaned up enough from personal code.
>
> Currently I have named it SmallTrader, hence the ST prefix for classes.
>
> I have an STClient class which subclasses ZnClient and sets it up with the necessary account authorization.
>
> Below is the STPriceStream#mainLoop method.
>
> Basically it is populating some OrderedCollections in the STPriceStream which are available to the app's mainLoop.
>
> It is working perfectly. I currently have it printing out data to stdout so that I can watch the data it collects to visually understand what is happening.
>
> The connection below is the instance of the class which connects to this specific broker. Since connections are specific to a broker I am putting all of the parsing into ST objects such as STPrice and STCandle in the broker specific class as it understands what the broker returns. This will allow my app to be able to use different brokers simply by changing which connection class we call. Currently I am writing for one specific one where my account is located. But I want to be able to add brokers in time.
>
> Thanks again for your help.
>
> mainLoop
>     | delay pricesStream reader |
>     delay := (Duration milliSeconds: 100) asDelay.
>     minuteDateTime := STCandle datetimeMinute: (DateAndTime now asUTC).
>     pricesStream := connection newPriceStreamConnectionForInstruments: instruments.
>     pricesStream get.
>     reader := STONReader on: (ZnCharacterReadStream on: pricesStream contents).
>     [ reader atEnd] whileFalse: [
>         stop
>             ifFalse: [ | price |
>                 price := connection pricesStreamNextPrice: (reader next).
>                 price ifNotNil: [
>                     prices add: price.
>                     price datetimeMinute = minuteDateTime
>                         ifTrue: [ minutePrices add: price ]
>                         ifFalse: [ self processMinutePrices: price ]]]
>             ifTrue: [
>                 reader close
>                 ^ self ].
>         delay wait ].
>
>
> Jimmie