Kafka protocol

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Kafka protocol

Robert Withers
This would be awesome in Pharo:
https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
Since we cannot yet call JARs, implementing the wire protocol and API
proxies would really be huge from an enterprise pharo standpoint. If
other folks are interested in working on this, I could help out.

Each topic has many partitions and replicates for fail-over. Say #pharo
has 101 partitions with 2 replicas each, totals 300 partitions and two
failovers per partition.  Incoming pharo events would be partition by
some algorithm.

The basic consumer talks to zookeeper to get topology and then talks to
partitions with "fetchers" to put blocks of in-sequence events into a
shed queue. Then single-thread process them in the application level
consumer, by having it consume from the shared queue.

The basic consumer reads from zookeeper then writes to various partitions.

regards,
robert

Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Frank Shearar-3
On 27 December 2015 at 16:11, Robert Withers <[hidden email]> wrote:
> This would be awesome in Pharo:
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
> Since we cannot yet call JARs, implementing the wire protocol and API
> proxies would really be huge from an enterprise pharo standpoint. If other
> folks are interested in working on this, I could help out.

Having a Kafka client in Pharo would indeed be awesome. But who cares
about calling JARs? It's a wire protocol; write a client against that
and you're done.

frank

Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Robert Withers


On 12/28/2015 12:19 PM, Frank Shearar wrote:
> On 27 December 2015 at 16:11, Robert Withers <[hidden email]> wrote:
>> This would be awesome in Pharo:
>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>> Since we cannot yet call JARs, implementing the wire protocol and API
>> proxies would really be huge from an enterprise pharo standpoint. If other
>> folks are interested in working on this, I could help out.
> Having a Kafka client in Pharo would indeed be awesome. But who cares
> about calling JARs? It's a wire protocol; write a client against that
> and you're done.

Yes, although I would still like to load JARs, but for different
reasons. That aside, yes implementing the wire protocol would be really
happening. I'll help. I don't want to own it. It would definitely be
enterprise.

robert


>
> frank
>

--
Robert
.  ..   ...    ^,^


Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Robert Withers
What I mean is that an undertaking of this scope should be in the Vision
and on the roadman for Pharo, with a broader concerted commitment to
enterprise BigData integration.

robert

On 12/28/2015 02:29 PM, Robert Withers wrote:

>
>
> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> On 27 December 2015 at 16:11, Robert Withers
>> <[hidden email]> wrote:
>>> This would be awesome in Pharo:
>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>
>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>> proxies would really be huge from an enterprise pharo standpoint. If
>>> other
>>> folks are interested in working on this, I could help out.
>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> about calling JARs? It's a wire protocol; write a client against that
>> and you're done.
>
> Yes, although I would still like to load JARs, but for different
> reasons. That aside, yes implementing the wire protocol would be
> really happening. I'll help. I don't want to own it. It would
> definitely be enterprise.
>
> robert
>
>
>>
>> frank
>>
>

--
Robert
.  ..   ...    ^,^


Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

kilon.alios
Should be ? There is no "should be" it is what it is, if anyone wants something more he is more than welcomed to stop writing to the mailing list and use that time instead to sit his ass down and make the code . The potential of Pharo is unlimited but taking advantage of it is hard work and nothing else. BigData frameworks dont come from BigWords.

I wanted Pharo to work with python and use python libraries . Did I came to this mailing list to write that pharo "should be" able to use python libraries, because python is super popular and its constantly growing ?

Nope !

I sat down my ass and worked hard to make it happen and I did accomplish it. Still much to improve but there is no substitute to hard work and putting the time to improve pharo.

You wanna use Jars from Pharo , no problemo, work hard and do it. But please dont tell me what should be in the vision of Pharo, why BigData and not 3d graphics, why 3d graphics and not network libraries, why network libraries and not more IDE tools, why IDE tools and not more advanced language features, why advance language features and not a C API , why C API and not web development frameworks ?

In the end nothing , NOTHING , should be outside the scope of a language that wants to call itself "turing complete" . No goal is higher than something else. Each coders has different needs . What it "Should be" is people that are willing to work hard to make THEIR OWN NEEDS a reality using a tool THEY LOVE using. Its a choice.

And a Stef so well says it

Pharo is Yours

On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]> wrote:
What I mean is that an undertaking of this scope should be in the Vision
and on the roadman for Pharo, with a broader concerted commitment to
enterprise BigData integration.

robert

On 12/28/2015 02:29 PM, Robert Withers wrote:
>
>
> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> On 27 December 2015 at 16:11, Robert Withers
>> <[hidden email]> wrote:
>>> This would be awesome in Pharo:
>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>
>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>> proxies would really be huge from an enterprise pharo standpoint. If
>>> other
>>> folks are interested in working on this, I could help out.
>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> about calling JARs? It's a wire protocol; write a client against that
>> and you're done.
>
> Yes, although I would still like to load JARs, but for different
> reasons. That aside, yes implementing the wire protocol would be
> really happening. I'll help. I don't want to own it. It would
> definitely be enterprise.
>
> robert
>
>
>>
>> frank
>>
>

--
Robert
.  ..   ...    ^,^


Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Robert Withers
You're right of course; I made bad choice of words. It is challenging me to explain what I meant as there is something reasonable I am trying to say. I will try.

We all have our own interests and projects we wish to achieve in Pharo. Often this means everyone is working on their own directions and interests and often this means working alone without a team-oriented project. People with similar interests will join efforts and plans. Other projects have been determined to align with pharo's core vision and BigTalk and discussion has yielded plans and work proceedeth collaboratively.

I work on SecureSession and CryptOCaps as my own interests and I get much welcome help but they are my projects. I tr to talk about them in hopes others will use them, but it is my efforts which pay off. SecureSession is solid and usable.

The realm of BigData is different. I have knowledge and experience there and I think it would be powerful for Pharo, but they aree not my main focus. If Pharo wishes to get into this space, it best be done collaboratively which requires some BigWords to coordinate. As they are not my main focus and needs collaboration, a team should form around it, whether it is in the PharoVision or not. This is where I think I misspoke, making it official.

I wish I had twice the time available, to work CryptOCaps and BigData, but I do not. If nobody is interested in seeing how Pharo does with BigData there is not anything I can do nor could I offer my experienmces there to assist.  My suggestions to form a BigData team around it, to coordinate and collaborate, fall short if there is no interest. I will stop encouraging this and refocus back on my work.

My apologies for the distractions,
Robert

On 12/29/2015 07:20 AM, Dimitris Chloupis wrote:
Should be ? There is no "should be" it is what it is, if anyone wants something more he is more than welcomed to stop writing to the mailing list and use that time instead to sit his ass down and make the code . The potential of Pharo is unlimited but taking advantage of it is hard work and nothing else. BigData frameworks dont come from BigWords.

I wanted Pharo to work with python and use python libraries . Did I came to this mailing list to write that pharo "should be" able to use python libraries, because python is super popular and its constantly growing ?

Nope !

I sat down my ass and worked hard to make it happen and I did accomplish it. Still much to improve but there is no substitute to hard work and putting the time to improve pharo.

You wanna use Jars from Pharo , no problemo, work hard and do it. But please dont tell me what should be in the vision of Pharo, why BigData and not 3d graphics, why 3d graphics and not network libraries, why network libraries and not more IDE tools, why IDE tools and not more advanced language features, why advance language features and not a C API , why C API and not web development frameworks ?

In the end nothing , NOTHING , should be outside the scope of a language that wants to call itself "turing complete" . No goal is higher than something else. Each coders has different needs . What it "Should be" is people that are willing to work hard to make THEIR OWN NEEDS a reality using a tool THEY LOVE using. Its a choice.

And a Stef so well says it

Pharo is Yours

On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]> wrote:
What I mean is that an undertaking of this scope should be in the Vision
and on the roadman for Pharo, with a broader concerted commitment to
enterprise BigData integration.

robert

On 12/28/2015 02:29 PM, Robert Withers wrote:
>
>
> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> On 27 December 2015 at 16:11, Robert Withers
>> <[hidden email]> wrote:
>>> This would be awesome in Pharo:
>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>
>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>> proxies would really be huge from an enterprise pharo standpoint. If
>>> other
>>> folks are interested in working on this, I could help out.
>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> about calling JARs? It's a wire protocol; write a client against that
>> and you're done.
>
> Yes, although I would still like to load JARs, but for different
> reasons. That aside, yes implementing the wire protocol would be
> really happening. I'll help. I don't want to own it. It would
> definitely be enterprise.
>
> robert
>
>
>>
>> frank
>>
>

--
Robert
.  ..   ...    ^,^



-- 
Robert
.  ..   ...    ^,^
Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

kilon.alios
and I spend an hour a day coding for pharo , and from that are maybe a 10% is for python support because I have many other needs, like Custom GUIs, visual coding tools, github integration etc etc.

I am sure for many other coders the case is quite similar. Free time is a luxury. But in the end as we say in Greece "bean by bean you make the sack full" or as the English say "in a long road make tiny steps" . It does not matter how much you can help, the fact that you provide a tiniest bit of code will be less time spent for the next coder. Thats helpful enough.

On Tue, Dec 29, 2015 at 2:38 PM Robert Withers <[hidden email]> wrote:
You're right of course; I made bad choice of words. It is challenging me to explain what I meant as there is something reasonable I am trying to say. I will try.

We all have our own interests and projects we wish to achieve in Pharo. Often this means everyone is working on their own directions and interests and often this means working alone without a team-oriented project. People with similar interests will join efforts and plans. Other projects have been determined to align with pharo's core vision and BigTalk and discussion has yielded plans and work proceedeth collaboratively.

I work on SecureSession and CryptOCaps as my own interests and I get much welcome help but they are my projects. I tr to talk about them in hopes others will use them, but it is my efforts which pay off. SecureSession is solid and usable.

The realm of BigData is different. I have knowledge and experience there and I think it would be powerful for Pharo, but they aree not my main focus. If Pharo wishes to get into this space, it best be done collaboratively which requires some BigWords to coordinate. As they are not my main focus and needs collaboration, a team should form around it, whether it is in the PharoVision or not. This is where I think I misspoke, making it official.

I wish I had twice the time available, to work CryptOCaps and BigData, but I do not. If nobody is interested in seeing how Pharo does with BigData there is not anything I can do nor could I offer my experienmces there to assist.  My suggestions to form a BigData team around it, to coordinate and collaborate, fall short if there is no interest. I will stop encouraging this and refocus back on my work.

My apologies for the distractions,
Robert


On 12/29/2015 07:20 AM, Dimitris Chloupis wrote:
Should be ? There is no "should be" it is what it is, if anyone wants something more he is more than welcomed to stop writing to the mailing list and use that time instead to sit his ass down and make the code . The potential of Pharo is unlimited but taking advantage of it is hard work and nothing else. BigData frameworks dont come from BigWords.

I wanted Pharo to work with python and use python libraries . Did I came to this mailing list to write that pharo "should be" able to use python libraries, because python is super popular and its constantly growing ?

Nope !

I sat down my ass and worked hard to make it happen and I did accomplish it. Still much to improve but there is no substitute to hard work and putting the time to improve pharo.

You wanna use Jars from Pharo , no problemo, work hard and do it. But please dont tell me what should be in the vision of Pharo, why BigData and not 3d graphics, why 3d graphics and not network libraries, why network libraries and not more IDE tools, why IDE tools and not more advanced language features, why advance language features and not a C API , why C API and not web development frameworks ?

In the end nothing , NOTHING , should be outside the scope of a language that wants to call itself "turing complete" . No goal is higher than something else. Each coders has different needs . What it "Should be" is people that are willing to work hard to make THEIR OWN NEEDS a reality using a tool THEY LOVE using. Its a choice.

And a Stef so well says it

Pharo is Yours

On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]> wrote:
What I mean is that an undertaking of this scope should be in the Vision
and on the roadman for Pharo, with a broader concerted commitment to
enterprise BigData integration.

robert

On 12/28/2015 02:29 PM, Robert Withers wrote:
>
>
> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> On 27 December 2015 at 16:11, Robert Withers
>> <[hidden email]> wrote:
>>> This would be awesome in Pharo:
>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>
>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>> proxies would really be huge from an enterprise pharo standpoint. If
>>> other
>>> folks are interested in working on this, I could help out.
>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> about calling JARs? It's a wire protocol; write a client against that
>> and you're done.
>
> Yes, although I would still like to load JARs, but for different
> reasons. That aside, yes implementing the wire protocol would be
> really happening. I'll help. I don't want to own it. It would
> definitely be enterprise.
>
> robert
>
>
>>
>> frank
>>
>

--
Robert
.  ..   ...    ^,^



-- 
Robert
.  ..   ...    ^,^
Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Robert Withers
I feel there is something missing from your analysis I would like to highlight, Dimitris. Some projects can be solo. Others require collaborative planning. BigData requires collaboration, especially given it's commercial focus. If there is not interest, it is not a problem for me.

As you, I have this big pile of beans on front of me. I think I'll go pick a few up and put them in the sack.

have a good one,
robert

On 12/29/2015 07:46 AM, Dimitris Chloupis wrote:
and I spend an hour a day coding for pharo , and from that are maybe a 10% is for python support because I have many other needs, like Custom GUIs, visual coding tools, github integration etc etc.

I am sure for many other coders the case is quite similar. Free time is a luxury. But in the end as we say in Greece "bean by bean you make the sack full" or as the English say "in a long road make tiny steps" . It does not matter how much you can help, the fact that you provide a tiniest bit of code will be less time spent for the next coder. Thats helpful enough.

On Tue, Dec 29, 2015 at 2:38 PM Robert Withers <[hidden email]> wrote:
You're right of course; I made bad choice of words. It is challenging me to explain what I meant as there is something reasonable I am trying to say. I will try.

We all have our own interests and projects we wish to achieve in Pharo. Often this means everyone is working on their own directions and interests and often this means working alone without a team-oriented project. People with similar interests will join efforts and plans. Other projects have been determined to align with pharo's core vision and BigTalk and discussion has yielded plans and work proceedeth collaboratively.

I work on SecureSession and CryptOCaps as my own interests and I get much welcome help but they are my projects. I tr to talk about them in hopes others will use them, but it is my efforts which pay off. SecureSession is solid and usable.

The realm of BigData is different. I have knowledge and experience there and I think it would be powerful for Pharo, but they aree not my main focus. If Pharo wishes to get into this space, it best be done collaboratively which requires some BigWords to coordinate. As they are not my main focus and needs collaboration, a team should form around it, whether it is in the PharoVision or not. This is where I think I misspoke, making it official.

I wish I had twice the time available, to work CryptOCaps and BigData, but I do not. If nobody is interested in seeing how Pharo does with BigData there is not anything I can do nor could I offer my experienmces there to assist.  My suggestions to form a BigData team around it, to coordinate and collaborate, fall short if there is no interest. I will stop encouraging this and refocus back on my work.

My apologies for the distractions,
Robert


On 12/29/2015 07:20 AM, Dimitris Chloupis wrote:
Should be ? There is no "should be" it is what it is, if anyone wants something more he is more than welcomed to stop writing to the mailing list and use that time instead to sit his ass down and make the code . The potential of Pharo is unlimited but taking advantage of it is hard work and nothing else. BigData frameworks dont come from BigWords.

I wanted Pharo to work with python and use python libraries . Did I came to this mailing list to write that pharo "should be" able to use python libraries, because python is super popular and its constantly growing ?

Nope !

I sat down my ass and worked hard to make it happen and I did accomplish it. Still much to improve but there is no substitute to hard work and putting the time to improve pharo.

You wanna use Jars from Pharo , no problemo, work hard and do it. But please dont tell me what should be in the vision of Pharo, why BigData and not 3d graphics, why 3d graphics and not network libraries, why network libraries and not more IDE tools, why IDE tools and not more advanced language features, why advance language features and not a C API , why C API and not web development frameworks ?

In the end nothing , NOTHING , should be outside the scope of a language that wants to call itself "turing complete" . No goal is higher than something else. Each coders has different needs . What it "Should be" is people that are willing to work hard to make THEIR OWN NEEDS a reality using a tool THEY LOVE using. Its a choice.

And a Stef so well says it

Pharo is Yours

On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]> wrote:
What I mean is that an undertaking of this scope should be in the Vision
and on the roadman for Pharo, with a broader concerted commitment to
enterprise BigData integration.

robert

On 12/28/2015 02:29 PM, Robert Withers wrote:
>
>
> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> On 27 December 2015 at 16:11, Robert Withers
>> <[hidden email]> wrote:
>>> This would be awesome in Pharo:
>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>
>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>> proxies would really be huge from an enterprise pharo standpoint. If
>>> other
>>> folks are interested in working on this, I could help out.
>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> about calling JARs? It's a wire protocol; write a client against that
>> and you're done.
>
> Yes, although I would still like to load JARs, but for different
> reasons. That aside, yes implementing the wire protocol would be
> really happening. I'll help. I don't want to own it. It would
> definitely be enterprise.
>
> robert
>
>
>>
>> frank
>>
>

--
Robert
.  ..   ...    ^,^



-- 
Robert
.  ..   ...    ^,^

-- 
Robert
.  ..   ...    ^,^
Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Ben Coman
In reply to this post by kilon.alios
On Tue, Dec 29, 2015 at 11:20 PM, Dimitris Chloupis
<[hidden email]> wrote:
> Should be ? There is no "should be" it is what it is, if anyone wants
> something more he is more than welcomed to stop writing to the mailing list
> and use that time instead to sit his ass down and make the code.

I think Its okay to sometimes advocate for features.  The *hope* of
course is that the idea sparks the interest of others.  It just needs
to be moderated by:  expectation, channel noise and own-effort (as you
say).  The word "should" is a loaded term with a subtle difference in
two
1. used to indicate obligation, duty, or correctness, typically when
criticizing someone's actions.
2. used to indicate what is probably [a good idea].

I often fall in the trap of meaning the second, but my wife hears the
first --> trouble.
cheers -ben

> The
> potential of Pharo is unlimited but taking advantage of it is hard work and
> nothing else. BigData frameworks dont come from BigWords.
>
> I wanted Pharo to work with python and use python libraries . Did I came to
> this mailing list to write that pharo "should be" able to use python
> libraries, because python is super popular and its constantly growing ?
>
> Nope !
>
> I sat down my ass and worked hard to make it happen and I did accomplish it.
> Still much to improve but there is no substitute to hard work and putting
> the time to improve pharo.
>
> You wanna use Jars from Pharo , no problemo, work hard and do it. But please
> dont tell me what should be in the vision of Pharo, why BigData and not 3d
> graphics, why 3d graphics and not network libraries, why network libraries
> and not more IDE tools, why IDE tools and not more advanced language
> features, why advance language features and not a C API , why C API and not
> web development frameworks ?
>
> In the end nothing , NOTHING , should be outside the scope of a language
> that wants to call itself "turing complete" . No goal is higher than
> something else. Each coders has different needs . What it "Should be" is
> people that are willing to work hard to make THEIR OWN NEEDS a reality using
> a tool THEY LOVE using. Its a choice.
>
> And a Stef so well says it
>
> Pharo is Yours
>
> On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]>
> wrote:
>>
>> What I mean is that an undertaking of this scope should be in the Vision
>> and on the roadman for Pharo, with a broader concerted commitment to
>> enterprise BigData integration.
>>
>> robert
>>
>> On 12/28/2015 02:29 PM, Robert Withers wrote:
>> >
>> >
>> > On 12/28/2015 12:19 PM, Frank Shearar wrote:
>> >> On 27 December 2015 at 16:11, Robert Withers
>> >> <[hidden email]> wrote:
>> >>> This would be awesome in Pharo:
>> >>>
>> >>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>> >>>
>> >>> Since we cannot yet call JARs, implementing the wire protocol and API
>> >>> proxies would really be huge from an enterprise pharo standpoint. If
>> >>> other
>> >>> folks are interested in working on this, I could help out.
>> >> Having a Kafka client in Pharo would indeed be awesome. But who cares
>> >> about calling JARs? It's a wire protocol; write a client against that
>> >> and you're done.
>> >
>> > Yes, although I would still like to load JARs, but for different
>> > reasons. That aside, yes implementing the wire protocol would be
>> > really happening. I'll help. I don't want to own it. It would
>> > definitely be enterprise.
>> >
>> > robert
>> >
>> >
>> >>
>> >> frank
>> >>
>> >
>>
>> --
>> Robert
>> .  ..   ...    ^,^
>>
>>
>

Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

Robert Withers


> On Dec 29, 2015, at 19:50, Ben Coman <[hidden email]> wrote:
>
> On Tue, Dec 29, 2015 at 11:20 PM, Dimitris Chloupis
> <[hidden email]> wrote:
>> Should be ? There is no "should be" it is what it is, if anyone wants
>> something more he is more than welcomed to stop writing to the mailing list
>> and use that time instead to sit his ass down and make the code.
>
> I think Its okay to sometimes advocate for features.  The *hope* of
> course is that the idea sparks the interest of others.  It just needs
> to be moderated by:  expectation, channel noise and own-effort (as you
> say).  The word "should" is a loaded term with a subtle difference in
> two
> 1. used to indicate obligation, duty, or correctness, typically when
> criticizing someone's actions.
> 2. used to indicate what is probably [a good idea].

+1

Thank you for putting it so clearly.

Robert

>
> I often fall in the trap of meaning the second, but my wife hears the
> first --> trouble.
> cheers -ben
>
>> The
>> potential of Pharo is unlimited but taking advantage of it is hard work and
>> nothing else. BigData frameworks dont come from BigWords.
>>
>> I wanted Pharo to work with python and use python libraries . Did I came to
>> this mailing list to write that pharo "should be" able to use python
>> libraries, because python is super popular and its constantly growing ?
>>
>> Nope !
>>
>> I sat down my ass and worked hard to make it happen and I did accomplish it.
>> Still much to improve but there is no substitute to hard work and putting
>> the time to improve pharo.
>>
>> You wanna use Jars from Pharo , no problemo, work hard and do it. But please
>> dont tell me what should be in the vision of Pharo, why BigData and not 3d
>> graphics, why 3d graphics and not network libraries, why network libraries
>> and not more IDE tools, why IDE tools and not more advanced language
>> features, why advance language features and not a C API , why C API and not
>> web development frameworks ?
>>
>> In the end nothing , NOTHING , should be outside the scope of a language
>> that wants to call itself "turing complete" . No goal is higher than
>> something else. Each coders has different needs . What it "Should be" is
>> people that are willing to work hard to make THEIR OWN NEEDS a reality using
>> a tool THEY LOVE using. Its a choice.
>>
>> And a Stef so well says it
>>
>> Pharo is Yours
>>
>> On Mon, Dec 28, 2015 at 9:44 PM Robert Withers <[hidden email]>
>> wrote:
>>>
>>> What I mean is that an undertaking of this scope should be in the Vision
>>> and on the roadman for Pharo, with a broader concerted commitment to
>>> enterprise BigData integration.
>>>
>>> robert
>>>
>>>> On 12/28/2015 02:29 PM, Robert Withers wrote:
>>>>
>>>>
>>>>> On 12/28/2015 12:19 PM, Frank Shearar wrote:
>>>>> On 27 December 2015 at 16:11, Robert Withers
>>>>> <[hidden email]> wrote:
>>>>>> This would be awesome in Pharo:
>>>>>>
>>>>>> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
>>>>>>
>>>>>> Since we cannot yet call JARs, implementing the wire protocol and API
>>>>>> proxies would really be huge from an enterprise pharo standpoint. If
>>>>>> other
>>>>>> folks are interested in working on this, I could help out.
>>>>> Having a Kafka client in Pharo would indeed be awesome. But who cares
>>>>> about calling JARs? It's a wire protocol; write a client against that
>>>>> and you're done.
>>>>
>>>> Yes, although I would still like to load JARs, but for different
>>>> reasons. That aside, yes implementing the wire protocol would be
>>>> really happening. I'll help. I don't want to own it. It would
>>>> definitely be enterprise.
>>>>
>>>> robert
>>>>
>>>>
>>>>>
>>>>> frank
>>>
>>> --
>>> Robert
>>> .  ..   ...    ^,^
>

cbc
Reply | Threaded
Open this post in threaded view
|

Re: Kafka protocol

cbc
In reply to this post by Frank Shearar-3
I wrote a low-level interface for this a while back:
(it does skip zookeeper, though).  And I've used it for reading from Kafka, but have not verified writing to it - and didn't finish that part (didn't need it).

I did need additional ByteArray at:put: and at: methods - for 32 and 64 bit access (and 8 bit signed access as well) - which where mostly added to Squeak (where I used this from).

Enjoy,
-cbc

On Mon, Dec 28, 2015 at 9:19 AM, Frank Shearar <[hidden email]> wrote:
On 27 December 2015 at 16:11, Robert Withers <[hidden email]> wrote:
> This would be awesome in Pharo:
> https://cwiki.apache.org/confluence/display/KAFKA/A+Guide+To+The+Kafka+Protocol.
> Since we cannot yet call JARs, implementing the wire protocol and API
> proxies would really be huge from an enterprise pharo standpoint. If other
> folks are interested in working on this, I could help out.

Having a Kafka client in Pharo would indeed be awesome. But who cares
about calling JARs? It's a wire protocol; write a client against that
and you're done.

frank