HTTP request question for an arango driver

On 9 Dec 2017, at 18:34, Yann Lesage [hidden email] wrote:



Le 09/12/2017 à 15:52, Sven Van Caekenberghe a écrit :

Hi Yann,

Zinc HTTP Components can do 1000s of requests per second, to localhost (so excluding a real network) and using a single ZnClient instance with a reused connection (HTTP/1.1's default). Of course, data size is also a factor, I am talking about small requests/responses.

I browsed your code a bit on GitHub. You do reuse an instance, so that is good. But I think you are using HTTPS (TLS), which is a real slowdown (encryption is native, but costs real resources). Also, your data payload is using JSON which also adds a cost (parsing, generating).

Thanks for this review.

So what you measured sounds about right. You might be able to optimise a bit, but that won't give you a factor 10 improvement, IMHO. The trick is usually to make as few requests as possible.

A factor of 10 maximum ? Ok, this indication help me.

If you cache and reuse a single ZnClient instance, that is already good. Maybe try once without SSL to see how much difference that makes. You could also try to disable logging. As in

ZnClient new in: [ :client |
  [ client get: 'http://localhost:8080' ] benchFor: 5 seconds ].

vs.

ZnClient new in: [ :client |
  client loggingOff.
  [ client get: 'http://localhost:8080' ] benchFor: 5 seconds ].

You can also try to run in a Time profiler, but the server time is hard to abstract away from.

For the trick, I know it. But it's must be performed by user no ?

Yes, the more you do in a single netwerk round trip, the better. Aggregation is good. 

Sven

On 8 Dec 2017, at 16:32, Yann Lesage [hidden email] wrote:

Hello,


I write an driver for Arangodb . So like it's indicated in Arango documentation, I use the HTTP API.  The repo is https://github.com/Valtena/Pharango


Now, the problem : Arango using Znclient make around 1 000 requests/second.


And the question : Are there any recommended pratice to have the better performance with ZNClient or a better way to perform lot of HTTP requests ?


Thanks for your attention,

Yann Lesage

Le 10/12/2017 à 19:18, Sven Van Caekenberghe a écrit :

On 9 Dec 2017, at 18:34, Yann Lesage [hidden email] wrote:



Le 09/12/2017 à 15:52, Sven Van Caekenberghe a écrit :

Hi Yann,

Zinc HTTP Components can do 1000s of requests per second, to localhost (so excluding a real network) and using a single ZnClient instance with a reused connection (HTTP/1.1's default). Of course, data size is also a factor, I am talking about small requests/responses.

I browsed your code a bit on GitHub. You do reuse an instance, so that is good. But I think you are using HTTPS (TLS), which is a real slowdown (encryption is native, but costs real resources). Also, your data payload is using JSON which also adds a cost (parsing, generating).

Thanks for this review.

So what you measured sounds about right. You might be able to optimise a bit, but that won't give you a factor 10 improvement, IMHO. The trick is usually to make as few requests as possible.

A factor of 10 maximum ? Ok, this indication help me.

If you cache and reuse a single ZnClient instance, that is already good. Maybe try once without SSL to see how much difference that makes. You could also try to disable logging. As in

ZnClient new in: [ :client |
  [ client get: 'http://localhost:8080' ] benchFor: 5 seconds ].

vs.

ZnClient new in: [ :client |
  client loggingOff.
  [ client get: 'http://localhost:8080' ] benchFor: 5 seconds ].

You can also try to run in a Time profiler, but the server time is hard to abstract away from.

Use SSL or not is set by ZNCLient in fonction of url (http or https), no ? If there an option to set completely off SSL, I don't think is a good idea to use it. SSL is a real improve in security when we connect to a distant server.