High thread allocation of manifold with Aleph?

Discussion:

Jeroen van Dijk

2015-12-03 13:32:41 UTC

Hi,

When I look at the Threads tab within jvisualvm I see a relative high
number of new manifold threads being generated. Other libraries like
core.async and httpkit seem to reuse threads (more). The Manifold threads
have a short live span and every second, roughly, 10 threads are being
created. This happens during a local load test of around 900 QPS.

Could I be doing something wrong or this is by design of Aleph/Manifold?

Thanks,
Jeroen

--
You received this message because you are subscribed to the Google Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aleph-lib+***@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Zach Tellman

2015-12-03 15:27:42 UTC

Permalink

What are the names of the threads?

Post by Jeroen van Dijk
Hi,
When I look at the Threads tab within jvisualvm I see a relative high
number of new manifold threads being generated. Other libraries like
core.async and httpkit seem to reuse threads (more). The Manifold threads
have a short live span and every second, roughly, 10 threads are being
created. This happens during a local load test of around 900 QPS.
Could I be doing something wrong or this is by design of Aleph/Manifold?
Thanks,
Jeroen
--
You received this message because you are subscribed to the Google Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

Jeroen van Dijk

2015-12-03 17:00:06 UTC

Permalink

manifold-pool-3-419
manifold-pool-3-420
manifold-pool-3-421
manifold-pool-3-422
manifold-pool-3-423
manifold-pool-3-424

Etc.

I have also seen higher numbers for the first number (up to 43). Maybe
that's because of me reloading the app.

Post by Zach Tellman
What are the names of the threads?

--
You received this message because you are subscribed to the Google Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send an
For more options, visit https://groups.google.com/d/optout.

Zach Tellman

2015-12-05 22:41:28 UTC

Permalink

So based on the name, I gather that you're calling
`instrumented-executor`. This pool is dynamically resized, which means
that if its size is reduced (by killing threads), and then grows, new
threads will be created. If you dislike this, you can increase the
:control-period parameter when creating the executor.

Hope that helps,
Zach

Post by Jeroen van Dijk
manifold-pool-3-419
manifold-pool-3-420
manifold-pool-3-421
manifold-pool-3-422
manifold-pool-3-423
manifold-pool-3-424
Etc.
I have also seen higher numbers for the first number (up to 43). Maybe
that's because of me reloading the app.

Post by Zach Tellman
What are the names of the threads?
On Thu, Dec 3, 2015 at 5:32 AM Jeroen van Dijk <

Post by Jeroen van Dijk
Hi,
When I look at the Threads tab within jvisualvm I see a relative high
number of new manifold threads being generated. Other libraries like
core.async and httpkit seem to reuse threads (more). The Manifold threads
have a short live span and every second, roughly, 10 threads are being
created. This happens during a local load test of around 900 QPS.
Could I be doing something wrong or this is by design of Aleph/Manifold?
Thanks,
Jeroen
--
You received this message because you are subscribed to the Google
Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

Jeroen van Dijk

2015-12-07 08:44:15 UTC

Permalink

Thanks Zach. I'm not directly calling instrumented-executor. I'm using
Aleph as a webserver for normal ring handlers and there is just one SSE
endpoint (via a manifold stream). The SSE endpoint has only one
consumer normally.

If you think the thread pool resizing is normal behaviour, then I'll
continue and test it in a more production like environment.

Thanks,
Jeroen

Post by Zach Tellman
So based on the name, I gather that you're calling
`instrumented-executor`. This pool is dynamically resized, which means
that if its size is reduced (by killing threads), and then grows, new
threads will be created. If you dislike this, you can increase the
:control-period parameter when creating the executor.
Hope that helps,
Zach

Post by Zach Tellman
What are the names of the threads?
On Thu, Dec 3, 2015 at 5:32 AM Jeroen van Dijk <

Post by Jeroen van Dijk
Hi,
When I look at the Threads tab within jvisualvm I see a relative high
number of new manifold threads being generated. Other libraries like
core.async and httpkit seem to reuse threads (more). The Manifold threads
have a short live span and every second, roughly, 10 threads are being
created. This happens during a local load test of around 900 QPS.
Could I be doing something wrong or this is by design of Aleph/Manifold?
Thanks,
Jeroen
--
You received this message because you are subscribed to the Google
Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.

Zach Tellman

2015-12-07 17:04:26 UTC

Permalink

Ah, apologies for putting it on you, the culprit is
https://github.com/ztellman/aleph/blob/master/src/aleph/http/server.clj#L422.
I'll give some thought as to whether I want to tweak the defaults, but at
the very least this behavior is harmless.

Post by Jeroen van Dijk
Thanks Zach. I'm not directly calling instrumented-executor. I'm using
Aleph as a webserver for normal ring handlers and there is just one SSE
endpoint (via a manifold stream). The SSE endpoint has only one
consumer normally.
If you think the thread pool resizing is normal behaviour, then I'll
continue and test it in a more production like environment.
Thanks,
Jeroen

Post by Zach Tellman
What are the names of the threads?
On Thu, Dec 3, 2015 at 5:32 AM Jeroen van Dijk <

Post by Jeroen van Dijk
Hi,
When I look at the Threads tab within jvisualvm I see a relative high
number of new manifold threads being generated. Other libraries like
core.async and httpkit seem to reuse threads (more). The Manifold threads
have a short live span and every second, roughly, 10 threads are being
created. This happens during a local load test of around 900 QPS.
Could I be doing something wrong or this is by design of
Aleph/Manifold?
Thanks,
Jeroen
--
You received this message because you are subscribed to the Google
Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send
For more options, visit https://groups.google.com/d/optout.