FW: The Impact of the Number of Collections on Indexing Performance in Solr 6.0

-----Original Message-----
From: 苗海泉 [mailto:mseaspring@gmail.com]
Sent: 10 March 2018 11:22
To: solr-user <solr-user@lucene.apache.org>
Subject: The Impact of the Number of Collections on Indexing Performance in
Solr 6.0

hello,We found a problem. In solr 6.0, the indexing speed of solr is
influenced by the number of solr collections. The speed is normal before the
limit is reached. If the limit is reached, the indexing speed will decrease
by 50 times.

In our environment, there are 49 solr nodes. If each collection has 25
shards, you can maintain high-speed indexing until the total number of
collections is about 900. To reduce the number of collections to the limit,
the speed will increase. Go up.
If each collection is 49 shards, the total number of collections can only be
about 700, exceeding this value will cause the index to drop dramatically.
In the explanation, we are single copies, and multiple copies will cause
serious stability problems in the large solr cluster environment.

At first I suspect that it was due to too many thread submissions, and there
are still problems with this method, so I'm inclined to searcherExecutor
thread pool thread. This is just my guess, I want to know the real reason.
Can someone know if I can help?

Also, I noticed that the searcherExecutor thread and solr collection's
shards basically correspond to each other. How can I reduce the number of
threads or even close it? Although there are many collections in our
environment, there are few queries and it is not necessary to keep the
threads open to provide queries. This is too wasteful.

thank you .