Sunday, March 11, 2018

FW: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

-----Original Message-----
From: Atita Arora [mailto:atitaarora@gmail.com]
Sent: 11 March 2018 23:57
To: solr-user@lucene.apache.org
Subject: CLUSTERSTATUS API and Error loading specified collection / config
in Solr 5.3.2.

Hi ,

I am working on an application which involves working on a highly
distributed Solr cloud environment. The application supports multi-tenancy
and we have around 250-300 collections on Solr where each client has their
own collection with a new shard being created as clientid-<timestamp_long>
where the timestamp is whenever the new data comes in for the client
(typically every 4-8 hrs) , the reason for this convention is to make sure
when the Indexes are being built (on demand) the timestamp matches closely
to the time when the last indexing was run (the earlier shard is
de-provisioned as soon as the new one is created). Whenever the indexing is
triggered it first makes a DB entry and then creates a catalog with
timestamp in solr.
The Solr cloud has 10 Nodes distributed geographically among 10 datacenters.
The replication factor is 2. The Solr version is 5.3.2.
Coming to my problem - I had to write a utility to ensure that the DB insert
timestamp matches closely to the Solr index timestamp wherein I can ensure
that if the difference between DB timestamp and Solr Index tinestamp is <= 2
hrs , we have fresh index. The new index contains revised prices of products
or offers etc which are critical to be updated as in when they come. Hence
this utility is to track that the required updates have been successfully
made.
I used *CLUSTERSTATUS* api for this task. It is serving the purpose well so
far , but pretty recently our solr cloud started complaining of strange
things because of which the *CLUSTERSTATUS* api keeps returning as error.

The error claims to be of missing config & sometime missing collections
like.

org.apache.solr.common.SolrException: Could not find collection :
> 1785-1520548816454

org.apache.solr.common.SolrException: Could not find collection :
1785-1520548816454
at
org.apache.solr.common.cloud.ClusterState.getCollection(ClusterState.java:16
5)
at
org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(ClusterStatus.j
ava:110)
at
org.apache.solr.handler.admin.CollectionsHandler$CollectionOperation$19.call
(CollectionsHandler.java:614)
at
org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(Collectio
nsHandler.java:166)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.
java:143)
at
org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:67
8)
at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
215)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:
179)
at
org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler
.java:1652)
at
org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)

The other times it would complain of missing the config for same or
different client id- timestamp like :

1532-1518669619526_shard1_replica3:
org.apache.solr.common.cloud.ZooKeeperException:org.apache.solr.common.cloud
.ZooKeeperException:
Specified config does not exist in ZooKeeper:1532-1518669619526I

I would really appreciate if :


1. Someone can possibly guide me as to whats going on Solr Cloud
2. If CLUSTERSTATUS is the right pick to build such utility. Do we have
any other option?


Thanks for any pointers and suggestions.

Appreciate your attention looking this through.

Atita

No comments:

Post a Comment