FW: CLUSTERSTATUS API and Error loading specified collection / config in Solr 5.3.2.

-----Original Message-----
From: Atita Arora [mailto:atitaarora@gmail.com]
Sent: 13 March 2018 20:49
To: solr-user@lucene.apache.org
Subject: Re: CLUSTERSTATUS API and Error loading specified collection /
config in Solr 5.3.2.

Hi Hendrik and Shalin,

Really appreciate your valuable inputs on this.

I looked up to the two issues that were being referred to (SOLR-8804 and
SOLR-10720) and thats exactly what I'm running into.
Glad they have been fixed in later versions.

Thanks much ,
Atita

On Tue, Mar 13, 2018 at 10:38 AM, Shalin Shekhar Mangar <
shalinmangar@gmail.com> wrote:

> I think you are running into race conditions in the API which have
> been fixed. See SOLR-8804 and SOLR-10720. The first is available in
> 5.5.1 but the latter fix will be released in the upcoming 7.3 release.
> The best workaround for your version is to just retry a few times
> until the API succeeds.
>
> On Sun, Mar 11, 2018 at 11:57 PM, Atita Arora <atitaarora@gmail.com>
> wrote:
>
> > Hi ,
> >
> > I am working on an application which involves working on a highly
> > distributed Solr cloud environment. The application supports
> multi-tenancy
> > and we have around 250-300 collections on Solr where each client has
> their
> > own collection with a new shard being created as
> clientid-<timestamp_long>
> > where the timestamp is whenever the new data comes in for the client
> > (typically every 4-8 hrs) , the reason for this convention is to
> > make
> sure
> > when the Indexes are being built (on demand) the timestamp matches
> closely
> > to the time when the last indexing was run (the earlier shard is
> > de-provisioned as soon as the new one is created). Whenever the
> > indexing
> is
> > triggered it first makes a DB entry and then creates a catalog with
> > timestamp in solr.
> > The Solr cloud has 10 Nodes distributed geographically among 10
> > datacenters.
> > The replication factor is 2. The Solr version is 5.3.2.
> > Coming to my problem - I had to write a utility to ensure that the
> > DB insert timestamp matches closely to the Solr index timestamp
> > wherein I
> can
> > ensure that if the difference between DB timestamp and Solr Index
> tinestamp
> > is <= 2 hrs , we have fresh index. The new index contains revised
> > prices
> of
> > products or offers etc which are critical to be updated as in when
> > they come. Hence this utility is to track that the required updates
> > have been successfully made.
> > I used *CLUSTERSTATUS* api for this task. It is serving the purpose
> > well
> so
> > far , but pretty recently our solr cloud started complaining of
> > strange things because of which the *CLUSTERSTATUS* api keeps returning
as error.
> >
> > The error claims to be of missing config & sometime missing
> > collections like.
> >
> > org.apache.solr.common.SolrException: Could not find collection :
> > > 1785-1520548816454
> >
> > org.apache.solr.common.SolrException: Could not find collection :
> > 1785-1520548816454
> > at
> > org.apache.solr.common.cloud.ClusterState.getCollection(
> > ClusterState.java:165)
> > at
> > org.apache.solr.handler.admin.ClusterStatus.getClusterStatus(
> > ClusterStatus.java:110)
> > at
> > org.apache.solr.handler.admin.CollectionsHandler$
> > CollectionOperation$19.call(CollectionsHandler.java:614)
> > at
> > org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(
> > CollectionsHandler.java:166)
> > at
> > org.apache.solr.handler.RequestHandlerBase.handleRequest(
> > RequestHandlerBase.java:143)
> > at
> > org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(
> > HttpSolrCall.java:678)
> > at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:444)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:215)
> > at
> > org.apache.solr.servlet.SolrDispatchFilter.doFilter(
> > SolrDispatchFilter.java:179)
> > at
> > org.eclipse.jetty.servlet.ServletHandler$CachedChain.
> > doFilter(ServletHandler.java:1652)
> > at
> > org.eclipse.jetty.servlet.ServletHandler.doHandle(
> ServletHandler.java:585)
> >
> > The other times it would complain of missing the config for same or
> > different client id- timestamp like :
> >
> > 1532-1518669619526_shard1_replica3:
> > org.apache.solr.common.cloud.ZooKeeperException:org.apache.
> > solr.common.cloud.ZooKeeperException:
> > Specified config does not exist in ZooKeeper:1532-1518669619526I
> >
> > I would really appreciate if :
> >
> >
> > 1. Someone can possibly guide me as to whats going on Solr Cloud
> > 2. If CLUSTERSTATUS is the right pick to build such utility. Do
> > we have any other option?
> >
> >
> > Thanks for any pointers and suggestions.
> >
> > Appreciate your attention looking this through.
> >
> > Atita
> >
>
>
>
> --
> Regards,
> Shalin Shekhar Mangar.
>