FW: Solr 4.9 - configs and collections

-----Original Message-----
From: Abhi Basu [mailto:9000revs@gmail.com]
Sent: 26 March 2018 20:13
To: solr-user@lucene.apache.org
Subject: Solr 4.9 - configs and collections

Running on MS HDInsight and Solr 4.9.What is the BKM for creation, update,
delete of configurations and collections?

I do the following:

1. First I create the zk config:
sudo zkcli.sh -cmd upconfig -zkhost zknode
<http://zk1-esohad.tzun3mpncofedp04lr3ird23xc.jx.internal.cloudapp.net/>:218
1
-confdir /home/sshuser/ems-collection-49/conf/ -confname ems-collection

2. Then I create the collection:
curl '
http://headnode0:8983/solr/admin/collections?action=CREATE&name=ems-collecti
on&numShards=2&replicationFactor=2&maxShardsPerNode=1
'

This works the first time. When I change the zk config, do I run the same
command #1? Also, do I do a reload:

curl '
http://headnode0:8983/solr/admin/collections?action=RELOAD&name=ems-collecti
on
'

Or, do I need to delete and recreate the collection?


Very familiar with CDH solrctl commands that make life easier by only having
one command for this. Any help is appreciated.

Thanks,

Abhi

--
Abhi Basu

FW: Phrase search with Solr 7.2

-----Original Message-----
From: Steven White [mailto:swhite4141@gmail.com]
Sent: 26 March 2018 20:20
To: solr-user@lucene.apache.org
Subject: Re: Phrase search with Solr 7.2

Please ignore this. It was a user error. I was pointing to the wrong
analyzer in my app's cfg file.

Steve

On Mon, Mar 26, 2018 at 10:17 AM, Steven White <swhite4141@gmail.com> wrote:

> Setting "sow=true" didn't make a difference.
>
> Here is what I'm using now: http://localhost:8983/
> solr/ccfts/select_test?q=%22record%20type%20session%22&
> wt=json&indent=true&sow=true&debugQuery=true
>
> And here is the output:
>
> {
> "responseHeader":{
> "status":0,
> "QTime":1,
> "params":{
> "q":"\"record type session\"",
> "indent":"true",
> "sow":"true",
> "wt":"json",
> "debugQuery":"true"}},
> "response":{"numFound":0,"start":0,"maxScore":0.0,"docs":[]
> },
> "debug":{
> "rawquerystring":"\"record type session\"",
> "querystring":"\"record type session\"",
> "parsedquery":"+DisjunctionMaxQuery((CC_ALL_FIELDS_DATA:\"record type
session\")~1.0)",
> "parsedquery_toString":"+(CC_ALL_FIELDS_DATA:\"record type
session\")~1.0",
> "explain":{},
> "QParser":"ExtendedDismaxQParser",
> "altquerystring":null,
> "boost_queries":null,
> "parsed_boost_queries":[],
> "boostfuncs":null,
> "timing":{
> "time":1.0,
> "prepare":{
> "time":1.0,
> "query":{
> "time":0.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "terms":{
> "time":0.0},
> "debug":{
> "time":0.0}},
> "process":{
> "time":0.0,
> "query":{
> "time":0.0},
> "facet":{
> "time":0.0},
> "facet_module":{
> "time":0.0},
> "mlt":{
> "time":0.0},
> "highlight":{
> "time":0.0},
> "stats":{
> "time":0.0},
> "expand":{
> "time":0.0},
> "terms":{
> "time":0.0},
> "debug":{
> "time":0.0}}}}}
>
>
> How do I debug this?
>
> Steve
>
> On Mon, Mar 26, 2018 at 12:50 AM, Mikhail Khludnev <mkhl@apache.org>
> wrote:
>
>> Hello, Steven.
>>
>> Have you tried sow=true?
>> see
>> https://lucene.apache.org/solr/guide/7_2/the-extended-dismax
>> -query-parser.html
>>
>>
>> Anyway, you can start from debugQuery=true, then try to explore
>> explainOther, and get to Analysis page after all.
>>
>> On Mon, Mar 26, 2018 at 3:10 AM, Steven White <swhite4141@gmail.com>
>> wrote:
>>
>> > Hi everyone,
>> >
>> > I switched over from Solr 5.2.1 to 7.2.1 other than re-indexing my
>> > data
>> and
>> > schema design remain the same.
>> >
>> > The issue I see now is I'm getting 0 hits on phrase searches, why?
>> >
>> > Here is the query I'm sending that gives me 0 hits:
>> >
>> > http://localhost:8983/solr/ccfts/select_test?q=%22cat+
>> > dog%22&wt=json&indent=true
>> >
>> > But this query will give me hits:
>> >
>> > http://localhost:8983/solr/ccfts/select_test?q=cat+dog&wt=
>> json&indent=true
>> >
>> > Here is my schema:
>> >
>> > <fieldType name="analyzer_text" class="solr.TextField"
>> > positionIncrementGap="100" autoGeneratePhraseQueries="true">
>> > <analyzer>
>> > <tokenizer class="solr.WhitespaceTokenizerFactory"/>
>> > <filter class="solr.SynonymFilterFactory" expand="true"
>> > synonyms="synonyms.txt" ignoreCase="true"/>
>> > <filter class="solr.WordDelimiterFilterFactory" preserveOriginal="0"
>> > generateNumberParts="1" splitOnCaseChange="0" catenateWords="1"
>> > splitOnNumerics="1" stemEnglishPossessive="1" generateWordParts="1"
>> > catenateAll="1" catenateNumbers="1"/>
>> > <filter class="solr.StopFilterFactory" words="lang/stopwords_en.txt"
>> > ignoreCase="true"/>
>> > <filter class="solr.LowerCaseFilterFactory"/>
>> > <filter class="solr.EnglishPossessiveFilterFactory"/>
>> > <filter class="solr.KeywordMarkerFilterFactory"
>> > protected="protwords.txt"/>
>> > <filter class="solr.PorterStemFilterFactory"/>
>> > <filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
>> > </analyzer>
>> > </fieldType>
>> >
>> > Here are my fields:
>> >
>> > <field name=" analyzer _text" type="text_general" indexed="true"
>> > required="false" stored="false" multiValued="true" />
>> > <field name="CC_UNIQUE_FIELD" type="string"
>> indexed="true"
>> > required="true" stored="true" multiValued="false" />
>> > <field name="CC_ALL_FIELDS_DATA" type=" analyzer_text" indexed="true"
>> > required="false" stored="false" multiValued="true" />
>> > <field name="CC_COMPONENT_NAME" type="string"
>> indexed="true"
>> > required="true" stored="false" multiValued="false" docValues="true"
/>
>> > <field name="CC_FILE_EXT" type="string"
>> indexed="true"
>> > required="true" stored="false" multiValued="false" docValues="true"
/>
>> > <field name="CC_FILE_NAME" type="string"
>> indexed="true"
>> > required="true" stored="false" multiValued="false" docValues="true"
/>
>> > <field name="CC_FILE_PATH" type="string"
>> indexed="true"
>> > required="true" stored="true" multiValued="false" docValues="true"
/>
>> >
>> > And here is my handler:
>> >
>> > {"requestHandler":{"/select_test":{
>> > "class":"solr.SearchHandler",
>> > "name":"/select_test",
>> > "defaults":{
>> > "defType":"edismax",
>> > "echoParams":"explicit",
>> > "fl":"CC_UNIQUE_FIELD,CC_FILE_PATH,score",
>> > "indent":"true",
>> > "qf":"CC_ALL_FIELDS_DATA",
>> > "rows":"10",
>> > "tie":"1.0",
>> > "wt":"xml"}}}}
>> >
>> > What am I dong wrong?
>> >
>> > Steven
>> >
>>
>>
>>
>> --
>> Sincerely yours
>> Mikhail Khludnev
>>
>
>

FW: Indexing multi level Nested JSON using curl

-----Original Message-----
From: Zheng Lin Edwin Yeo [mailto:edwinyeozl@gmail.com]
Sent: 26 March 2018 15:00
To: solr-user@lucene.apache.org
Subject: Indexing multi level Nested JSON using curl

Hi,

I'm trying to index the following JSON with 2 child level using the
following curl command using cygwin:

curl 'http://localhost:8983/solr/collection1/update/json/docs?split=/|/orgs'
-H 'Content-type:application/json' -d '
{
"id":"1",
"name_s": "JoeSmith",
"phone_s": 876876687,
"orgs": [
{
"name1_s" : "Microsoft",
"city_s" : "Seattle",
"zip_s" : 98052,
"orgs":[{"name2_ss":"alan","phone2_ss":"123"},{"name2_ss":
"edwin","phone2_ss":"456"}]
},
{
"name1_s" : "Apple",
"city_s" : "Cupertino",
"zip_s" : 95014,
"orgs":[{"name2_ss":"alan","phone2_ss":"123"},{"name2_ss":
"edwin","phone2_ss":"456"}]
}
]
}'

However, after indexing, this is what is shown in Solr. The 2nd child have
been place together under the 1st child as a multi-valued field, which is
wrong. If I have set the field for the 2nd child to be non-multi-valued
field, it will have error saying "multiple values encountered for non
multiValued field orgs2.name2_s:".

{
"responseHeader":{
"zkConnected":true,
"status":0,
"QTime":41,
"params":{
"q":"phone_s:876876687",
"fl":"*,[child parentFilter=phone_s:876876687]",
"sort":"id asc"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"1",
"name_s":"JoeSmith",
"phone_s":"876876687",
"language_s":"en",
"_version_":1595632041779527680,
"_childDocuments_":[
{
"name1_s":"Microsoft",
"city_s":"Seattle",
"zip_s":"98052",
"orgs.name2_ss":["alan",
"edwin"],
"orgs.phone2_ss":["123",
"456"],
"_version_":1595632041779527680},
{
"name1_s":"Apple",
"city_s":"Cupertino",
"zip_s":"95014",
"orgs.name2_ss":["alan",
"edwin"],
"orgs.phone2_ss":["123",
"456"],
"_version_":1595632041779527680}]}]
}}


How can we structure the curl command so it will be able to accept child of
child relationship? We should not be doing any pre-processing to the JSON to
achieve that.

I'm using Solr 7.2.1.

Regards,
Edwin

FW: edit gc parameters in solr.in.sh or solr?

-----Original Message-----
From: Derek Poh [mailto:dpoh@globalsources.com]
Sent: 26 March 2018 13:53
To: solr-user@lucene.apache.org
Subject: edit gc parameters in solr.in.sh or solr?

Hi

From your experience, would like to know if It is advisable to change the gc
parameters in solr.in.sh or solrfile?
It is mentioned in the documentation to edit solr.in.sh but would like
toknow which file you actually edit.

I am using Solr 6.6.2at the moment.

Regards,
Derek


----------------------
CONFIDENTIALITY NOTICE

This e-mail (including any attachments) may contain confidential and/or
privileged information. If you are not the intended recipient or have
received this e-mail in error, please inform the sender immediately and
delete this e-mail (including any attachments) from your computer, and you
must not use, disclose to anyone else or copy this e-mail (including any
attachments), whether in whole or in part.

This e-mail and any reply to it may be monitored for security, legal,
regulatory compliance and/or other appropriate reasons.

FW: Solr 7 or 6 - stability and performance

-----Original Message-----
From: S G [mailto:sg.online.email@gmail.com]
Sent: 26 March 2018 09:26
To: solr-user@lucene.apache.org
Subject: Solr 7 or 6 - stability and performance

Hi,

Solr 7 has been out for about 6 months now. (Sep-2017 to Mar-2018) We are
planning some major upgrades from 6.2 and 6.4 versions of Solr and I wanted
to see how is Solr 7 looking in terms of stability and performance.
(Have seen http://lucene.apache.org/solr/news.html but some real users'
experience would be nice)

1) Has anyone encountered major stability issues that made them move back to
6.x version?

2) Did anyone see more than 10% change in performance (good or bad)? I know
about https://issues.apache.org/jira/browse/SOLR-11078 and wish trie fields
were still kept in schema until point fields completely got over the
performance issue.

Thanks
SG

FW: Phrase search with Solr 7.2

-----Original Message-----
From: Steven White [mailto:swhite4141@gmail.com]
Sent: 26 March 2018 05:40
To: solr-user@lucene.apache.org
Subject: Phrase search with Solr 7.2

Hi everyone,

I switched over from Solr 5.2.1 to 7.2.1 other than re-indexing my data and
schema design remain the same.

The issue I see now is I'm getting 0 hits on phrase searches, why?

Here is the query I'm sending that gives me 0 hits:

http://localhost:8983/solr/ccfts/select_test?q=%22cat+dog%22&wt=json&indent=
true

But this query will give me hits:

http://localhost:8983/solr/ccfts/select_test?q=cat+dog&wt=json&indent=true

Here is my schema:

<fieldType name="analyzer_text" class="solr.TextField"
positionIncrementGap="100" autoGeneratePhraseQueries="true">
<analyzer>
<tokenizer class="solr.WhitespaceTokenizerFactory"/>
<filter class="solr.SynonymFilterFactory" expand="true"
synonyms="synonyms.txt" ignoreCase="true"/>
<filter class="solr.WordDelimiterFilterFactory" preserveOriginal="0"
generateNumberParts="1" splitOnCaseChange="0" catenateWords="1"
splitOnNumerics="1" stemEnglishPossessive="1" generateWordParts="1"
catenateAll="1" catenateNumbers="1"/>
<filter class="solr.StopFilterFactory" words="lang/stopwords_en.txt"
ignoreCase="true"/>
<filter class="solr.LowerCaseFilterFactory"/>
<filter class="solr.EnglishPossessiveFilterFactory"/>
<filter class="solr.KeywordMarkerFilterFactory"
protected="protwords.txt"/>
<filter class="solr.PorterStemFilterFactory"/>
<filter class="solr.RemoveDuplicatesTokenFilterFactory"/>
</analyzer>
</fieldType>

Here are my fields:

<field name=" analyzer _text" type="text_general" indexed="true"
required="false" stored="false" multiValued="true" />
<field name="CC_UNIQUE_FIELD" type="string" indexed="true"
required="true" stored="true" multiValued="false" />
<field name="CC_ALL_FIELDS_DATA" type=" analyzer_text" indexed="true"
required="false" stored="false" multiValued="true" />
<field name="CC_COMPONENT_NAME" type="string" indexed="true"
required="true" stored="false" multiValued="false" docValues="true" />
<field name="CC_FILE_EXT" type="string" indexed="true"
required="true" stored="false" multiValued="false" docValues="true" />
<field name="CC_FILE_NAME" type="string" indexed="true"
required="true" stored="false" multiValued="false" docValues="true" />
<field name="CC_FILE_PATH" type="string" indexed="true"
required="true" stored="true" multiValued="false" docValues="true" />

And here is my handler:

{"requestHandler":{"/select_test":{
"class":"solr.SearchHandler",
"name":"/select_test",
"defaults":{
"defType":"edismax",
"echoParams":"explicit",
"fl":"CC_UNIQUE_FIELD,CC_FILE_PATH,score",
"indent":"true",
"qf":"CC_ALL_FIELDS_DATA",
"rows":"10",
"tie":"1.0",
"wt":"xml"}}}}

What am I dong wrong?

Steven

FW: Some performance questions....

-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: 26 March 2018 05:38
To: solr-user@lucene.apache.org
Subject: Re: Some performance questions....

> On Mar 24, 2018, at 5:21 PM, Deepak Goel <deicool@gmail.com> wrote:
>
> My first test was to test with static queries. Does Solr scale-up as
> we increase the load of same query?
>
> The second test would be to check with 'Different Queries'.
>
> And then finally check with 80% similar queries and 20% different queries.

You insulted me when I gave a clear explanation about how to run a
meaningful benchmark.

Now you give results from a totally invalid benchmark. If you are getting
slow responses from a one-query "benchmark", you have serious system or
configuration mistakes. That should be returning in a few milliseconds.

Also, 80/20 isn't even close to a realistic query load.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)