-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: 15 March 2018 20:22
To: solr-user@lucene.apache.org
Subject: Re: solr query
We have an index with thousands of fields. Only a few are accessed on each
query.
These fields break out three different kinds of weights for a thousand or so
different school subjects. Each query is just for one subject, so the
scoring uses those three fields. Like:
* weight_a_1234
* weight_b_1234
* weight_c_1234
Another query would use:
* weight_a_4567
* weight_b_4567
* weight_c_4567
And so on. Lots of fields, but queries run in 15 milliseconds even with
updates every few minutes.
wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/ (my blog)
> On Mar 15, 2018, at 2:51 AM, Stefan Matheis <matheis.stefan@gmail.com>
wrote:
>
>> Is this practical adding so much additional fields?
>
> Well, as always "it depends" .. the way I see it: what are 20 fields?
> They just sit around and make your life way easier.
>
> You have two choices: stay with one field and have a hard time ... or
> add another 20 or so which do exactly what you want.
>
> To complete this with another use case: full text search including
> phonetic matches. Instead of just having _one_ field for "name" and a
> awful list of analyzers on it ... I'd probably go with a bunch of
> them: Separate first- and last name, another two (one for each part of
> the name) for lowercase variants, another two for phonetic variants ..
> which already makes six. You get the possibility to do boosting on
> those aspects more or less for free ..
>
> Almost always it's write once, read often .. so do the heavy lifting
> while indexing and enjoy simple queries.
>
> - Stefan
>
> On Mar 15, 2018 8:12 AM, "Albert Lee" <albertlee8901@gmail.com> wrote:
>
>> Cause I got about 20 date fields or more. If add a separate field for
>> it, then I have to add additional 3 field for each of them. For
>> example, for the field birthdate, I need to add birthdate_year,
>> birthdate_month, birthdate_day.
>> Is this practical adding so much additional fields?
>>
>> Albert
>> From: Stefan Matheis
>> Sent: Thursday, March 15, 2018 3:05 PM
>> To: solr-user@lucene.apache.org
>> Subject: RE: solr query
>>
>>> You have any other idea?
>>
>> Yes, we go back to start and discuss again why you're not adding a
>> separate field for that. It's the simplest thing possible and avoids
>> all those workarounds that got mentioned.
>>
>> -Stefan
>>
>> On Mar 15, 2018 4:08 AM, "Albert Lee" <albertlee8901@gmail.com> wrote:
>>
>>> Hi Emir,
>>>
>>> If using OR-ed conditions for different years then the query will be
>>> very long if I got 100 years and I think this is not practical.
>>> You have any other idea?
>>>
>>> Regards,
>>> Albert
>>> From: Gus Heck
>>> Sent: Thursday, March 15, 2018 12:43 AM
>>> To: solr-user@lucene.apache.org
>>> Subject: Re: solr query
>>>
>>> I think you have inadvertently "corrected" the intentional exclusive
>>> end
>> on
>>> my range... [NOW/MONTH TO NOW/MONTH+1MONTH}
>>>
>>> On Wed, Mar 14, 2018 at 12:08 PM, Emir Arnautović <
>>> emir.arnautovic@sematext.com> wrote:
>>>
>>>> Hi Gus,
>>>> It is just current month, but Albert is interested in month,
>>>> regardless
>>> of
>>>> year. It can be done with OR-ed conditions for different years:
>>>> birthDate:[NOW/MONTH TO NOW/MONTH+1MONTH] OR
>>>> birthDate:[NOW-1YEAR/MONTH
>>> TO
>>>> NOW-1YEAR/MONTH+1MONTH] OR birthDate:[NOW-2YEAR/MONTH TO
>>>> NOW-2YEAR/MONTH+1MONTH] OR.
>>>>
>>>> Emir
>>>> --
>>>> Monitoring - Log Management - Alerting - Anomaly Detection Solr &
>>>> Elasticsearch Consulting Support Training -
>> http://sematext.com/
>>>>
>>>>
>>>>
>>>>> On 14 Mar 2018, at 16:55, Gus Heck <gus.heck@gmail.com> wrote:
>>>>>
>>>>> I think you can specify the current month with
>>>>>
>>>>> birthDate:[NOW/MONTH TO NOW/MONTH+1MONTH}
>>>>>
>>>>> does that work for you?
>>>>>
>>>>> On Wed, Mar 14, 2018 at 6:32 AM, Emir Arnautović <
>>>>> emir.arnautovic@sematext.com> wrote:
>>>>>
>>>>>> Actually you don't have to add another field - there is function
>>>>>> ms
>>> that
>>>>>> converts date to timestamp. What you can do is use frange query
>> parser
>>>> and
>>>>>> play bit with math, e.g. sub(ms(date_field),ms(NOW/YEAR)) will
>>>>>> give
>>> you
>>>>>> ms elapsed since this year and you know that from 0 to
>>>>>> 31*86400000
>> is
>>>>>> January, from 31*86400000+1 to . is February and so on.
>>>>>>
>>>>>> If you go this path, I would suggest custom function that will
>> convert
>>>>>> date to month/year.
>>>>>>
>>>>>> HTH,
>>>>>> Emir
>>>>>> --
>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection Solr &
>>>>>> Elasticsearch Consulting Support Training -
>>> http://sematext.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On 14 Mar 2018, at 10:53, Albert Lee <albertlee8901@gmail.com>
>>> wrote:
>>>>>>>
>>>>>>> I don't want to add separate fields since I have many dates to
>> index.
>>>>>> How to index it as timestamp and do function query, any example
>>>>>> or documentation?
>>>>>>>
>>>>>>> Regards,
>>>>>>> Albert
>>>>>>>
>>>>>>> From: Emir Arnautović
>>>>>>> Sent: Wednesday, March 14, 2018 5:38 PM
>>>>>>> To: solr-user@lucene.apache.org
>>>>>>> Subject: Re: solr query
>>>>>>>
>>>>>>> Hi Albert,
>>>>>>> The simplest solution is to index month/year as separate fields.
>>>>>> Alternative is to index it as timestamp and do function query to
>>>>>> do
>>> some
>>>>>> math and filter out records.
>>>>>>>
>>>>>>> Emir
>>>>>>> --
>>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection Solr
>>>>>>> & Elasticsearch Consulting Support Training -
>>>> http://sematext.com/
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> On 14 Mar 2018, at 10:31, Albert Lee <albertlee8901@gmail.com>
>>> wrote:
>>>>>>>>
>>>>>>>> NOW/MONTH and NOW/YEAR to get the start of month/year, but how
>> can I
>>>>>> get current month of regardless year. Like the use case, people
>> who's
>>>>>> birthdate is this month?
>>>>>>>>
>>>>>>>> Regard,
>>>>>>>> Albert
>>>>>>>>
>>>>>>>>
>>>>>>>> From: Emir Arnautović
>>>>>>>> Sent: Wednesday, March 14, 2018 5:26 PM
>>>>>>>> To: solr-user@lucene.apache.org
>>>>>>>> Subject: Re: solr query
>>>>>>>>
>>>>>>>> Hi Albert,
>>>>>>>> It does - you can use NOW/MONTH and NOW/YEAR to get the start
>>>>>>>> of
>>>>>> month/year. Here is reference to date math:
>>> https://lucene.apache.org/
>>>>>> solr/guide/6_6/working-with-dates.html#WorkingwithDates-
>>> DateMathSyntax
>>>> <
>>>>>> https://lucene.apache.org/solr/guide/6_6/working-with-
>>>>>> dates.html#WorkingwithDates-DateMathSyntax>
>>>>>>>>
>>>>>>>> HTH,
>>>>>>>> Emir
>>>>>>>> --
>>>>>>>> Monitoring - Log Management - Alerting - Anomaly Detection Solr
>>>>>>>> & Elasticsearch Consulting Support Training -
>>>> http://sematext.com/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 14 Mar 2018, at 04:21, Albert Lee <albertlee8901@gmail.com>
>>>> wrote:
>>>>>>>>>
>>>>>>>>> Dear Solr,
>>>>>>>>> I want to whether solr support query by this year or this month?
>>>>>>>>> If can, how to do that.
>>>>>>>>> Thanks.
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>> Albert
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> http://www.the111shift.com
>>>>
>>>>
>>>
>>>
>>> --
>>> http://www.the111shift.com
>>>
>>>
>>
>>
No comments:
Post a Comment