[Glass] Queries in Gemstone ...

... and I sometimes ask me, how this can be done in an efficient way in
Gemstone:

select x from y ordered by x.q where x.z = .... limit n


Get thy entries from set/bag y where the attribute z of the instance has
the value ... and the result should be ordered via attribute q and I
only need the first n entries.


Marten

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Marten,

If you were to use indexes, then you'd put an index on x.z, query on x.z
= ??? and then sort the result set on x.q.

If the result set for x.z = ??? is very large then then the cost of
sorting the result can outweigh the advantage of the quick initial lookup.

You could maintain a KeyValueDictionary where the keys map to the value
of x.z and the values are sorted collections or indexed collections. Use
an index if the x.q value is expected to change.

If the value of x.q is likely to change and you need automatic updates,
you can arrange to use modification tracking to update the dictionary
when x.q changes ...

Dale


On 2/8/15 7:02 AM, [hidden email] via Glass wrote:
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Yes, I know ... but for a simple question this seems to be lots of work
and with the introduction of the new query manager in 3.2.3 I hope that
Gemstone does more work possible in this direction ...

Am 08.02.2015 um 17:46 schrieb Dale Henrichs:

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

On 2/8/15 10:21 AM, [hidden email] wrote:
> Yes, I know ... but for a simple question this seems to be lots of work
> and with the introduction of the new query manager in 3.2.3 I hope that
> Gemstone does more work possible in this direction ...
>
Marten,

Since you appear to be disappointed with my "put an index on x.z, query
on x.z = ??? and then sort the result set on x.q" answer, I am curious
what answer you were looking for?

You used the word "efficient" in your question and I assumed that you
meant "efficient" from a performance perspective, but perhaps you meant
"efficient" from a coding perspective?

If so, do you have suggestions for an API that would satisfy you?

Dale
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Well, the simplest stuff would be: execute a query and give me back the
results - if indices can be used: use them (coding perspective).

If its too slow the programmer must have some hints from the query
executing machine like it is on the relational database - then the
programmer can add indices as wanted to improve the situation
(performance perspective).

That's the idea I would like to see ... and which is not trivial.

Marten

Am 08.02.2015 um 20:22 schrieb Dale Henrichs:

--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Marten,

Okay, I was looking for examples of the actual api, I think I understand
what you want, but I was looking for suggestions as to how you would
express what you want.

If I look at your original message, again, I guess you were actually
making a feature request asking that we support a more complicated query
syntax? In other words, you were wondering if we would expand the query
syntax to support something like this:

    select x from y ordered by x.q where x.z = .... limit n

So are you thinking about something along the lines of OQL (Chapter 4 of
The Object Data Standard book[1]), SQL++[2], one of the other languages
mentioned in the Object-Oriented Query Languages survey[3] or something
else?

I think I understand the theoretical advantage of having an
Object-Oriented Query Language, but from a practical perspective, I am
wondering about the incremental advantage of having a query language
beyond being able to write simple Smalltalk code to augment the existing
GemStone query language.

It seems to me that often the right answer is to explicitly use a
completely different object representation than to provide a (limited)
palette of standard data structures accessed via a standard query language.

The GemStone query language basically replaces the select/reject/detect
expression by providing an underlying representation that is more
efficient for doing equality/range comparisons.

The "ordered by" clause implies that a sort block be used and given an
arbitrary "where" clause, the "ordered by" cannot be optimized by
pre-calculation (the trick applied by GemStone indexes)...so there's not
much that we can do here ...

I think there might be areas where we could "do better", for example if
you included a "in range of s to t by u" in a query:

   where x.z = .... in range of s to t by u

I can see where some significant optimizations could be made, but I
don't know if such corner cases exist in the wild.

Soooo, in a nutshell, I am wondering about the actual opportunity for
being able to "do better than a good programmer" by expanding the query
language ...

Dale


[1] http://www.odbms.org/odmg-standard/odmg-book/odmg-2-0-book-extract/
[2]
http://www.objectivity.com/products/objectivitydb/features/sql/#.VNfMnbDF-rw
[3] http://www.cs.cornell.edu/home/ulfar/oodbms/ooqlang.html

On 2/8/15 12:14 PM, [hidden email] wrote:
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Dale,

yes indeed it was more an indirect feature request :-).

With my REST system here I have nearly always this problem and it would
be nicer to create a (regardless of syntax) query and let the system do
the work. That would speed up the initial development time very well.

Actually in so many data-query-requests it ends up coding Smalltalk code
which actually do something like:

-> get request

-> code Smalltalk based on this request to do something like

select x from y
  [ where ...... ]
  [
    ordered by x.q
    [ limit n  ]
    [ offset m ]
  ]

-> serialize the result via json

-> answer request

At least these dammned paged request (limit and offset: typically
supported by all UI browser frameworks) are candidates for expensive
statements.

And yes: manual coding would give better results ... and yes I have to
think for myself to produce better code when having limit and offset
options.

I just wanted to say, what pattern I find very frequently when writing
here REST oriented systems.


Marten


--
Marten Feldtmann
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass

Marten,

I've submitted an internal feature request (45014) .. not sure when I
will get to it as my plate is pretty full right now:)

Dale
On 2/9/15 12:27 AM, [hidden email] wrote:
_______________________________________________
Glass mailing list
[hidden email]
http://lists.gemtalksystems.com/mailman/listinfo/glass