BI and 'What-if's
Locked
 
|
Thank you all for your answers about my SQL question in another thread.
As our negotiations with our customer are evolving, the latest objection we are facing now is about these Business Intelligence systems. Running fancy reports and running 'What-if' scenarios are now considered important features. Again this is not based on proven needs... From an architecture point of view, I was wondering how you use GemStone to run dynamic data scenarios if you export to SQL. Do you export enough data such that a BI tool could take over? Or do you have some loops back to GemStone to play with the data? Or do you run What-if scenarios directly in GemStone? In other words, I am interested in stories about how GemStone/S and BI systems cohabit together. ( I apologize if this is not a purely technical question about GemStone/S for this forum. ) Thierry Thelliez |
|
|
Thierry,
We use tools created with VW graph support to analyze data and logs. The graphing tools were amazingly capable of analyzing data, but the functionality
isn't used much.
I get the feeling your customer is asking "for the moon" with little regard to what is truly necessary--often it is a delay tactic where you pay the cost
of a decision they are unable to make. Consider a focus on the simple stuff that gives them data they can play with themselves later if they really need to. If they aren't happy with that then feel grateful that you didn't waste your time adding useless features
only to find out that they'd have backed out anyway.
We use dynamic queries in some places. Our dynamic queries are currently programmer-specified and used on recently added functions. If there was a window
that configures a query from an interface definition then end users could create their own dynamic queries and export to their choice.
This is how our dynamic queries work...
A row object is created that contains a collection of attribute paths. An attribute path is a sequence of unary messages that would answer the value of
one attribute for a result row. The row is applied to a collection of objects to gather a sequence of attributes (into something like a single GS array). The row object knows of the result array and also acts like an iterator of that array. You can treat the
row like a stream to ask things like #next to jump the pointer to the next "row" of data. The Row object has multiple responsibilities, but works well. One responsibility could be to write all data to a SQL table or CSV file.
Prior to this design we created a marshal object for each "row" result. This proved to be much slower than creating one result array and one row iterator.
The new design is dynamic, fast, and easy to use. It replicates to client images very quickly.
Here is an advanced application that probably doesn't apply to your needs, but which you may find interesting...
Since the Row is an iterator, you can design it for parallel processing between VW and GS. VW creates a row and applies (in GS) to a collection. GS fills
the first set of results, forks a process that would fetch the remainder, and responds immediately with first results (the forked GS activity is suspended). VW realizes more would be gathered and forks a VW process that returns control back to GS to do work
up to one second before returning back to VW (to update the result collection replicate in VW). At this point, both VW and GS are doing work at the same time with only delays for object transport. A semaphore is used in VW to wait (if necessary) for rows from
GS. I've left out details related to replication efficiency and lifecycle, but you should get the idea. It allows VW to always remain responsive and achieves parallel work between VW and Gem.
Paul Baumann
From: [hidden email] [mailto:[hidden email]] On Behalf Of Thierry Thelliez Sent: Tuesday, August 31, 2010 1:54 PM To: GemStone Smalltalk Customer Forum Subject: [gemstone-smalltalk] BI and 'What-if's As our negotiations with our customer are evolving, the latest objection we are facing now is about these Business Intelligence systems. Running fancy reports and running 'What-if' scenarios are now considered important features. Again this is not based on proven needs... From an architecture point of view, I was wondering how you use GemStone to run dynamic data scenarios if you export to SQL. Do you export enough data such that a BI tool could take over? Or do you have some loops back to GemStone to play with the data? Or do you run What-if scenarios directly in GemStone? In other words, I am interested in stories about how GemStone/S and BI systems cohabit together. ( I apologize if this is not a purely technical question about GemStone/S for this forum. ) Thierry Thelliez This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of IntercontinentalExchange, Inc. (ICE), its subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired. |
«
Return to Gemstone/S
|
1 view|%1 views
| Free forum by Nabble | Edit this page |