SCM'ed VW

> Steven:
>
>> I think doing the export for weekly builds is overkill, because of  the
>> version numbering issue. I imagine most VW users aren't in the  vw-dev
>> program anyway, or at least aren't actively downloading each  week. It
>> would be enough to test the export in the release  candidates in vw-dev,
>> with the warning not to replicate them to  another db. Actually, there
>> seems no point replicating to another db  in any case, for users on
>> Windows: just keep the VW packages linked  only to the local SQLite
>> Cincom db, which can even be used by  multiple users (all access would be
>> read-only). (BTW A nice addition  to Store would be that if a package in
>> the image doesn't have  version info for the currently selected
>> repository, but does for one  other, it would automatically use that
>> repository for compares etc.)
>
>
> Ok, I may be a bit off on this, but as Store Lead, that's my job.
>
> Let's for a second take as a notion that for releases, we produce a
> SQLite DB file, with everything that is JUST in visual.im (leaving  aside
> issues with that for the moment). Of course, that means the  visual.im
> will have Store, A StoreFor(SQLite or ODBC) and a (SQLite or  ODBC)EXDI in
> it OR maybe a single Parcel you can load to bring that in  if you want.
> The details there aren't really important for this  thought experiment.
>
> Ok, you get your brand new 7.7 (for instance) and you load up/or  connect
> to the DB file.
>
> For the next 12-16 months or so, you use this database file, pushing  your
> changes to it and so on.
>
> Now 7.8 comes along with it's own DB file (we'll assume for the moment
> that it is named something different from the one that came with 7.7).
>
> From cursory investigation, SQLite allows you to attach to multiple  DB
> files.
>
> With that as our posit:
>
> It seems to me that at the very least, a mini-merge/replicator is  needed,
> which would take the 7.8 database and merge it into the 7.7  database (or
> visa versa). Details of where to merge it to aren't  really important at
> this stage (probably parent to "your choice" of  some kind).
>
> Is that good enough? Or
>
> Is a full merge/replicator which allows you to cross database target  the
> merge/replication to some other database (DB/2, Oracle, whatever)?
>
> And finally, is JUST visual.im enough?
>
> Should it be everything that is supported and not contributed?
>
>                                 And So It Goes
>                                      Sames
> ______________________________________________________________________
>
> Samuel S. Shuster [|]
> VisualWorks Engineering, Store Project
> Smalltalk Enables Success -- What Are YOU Using?
>
>
>
>
>

> I just tried StoreForSQLite3 with SQLite 3.5.4 DLL on Windows. 4
> minutes to publish BaseVisualWorks from 7.4.1, about 1.5 minutes to
> reconcile it. Results in a single 19MB file which zips to under 5MB.
> Adding all the VW parcels that we use takes another 3 minutes and
> bumps it up to 40MB, 10MB zipped. Reconciling 7.5 (with the same
> parcels) with that repository takes 4 minutes, opening all the publish
> dialogs for all 60 changed bundles or packages takes 1 minute (!),
> using hacky code to answer all of these dialogs automatically takes 3
> minutes. There was an error while installing the blessing for the
> first pundle, but proceeding and accepting to re-open the Store
> connection worked (so no worse than PostgreSQL over the
> Atlantic!). The final size of the db was 50MB. All of this was over
> Remote Desktop Connection (so possibly slowed down by cursor flicker,
> but I tried to minimize that), running on a P4 3GHz.
>  
> Reconciling 7.5 again as a check gave errors allocating Pixmaps when
> updating the DoubleBuffering display in Store's Notices: I guess
> SQLite3 or its use in VW is leaking handles, or then the large number
> of notices somehow overwhelmed finalization (or then there's a bug in
> them). I can't check now, as Windows ran out of resources and killed
> my remote session, and needs me to go back to the office to reboot it.
>  
> zipping the db gives the warning that the file is opened by another
> application, apparently even after disconnecting from Store. Sounds
> like that should be corrected; shouldn't be hard.
>  
> BTW, does anyone know who "olaf" is or have his contact info? Overall
> it looks like his parcels are pretty close to being just what the
> doctor ordered for a local Store db, and it would be good to check the
> licensing.
>  
> Also in the public repository: StoreForSqlite by gjdvgj, Jürgen Gmeiner.
> http://www.gjdv.at/software/store_for_sqlite
> LGPL, looks a couple of years older, for SQLite 2.8.6 rather than 3.3,
> but has a LOT more code. Whether this is good or bad remains to be
> seen! The web page has installation instructions, which were useful
> when trying out StoreForSQLite3.
>  
> Cheers,
> Steve
>
>     -----Original Message-----
>     *From:* Tim Hutchison [mailto:[hidden email]]
>     *Sent:* 23. tammikuuta 2008 18:50
>     *To:* VWNC
>     *Subject:* Re: SCM'ed VW
>
>     Currently in the Cincom Public Repository;
>     SQLite3EXDI-Preload (sqlite3-vw7.x-2,olaf)
>     SQLite3EXDI (sqlite3-vw7.x-2,olaf)
>     StoreForSQLite3 (sqlite3-vw7.x-2,olaf)
>
>     Could be that these may need a little more work.
>
>     Alan Knight wrote:
>
>         At 03:22 AM 1/23/2008, Steven Kelly wrote:
>
>>         Hi,
>>
>>             From: Alan Knight [ mailto:[hidden email]]
>>             Indeed, the reasons are all procedural, logistical, and
>>             financial :-)
>>
>>             At 03:52 PM 1/22/2008, Charles Adams wrote:
>>
>>>                 With all the talk about Store and the best database
>>>                 product to use, its time, once again, to bring up
>>>                 the issue of a published distribution. It is long
>>>                 past time for Cincom to produce versioned code in
>>>                 some sort of Store configuration. There's no
>>>                 technical, legal or even philosophical reason why
>>>                 this can't or shouldn't be done.
>>>
>>>                  
>>>                 If I tried to hand my customers files instead of
>>>                 software under configuration management, I'd be out
>>>                 of business. Look... I'm not going to stand here and
>>>                 make the case for SCM. We are all seasoned
>>>                 professionals. Its time for Cincom to acknowledge
>>>                 their responsibilities in this area. '
>>
>>
>>         I'd second the request, something I first asked for in 5i.3
>>         :-). Alan says "procedural, logistical and financial", so
>>         let's look at those.
>>          
>>         "Procedural" - you mean someone at Cincom would have to
>>         publish the base, or more likely replicate the release pundle
>>         lineup. Currently, every customer wanting to see VW version
>>         history in Store, or wanting to know what they've
>>         accidentally changed in base VW packages, needs to do that.
>>         We've even partly automated the reconcile + publish process,
>>         and I'm sure you could do better.
>>          
>>         "Logistical" - you mean Cincom has to find a way to
>>         distribute the published repository. An SQLite file sounds
>>         like an obvious solution. A method in the base to script the
>>         reconcile + publish would of course require zero effort to
>>         distribute.
>>          
>>         "Financial" - there are no direct costs involved, only time:
>>         tell us how long it will take you in hours. We already spend
>>         something like that number of hours, but without the
>>         certainty that what we arrive at is "official". You even have
>>         the offer of using Adventa's (or MetaCase's) publish, if that
>>         helps.
>>          
>>         Making the upgrade easier will also lower the perceived
>>         barrier to upgrading to a new VW version. The time and
>>         annoyance of having to do the reconcile and publish ourselves
>>         are one thing. Another is the worry that in a partially
>>         manual and hackily automated process, we may cause ourselves
>>         versioning problems. With a correct reconcile + publish, it's
>>         easy to check for potential problems by comparing with the
>>         previous version, and of course it's a major help in
>>         debugging such problems.
>
>
>         I think there's more to it than that. There are, for example,
>         some interesting issues about what exactly we're exporting.
>         So, presumably we'd want to be creating such an export with
>         each weekly build, and with release candidates. I'd certainly
>         hate to have something that we only did once a release and
>         expected to get precisely right that one time. So then what
>         are exporting? Do we use our own internal version numbers or
>         something else? What's the parent version? When we release, I
>         think what customers would ideally like is to have version
>         numbers like "7.6" whose parent versions are like "7.5". So
>         should we do an export that warps the version numbers and the
>         parents to that? And if we do, what do we do in the weekly
>         builds? Are their parents the previously weekly builds, or are
>         they the previous released version? And do release candidates
>         then each provide a set of versions named 7.6, but which are
>         different? Or should we just expose our internal names (These
>         currently tend to look something like 7.6jan08.3 3, reflecting
>         the build they're targeted at, plus an increment. But things
>         that haven't changed don't get re-published, so may have
>         version numbers from different builds, or if they haven't been
>         changed in this release, previous releases. Our internal names
>         also aren't always consistent). Should we change our internal
>         naming convention to be something that's more convenient for
>         customers, and if so, what would it be?
>
>         There's also the issue of support. If we're putting something
>         like this into supported status, then that implies also
>         putting into supported status and maintaining various other
>         things. Assuming SQLLite worked out, it means a SQLLite EXDI
>         layer, Store for SQLLite, some form of Store Replication (at
>         least to get it from that repository into the customers, and
>         probably to get it into that one in the first place), and the
>         specific bits of that required for SQLLite, if any. That's not
>         a trivial amount of stuff. If SQLLite is not supported on one
>         of our supported platforms (how's their SGI support? MacOS9?)
>         then that's also an issue. It's possible we could ignore that
>         on the assumption that everyone's got a machine around
>         somewhere that can talk to both SQLLite and their store
>         database, but that's an interesting decision. Or we could say
>         that we're distributing this, but not in a supported way -
>         again, an interesting product management type of decision.
>
>         Aside: I'm assuming we'd want to replicate because I think
>         we'd want to do this for the full set of things we're
>         distributing (or at least the full set of supported things),
>         not just for the things that are in the standard base image,
>         so reconciling and publishing with regular Store, which
>         requires things to be loaded into the image, seems less good.
>         There are also alternative ways of distributing it. One would
>         be as ExportDatabase files. But that needs some work, might
>         have some scaling issues for large databases, and relies on
>         John Brant's Replicator for its underlying support. Another
>         would be, rather than distributing a file, to give customers
>         access to a Store database online with the information in it
>         and let them replicate it however they want. That would still
>         require replication, but might address some of the issues of
>         version naming. We don't put the versions into that database
>         until we have a shipped release, and we rename them to
>         correspond to the release at that point. It could also just be
>         a Postgresql database, removing the issue of supporting
>         another one (although SQLLite might be useful in its own right
>         as a portable starter database for Store). And we could easily
>         make past versions available - handling the parentage if we
>         only ship one release's versions in the distribution is an
>         interesting question.
>
>         So, I think this is a good idea, I think it would be valuable
>         to customers (and a good number of customers seem to strongly
>         agree) and I've spent time thinking about how to implement it
>         previously. But it's also not trivial, and I don't even have a
>         good idea of how many hours it would take to do it. I don't
>         think that access to someone else's publish of the base is all
>         that helpful. We do have our own. The issues are mostly around
>         exactly what and how to distribute, and how this compares to
>         all the other things that customers think we ought to be doing.
>
>         One of the responders asked how people could help. The most
>         obvious way, of course, is by giving us money. It makes it
>         ever so much easier to find time and money when people give us
>         the latter. The more the better :-) However, barring that,
>         useful suggestions about how people would want to receive such
>         a distribution and what they would want it to look like are
>         probably the most valuable. And writing code is good - e.g.,
>         if someone wanted to write "rename and reparent while
>         replicating" functionality. But that's assuming that we knew
>         exactly what we wanted to do.
>
>
>
>         --
>         Alan Knight [|], Cincom Smalltalk Development
>         [hidden email]
>         [hidden email]
>         http://www.cincom.com/smalltalk
>