[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: scsh in PLT Scheme?

Alex Peake wrote:
> Large set operations should be done in the database. Almost every book,
> article, ... suggests NOT using cursors.
In an ideal world that is true; you should be able to express any query 
in the relational algebra to extract a reasonable set of tuples 
(reasonable meaning size in this context.) However in the actual world 
of Oracle, Sybase, insane database designers, crappy data, etc. you 
often must make alternate arrangements.

> Most of the time you want the whole set you queried, or else why did you not
> query a subset?
Most of the time you're correct.

> I think the default behavior should be the whole set.
I can live with that if there is an extended behavior that won't break 
if presented (intentionally or inadvertently) with a Big Data Set.

> Alex

Folks, I know this isn't easy; I would be very happy to live within some 
arbitrary boundaries just to get direct access to the DB from mzscheme. 
As a brief aside: the main reason I ported my application code from the 
supplied xml collection to SSAX was to handle large datasets; I needed 
an event-driven model so I didn't have to read in an entire tree of 
sxprs before doing any processing; I ran out of virtual memory (and 
patience) dealing with 300MB XML files that needed simple transforms. 
Hmm, event-driven, lazy evaluation, streams...