Iterators fetch the entire result set into memory

[prev] [thread] [next] [Date index for 2005/01/22]

I have some SQL statements that have the potential to return many rows (tens
to hundreds of thousands) and I noticed that in version 0.95 of the
Class::DBI that when you create an iterator, it's passed over the whole data
for the entire fetch and then returned a piece at a time.  Everything that
returns an iterator is called via Class::DBI->sth_to_objects which fetches
every row from the statment into memory.  I need to be able to deal with all
of the data, but I don't think it will all fit into memory at one time.  I
thought about overiding the Class::DBI::Iterator, but that won't help, since
sth_to_objects is what creates the iterator, so it's allready all been
fetched into memory by the time the constructor is invoked.  Is there any
way to emulate the equivalent of the DBI fetchrow statements with
Class::DBI?