Re: Performance of large queries

[prev] [thread] [next] [Date index for 2004/07/15]

On Thu, Jul 15, 2004 at 10:35:52AM +0100, Tony Bowden wrote:
> On Thu, Jul 15, 2004 at 10:27:05AM +0100, colm-cdbi@xxxxxxxxx.xx.xx wrote:
> > AIUI the issue is not that you are getting objects back but that
> > search() is internally implemented to only get ids and then each id has
> > to go back to the database to do a seperate retrieve() for the
> > 'Essential columns'.  It may very well be that this is no longer true, this
> > is just what it seemed to do last time I looked.  I've often
> > wondered if there was a way I was unaware of to (as an option) inject all the 'Essential'
> > data directly into the objects or if there was any big reason this would
> > be bad - can anyone shed any light on this?
> 
> If this is true it's a bug.
> 
> search() calls sql_Retrieve(), which is implemented as:
> 
> 	__PACKAGE__->set_sql(Retrieve => <<'');
> 	SELECT __ESSENTIAL__
> 	FROM   __TABLE__
> 	WHERE  %s

I had another look and it turns out I was talking rubbish.  Sorry for
the mis-information.  I must have made an incorrect assumption
about the function of _ids_to_objects.  I'm glad this came up as now I
know.

On Thu, Jul 15, 2004 at 10:35:52AM +0100, Tony Bowden wrote:
> The 'only fetching IDs' should only happen, therefore, if Essential ==
> Primary. Making your Essential group bigger (e.g. to All) should speed
> things up if you're going to need all the data anyway. 

On Thu, Jul 15, 2004 at 07:00:36AM +0000, cdbi-talk@xxxxxxxxx.xx wrote:
> I just started to use Class::DBI. It is a very nice tool and makes
> database-handling quite easy. But I have a little performance-problem
> when I use a 'search' that returns a large set of data (e.g. 2000
> objects). In this case the use of Class::DBI slows down the program by
> a factor of 3-4.

In light of this I'm not sure that your code should be 3-4 times slower.
Have you made sure (as above) that your setup means that all the methods
you are using are marked as Essential or All?  If so can you post an
example of the type of thing you are doing just in case there is something we can
suggest?

Colm