Saturday, April 18. 2009How to force PostgreSQL to use a pre-calculated valuePrinter FriendlyRecommended Books: PostGIS In Action PostgreSQL 8.4 Official The SQL Language PostgreSQL 8.4 Server Administration
Comments
Display comments as
(Linear | Threaded)
It would be straightforward to put a caching function in front of the real function. The caching function would check if there's a precalculated result for the given arguments in the cache, and return the result from the cache if so. Otherwise, run the real function, and put the result in the cache.
Heikki,
That's an interesting thought. So are you thinking creating a function such as fn_cache_me(somekey,fn_slow_function(args)) and the cache would keep say a max of 1000 records and pull out if it sees a match or is there an easier way to do this. somekey the users would guarantee is unique foa given call. I guess what I'm finding strange is that sometimes it cahces and sometimes it doesn't and not quite sure what controls that. For example as one users pointed out the construct in PostGIS ST_Distance(a1,b1) As dist1, ST_Distance(a1,b1) As dist2 almost always caches, though for the above example I gave with pg_sleep it doesn't. Though not sure if that is because ST_Distance is implemented as a C function. However ST_Distance(a1,b1) + 1 as dist1, ST_Distance(a1,b1) + 2 As dist2 doesn't seem to cache the first call.
Checking for immutability and repeated calls with the same arguments could be an optimization target. Other factors might include the function cost, which I'm guessing is (or should be) set high for at least some of the PostGIS functions.
In 8.4, you'll be able to use CTEs as a way to materialize whole result sets including such function calls.
David,
I tried the immutable and also set function cost high to about 1000 and that didn't seem to help at all in the tests I have run. though it does help in use of && verses costly intersects vs I think what index it applies first when btree indexes are options.
Using "OFFSET 0" instead of "ORDER BY 1" has the same effect, without the overhead of sorting. It's still an undesirable trick-the-planner hack, though :(
The main use of the caching is for repeated calls over large sets. So the caching is performed across rows!
\timing select fn_very_slow(1) from generate_series(1,100) as i; Time: 1002.019 ms Even though fn_very_slow is called 100 times the overall time is still just about 1s. select fn_very_slow(1), fn_very_slow(1) from generate_series(1,100) as i; Time: 2003.386 ms Here the function is executed twice (although we are selecting from a 100 row table) Oracle (9i and 10g at least) behaves exactly the same.
But what about:
select fn_very_slow(1) from thetable where fn_very_slow(1) > 0; (assuming "fn_very_slow(1) > 0" returns all the records) even in this case fn_very_slow is called twice. a subquery won't help select a.theanswer from (select fn_very_slow(1) as theanswer from thetable) a where a.theanswer>0; uses the double query run-time as select 'constant' from (select fn_very_slow(1) as theanswer from thetable) a where a.theanswer>0; or select a.theanswer from (select fn_very_slow(1) as theanswer from thetable) a;
Nicklas,
This is pretty interesting - I was totally wrong about the ORDER BY. I forgot that ORDER BY anumber means ORDER BY the first column in the SELECT and not a constant. So guess I should change this to OFFSET 0 as moltonel suggested. Do you get these timings? But as Lars pointed out -- it is caching across rows not columns since its not calltime*numrows but rather calltime*numcols unless you use the OFFSET hack. -- 1000 ms select 'constant' from (select fn_very_slow(1) as theanswer from testi where i> 0) a where a.theanswer > 0; -- 2000 ms select theanswer from (select fn_very_slow(1) as theanswer from testi where i > 0) a WHERE a.theanswer > 0; -- 1000 ms SELECT a.theanswer from (select fn_very_slow(1) as theanswer from testi WHERE i > 0 OFFSET 0) a where a.theanswer > 0; --2000 ms SELECT a.theanswer from (select fn_very_slow(1) as theanswer from testi WHERE i > 0 ORDER BY 1) a where a.theanswer > 0; |
Entry's LinksQuicksearchCalendar
Categories
Blog Administration |
Simon Greener, wrote an article on how to load GPX xml files into Oracle XMLDB. That got me thinking that I haven't really explored all the XML features that PostgreSQL has to offer and to some extent I've been reticent about XML processed in any datab
Tracked: Apr 29, 00:23
Common table expressions are perhaps our favorite feature in PostgreSQL 8.4 even more so than windowing functions. Strangely enough I find myself using them more in SQL Server too now that PostgreSQL supports it. CTEs are not only nice syntactic sugar,
Tracked: Jul 17, 02:47
Tracked: Jul 17, 12:17