Hello,
I'm trying to figure out how to increase the memory space used by the Ciao system. I'm running some very large programs.
Mike
Michael M. Groat Doctoral Student Department of Computer Science University of New Mexico Cell: (510) 676-9235
============================================================================== Message: Address: Action: help majordomo(a)clip.dia.fi.upm.es Info. on useful commands subscribe ciao-users-request(a)clip.dia.fi.upm.es Subscribe to this list unsubscribe ciao-users-request(a)clip.dia.fi.upm.es Unsubscribe from this list <whatever> ciao-users(a)clip.dia.fi.upm.es Send message to list ----------------------------------------------------------------------------- Archived messages: http://www.clip.dia.fi.upm.es/Mail/ciao-users/ -----------------------------------------------------------------------------
I'm trying to figure out how to increase the memory space used by the Ciao system. I'm running some very large programs.
In Ciao the memory areas are resized automatically, i.e., each execution thread starts with a small set of stacks and when it runs out of space larger stacks are allocated and everything moved there (reallocation). If the system reports that it runs out of memory it means it really cannot use more memory, so in order to be able to terminate you have to optimize your program or reduce problem size. Alternative you can use 'old Prolog tricks'. A well known one is:
main :- read(X), large_deterministic_computation_part1(X,Y), large_deterministic_computation_part2(Y,Z), write(Z).
==>
main :- read(X), large_deterministic_computation_part1(X,Y), assert(foo(Y)), fail. main :- retract(foo(Y)), large_deterministic_computation_part2(Y,Z), write(Z).
Has saved many but, ouch! ;-)
In Ciao the memory areas are resized automatically, i.e., each execution thread starts with a small set of stacks and when it runs out of space larger stacks are allocated and everything moved there (reallocation). If the system reports that it runs out of memory it means it really cannot use more memory, so in order to be able to
Actually, although ideally all addressable memory should eventually be used up, in some cases there may be some holes left behind because stack expansion can happen in may different ways, and an optimal reallocation cannot be predicted beforehand. One possibility which may help you is to give very large sizes to the stacks so that they allocate all possible memory from the beginning. You can do this by setting the environment variables
GLOBALSTKSIZE LOCALSTKSIZE CHOICESTKSIZE TRAILSTKSIZE
which affect, respectively, the heap, the environment stack, the choicepoint stack, and the trail. The numbers you can give are mesured in 4-byte tagged words on a 32-bit machine. The maximum addressable space is 256Mb, so an upper bound of combined amount of GLOBALSTKSIZE + LOCALSTKSIZE + CHOICESTKSIZE + TRAILSTKSIZE you can give is 67108864. Now, some memory is used up by other bits of the machine, so a possible almost-maxima setup could be:
export GLOBALSTKSIZE=59000000 export LOCALSTKSIZE=2000000 export CHOICESTKSIZE=2000000 export TRAILSTKSIZE=2000000
The balance among the sizes of the different stacks depends heavily on your application. _Very_ roughly speaking, if it creates and actively uses large data structures, probably the global stack should take most of the addressable space (as in the example above). If it is highly non-deterministic then the share of the local and choice stacks and of the trail should take a higher part of the total.
If you call from time to time statistics/0 (in library prolog_sys) within your code you'll get a sample of where memory is being used. You can also get messages about when garbage collection runs (and how much memory it manages to reclaim) by appropriately setting the flag
`gc_trace' Governs garbage collection trace messages. An element off `[on,off,terse,verbose]'. Initially `off'.
FYI, the sizes Ciao starts with are:
GLOBALSTKSIZE 16384 LOCALSTKSIZE 4096 CHOICESTKSIZE 4096 TRAILSTKSIZE 4096
Notwithstanding, spending some time in tuning the Prolog code, especially if you have performance (time- or memory-wise) problems is usually a good investment. Besides the generic advice of "improving the algorithm", the single most useful advice is probably "avoid search and non-determinism unless necessary". The Art of Prolog and The Craft of Prolog give very valuable advices on this.
MCL (mcarro_at_fi_dot_upm_dot_es) _____________________________________________________________ If you can survive death, you will probably survive anything.
============================================================================== Message: Address: Action: help majordomo(a)clip.dia.fi.upm.es Info. on useful commands subscribe ciao-users-request(a)clip.dia.fi.upm.es Subscribe to this list unsubscribe ciao-users-request(a)clip.dia.fi.upm.es Unsubscribe from this list <whatever> ciao-users(a)clip.dia.fi.upm.es Send message to list ----------------------------------------------------------------------------- Archived messages: http://www.clip.dia.fi.upm.es/Mail/ciao-users/ -----------------------------------------------------------------------------