diff --git a/proof/doc/v526/index.html b/proof/doc/v526/index.html index aa4a8cfdb50bf434176429d09ca108fbbc5c5d4b..0096349cee374046a8ad22904254bb4bb7d151c1 100644 --- a/proof/doc/v526/index.html +++ b/proof/doc/v526/index.html @@ -1,6 +1,7 @@ <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html><head> -</head><body> +<title>PROOF</title></head> +<body> <br> <hr> <a name="proof"></a> <h3>Proof</h3> @@ -29,6 +30,94 @@ created on the worker nodes by worker processes. The implementation is an extension of the functionality of the class TProofOutputFile used for merging via file. See http://root.cern.ch/drupal/content/handling-large-outputs-root-files</li> +<li>Add the possibility to enable/disable the tree cache and +to change its size on per-query base; two new parameters are available:</li> +<ul> +<li>PROOF_UseTreeCache +Int_t +Enable (0) or Disable (1) the tree cache (default 1)</li> +<li>PROOF_CacheSize +Long64_t Cache size in bytes +(default 10000000)</li> +</ul> +Examples:<br> + +a) to disable the cache for the next run enter:<br> + +proof->SetParameter("PROOF_UseTreeCache", 0)<br> + +b) to set the cache size to 20M<br> + +proof->SetParameter("PROOF_CacheSize", 20000000)<br> +<li> Add the parameter +PROOF_UseParallelUnzip to toggle the use of the parallel unzip +(default off for now); to enable it add the following call<br> + + + + proof->SetParameter("PROOF_UseParallelUnzip", 1) +</li> +<li> Add the possibility to give indications about +the number of workers at startup.<br> + E.g.<br> + 1. To +start max 5 workers<br> + +TProof::Open("<master>","workers=5")<br> + 2. To +start max 2 workers per physical machine<br> + +TProof::Open("<master>","workers=2x")<br> + This is useful in general when +running tests (equivalent but quicker then full startup<br> + followed by +TProof::SetParallel(n) or TProof::DeactivateWorker(...)).</li> +<li>Add support for the worker SysInfo_t in TSlaveInfo +(obtained via TProof::GetListOfSlaveInfos())</li> +<li>Add new submerger functionality to speed up the merging +phase. At the end of the query, a set of workers are promoted +submergers and assigned a sub-set of workers to merge. Once each +sub-merger has merged its sub-set of workers, it sends its result to +the master, which merges the partial results into the final<br> +set of results.<br> +The determination of the sub-mergers is always done dynamically, based +on the recent performance of workers. An optimal (i.e. giving the +highest speed-up) number can be calculated analytically under simple +assumptions.<br> +Merging via submergers is by default disabled. To enable it, with the +optimal number of sub-mergers, one should set the integer parameter +'PROOF_UseMergers' to 0, i.e.<br> +<br> + +proof->SetParameter("PROOF_UseMergers", 0)<br> +<br> +To force S sub-mergers (regardless of the optimal number) do<br> +<br> + +proof->SetParameter("PROOF_UseMergers", S)</li> +<br> +The new functionality can be tested in tutorials by adding the argument +'submergers' to runProof, e.g. <br> +<br> + + + root [0] .L +tutorials/proof/runProof.C+ <br> + + + root [1] +runProof("simple(nhist=10000,submergers)") <br> +<br> +(see the top of tutorials/proof/runProof.C for additional options). +<br> +A test for the submerger functionality has also been added to +test/stressProof.cxx . +<li>In PROOF-Lite, add the possibility for the administrator +to control the number of workers. This is done using +the rootrc variable ProofLite.MaxWorkers, which is read out of +/etc/system.rootrc and cannot be overwritten by users. Setting the +value to 0 disables PROOF-Lite.</li> + </ul> <li>Improvements</li> <ul> @@ -47,7 +136,7 @@ hadd).</li> </ul> <li>TProofOutputFile</li> <ul> -<li>Add support for the placeholded <file> +<li>Add support for the placeholder <file> the definition of the outputfile. This allows to have complete URL and to pass options to TFile::Open.</li> </ul> @@ -58,6 +147,57 @@ session rootrc file if the ROOT version is < 5.24/00 ; this acts as a workaround for the wrong TTreeCache initialization at the transition between local and remote files fixed in 5.24/00 .</li> </ul> +<li>Enable mass storage domain settings when working with +TChain's +in multi-master mode. The Mass Storage Domain must be specified as +option in the URL<br> +<br> + +chain.AddFile("root:// .....?msd=CERN")<br> +<br> + and the string must match the value specified in defining the +submaster node.</li> +<li>Improved performance monitoring: the 'Rate plot' button +in the dialog box has been renamed 'Performance Plot' and now shows up +to 4 plots as a function of the processing time:</li> +<ul> +<li>Instantaneous processing rate, which is now better +estimated by a better estimation of the normalizing times</li> +<li>Average read chunck size, defined as +TFile::GetFileBytesRead() / TFile::GetFileReadCalls() during the last +unit of time; this allows to monitor the usage of the cache; this plot +is present only if some I/O is done, i.e. not for pure CPU tasks.</li> +<li>The number of active workers</li> +<li>The number of total and effecive sessions running +concurrently on the cluster (started by the same daemon); this plot is +present only is the number is at least onec different from 1.</li> +</ul> +See also http://root.cern.ch/drupal/content/progress-dialog . +<li>If enabled, send monitoring information from the master +at each GetNextPacket (at each call of TPerfStat::PacketEvent) to allow +extrnal real-time progress monitoring.</li> +<li>Save the status of a 'proofserv' session into a new file +in the 'activesessions' area. The full path of the new file is<br> + +<admin_path>/.xproofd.<port>/activesessions/<user>.<group>.<pid>.status<br> +<br> +The status indicates whether the session is idle, running or queued.<br> +The status is updated every 'checkfq' secs (see xpd.proofservmgr; +default 30 s). The status is dumped by the reader thread of TXProofServ +and therefore its r/w access is protected.<br> +</li> +<li>Enable the use of the tree cache also for local files, +adapting the default settings for the cache to the recent changes</li> +<li>In the XrdProofd plug-in</li> +<ul> +<li>Improve synchronization between parent and child during +fork</li> +<li>Optimize loops over directory entries</li> +<li>Improve error and notification messages</li> +</ul> +<li>Improved handling of Ctrl-C; this follows from a fix in +TMonitor and an improved handling of non-finished query state in the +workers (results are not send to master if the query was aborted) </li> </ul> <li>Fixes</li> <ul> @@ -69,6 +209,10 @@ non-mergeable objects (fixes bug #52886)</li> MergeRecursive</li> <li>Do not remove the first file in the list when returning from MergeRecursive (fixes bug #54591)</li> +<li>Fix a major leak when merging files with collections +written using kSingleKey option. The merger was reading each +key in memory and deleted the object at the end, but the container is +not owner by default, so all objects inside leaked.</li> </ul> <li>PROOF-Lite</li> <ul> @@ -87,9 +231,16 @@ outputfile just after having open it</li> </ul> <li>XrdProofd plugin</li> <ul> -<li> Make sure that the limit on the number of old +<li>Make sure that the limit on the number of old sessions is applied whenever a new session is started and not only when the daemon is started.</li> +<li>Fix the behaviour of the xpd.allowedusers directive: if +at least one of these directives is present, users in the password file +are not allowed by default but must be explicitly appear in one +xpd.allowedusers directive </li> +<li>Fix a source for memory leak in +XrdProofdProtocol::SendMsg</li> +<li>Optimize the usage of strings in a few places</li> </ul> <li>DataSet manager</li> <ul> @@ -107,7 +258,49 @@ histogram in PolyMarker3D</li> creation of one pad per object</li> <li>Add wrapper to handle the feedback default canvas</li> </ul> +<li>TEventIter</li> +<ul> +<li>Fix a problem with changing the tree cache size: the +size was reset to the default value after the first file.</li> +</ul> +<li>TDataSetManagerFile</li> +<ul> +<li>Solve a consistency problem in checking URLs for +duplication when adding them to the relevant TFileInfo</li> +<li>During dataset validation, do not fail on duplications +but notify and add them to the bad file list</li> +</ul> +<li>TPacketizerAdaptive, TPacketizer</li> +<ul> +<li>Improve data node / worker matching by always using the +host FQDN</li> +</ul> +<li>TPacketizerUnit, TEventIter</li> +<ul> +<li>Make sure that the entry +number passed to TSelector::Process is unique and in increasing order +for non-data driven processing (packetizer TPacketizerUnit). This +allows to give a meaning to this variable, for example to related it to +one dimension of an integration.</li> +</ul> +<li>Fixes in PROOF-Lite:</li> +<ul> +<li>Make sure that with envs settings via TProof::AddEnvVar +are effective; this enables, for example, the automatic valgrind setup +introduced in 5.24/00 or the experiment specific settings via the +script defined by the env PROOF_INIT</li> +<li>Fix a problem with TProof::Load so that now it can be +also be used for PROOF-Lite</li> +</ul><li>TProofPlayerRemote</li><ul><li>In SendSelector, add misisng +option kCpBin when sending the selector source; the binary files were + never retrieved, even if present and valid</li></ul><li>TProofPlayerSlave</li><ul><li>In +Process, fix a problem with cache directory locking while building +the selector; the net effect was that each worker process was +re-buidling its own selector binary. </li></ul><li>TProofServ</li><ul><li>Fix +the order in which the log file is sent in asynchronous processing; the +wrong order was screwing up an immediate synchronous query submission +after an asynchronous run; this case occured, for example, in +'stressProof' .</li></ul> </ul> </ul> -</body> -</html> +</body></html> \ No newline at end of file