- Apr 03, 2020
-
-
Enrico Guiraud authored
...and add a test. The new argument is to make TTreeProcessorMT consistent with other multi-threading interfaces, namely TThreadExecutor. This is the last ingredient required to fix ROOT-10561.
-
- Apr 01, 2020
-
-
Enrico Guiraud authored
-
Enrico Guiraud authored
Internal logic has been adapted to support the case in which TTreeProcessorMT is acting on a TChain and the trees in the TChain do not all have the same name. Importantly, the fTreeName data-member changed name to fTreeNames and is now a vector of strings rather than a single string. This fixes ROOT-10178.
-
- Mar 24, 2020
-
-
Axel Naumann authored
Before, the TThreadedObject had to guess 64 slots were enough for the use case of TTreeProcessorMT - and with 256 cores that was not the case. Be explicit about the use of IMT for TThreadedObject by providing the new argument. IMT is enabled before the construction of the TThreadedObject (and thus TThreadedObject now knows how many slots to expect), because the pool is constructed before. If the order ever gets changes, the TThreadedObject will complain as it is now told to expect IMT usage. A future TTreeProcessorMT ctor overload might provide a specific thread count, and can pass it down to TThreadedObject.
-
- May 09, 2019
-
-
Danilo Piparo authored
-
Danilo Piparo authored
-
Danilo Piparo authored
-
Danilo Piparo authored
-
- Jan 21, 2019
-
-
Danilo Piparo authored
and not one task per cluster. We need to "fuse" together clusters if the number of clusters is to big with respect to the number of workers, otherwise we can incur in an overhead which is so big to make the parallelisation detrimental for performance. For example, this is the case when following a merging of many small files a file contains a tree with many entries and with clusters of just a few entries. The criterium according to which we fuse clusters together is to have at most TTreeProcessorMT::GetMaxTasksPerFilePerWorker() clusters per file per slot. For example: given 2 files and 16 workers, at most 16 * 2 * TTreeProcessorMT::GetMaxTasksPerFilePerWorker() clusters will be created, at most 16 * TTreeProcessorMT::GetMaxTasksPerFilePerWorker() per file.
-
- Nov 28, 2018
-
-
Enrico Guiraud authored
-
- Nov 15, 2018
-
-
Enrico Guiraud authored
We get races if all threads access the same TEntryList reference. This reverts commit f0556c0b, and adds a comment so we remember in the future.
-
Enrico Guiraud authored
-
- Nov 13, 2018
-
-
Danilo Piparo authored
-
- Oct 15, 2018
-
-
Danilo Piparo authored
in addition, remove the MakeClusters helper function from the header file.
-
- Jun 08, 2018
-
-
Enrico Guiraud authored
-
Enrico Guiraud authored
-
Enrico Guiraud authored
...when we are using global entry numbers and therefore each thread-local TChain has already been constructed with all files.
-
Enrico Guiraud authored
Previously all the dataset information was stored in the thread-local TTreeViews, causing data duplication between threads and requiring that TTreeProcessorMT queries it when it needs part of it. Now TTreeProcessorMT stores all the dataset information (tree name, filenames, friend names, friend file names, entry list) as constant thread-global data which is passed down to TTreeView::GetTreeReader.
-
- Jun 06, 2018
-
-
Enrico Guiraud authored
-
Enrico Guiraud authored
...i.e. when no friends and no TEntryList are present, so that we can use local rather than global entry numbers. If this is the case, now TTreeProcessorMT spawns one task per input file which reads the TTree cluster boundaries and spawns one sub-task per cluster.
-
Enrico Guiraud authored
This should decrease the amount of file switches that each thread-local TChain performs during an event loop, as each thread will tend to process clusters that belong to the same file.
-
- Jun 05, 2018
-
-
Enrico Guiraud authored
-
- Jun 04, 2018
-
-
Enrico Guiraud authored
This allows to error out early, as it was before "[TREEPROCMT] Construct thread-local chain at event-loop time".
-
Enrico Guiraud authored
This ensures that the friends of each thread-local TChain know which tree contains which global entry number without having to open all intermediate files to check how many entries they contain, resulting in much less contention in TTreeProcessorMT when multiple threads are loading the right file to process and friend trees are present. This fix partially addresses ROOT-9437.
-
Enrico Guiraud authored
-
Enrico Guiraud authored
This ensures that each thread-local TChain knows which tree contains which global entry number without having to open all intermediate files to check how many entries they contain, resulting in much less contention in TTreeProcessorMT when multiple threads are loading the right file to process.
-
Enrico Guiraud authored
The new signature will come in handy to let different tasks generate clusters for different files or sets of files in the future.
-
Enrico Guiraud authored
At event-loop time we will already have the list of clusters of entries, and we will be able to generate thread-local chains more efficiently, or even switch to thread-local trees if we can/wish to.
-
Enrico Guiraud authored
-
Enrico Guiraud authored
-
Enrico Guiraud authored
-
Enrico Guiraud authored
-
- Mar 01, 2018
-
-
Enrico Guiraud authored
This works around the issue described in ROOT-9281.
-
- Nov 10, 2017
-
-
Enric Tejedor Saavedra authored
This implementation intends to pave the way for the support of tree/chain friends in TDataFrame MT. For that purpose, now TTreeProcessorMT stores the information about the friends of the processed tree/chain. In order to provide support for friends, this implementation proposes a new way of dealing with chains in TTreeProcessorMT. The old way consisted in operating file by file, obtaining a TTree from each file and then constructing a TTreeReader for the particular range a task was going to process in that TTree. The aforementioned implementation prevented the support for friend chains, since there was no chain to add the friends to, only trees. In this new implementation, TTreeProcessorMT always constructs a chain per TTreeView, from which TTreeReaders are created to operate on ranges of the chain. If necessary, friends could be added to that chain.
-
- Aug 08, 2017
-
-
Enrico Guiraud authored
-
Enrico Guiraud authored
When processing a TTree with TTreeProcessorMT, each thread creates and uses its own TTreeView (by using a TThreadedObject<TTreeView>). Until now, in the case of interleaved execution of tasks, the second task could overwrite the first's TFile and TTree, causing crashes when the first task resumed execution. This commit modifies TTreeView so that it supports several tasks opening and using different TFiles at the same time, solving the issue described above.
-
Enrico Guiraud authored
-
Enrico Guiraud authored
Future commits will remove `fCurrentFile` and `fCurrentTree` altogether in order to make TTreeView play nice with TBB tasks. Then it will make little sense to pre-emptively open the first filename in each thread.
-
Enrico Guiraud authored
-
Enrico Guiraud authored
TTreeView's internal logic will be changed by future commits to play nice with TBB tasks, and will not be able to provide the interface that this logic was relying on.
-