Uploaded image for project: 'Grouper'
  1. Grouper
  2. GRP-2059

changelogtemptochangelog make configurable number of entries to process at once

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • 2.4.0
    • grouperLoader
    • None
    • TIER packaging

    Description

      As discussed in tier-grouper slack channel:
      Since the changelogtemptochangelog processor is singular and exclusive in what it does… why not grab 500 or 1000 entries at a time, map and then insert in one statement or some other more efficient manner and then delete in bulk? It seems all the popular DBs would support this rather well and should dramatically increase the throughput of this process. Of course, make transactional where possible to limit replay. Default to 1 (like now) or make it 10, preferable. Should have significant impact on throughput of grouper changes.

      Attachments

        Activity

          People

            chris.hyzer@at.internet2.edu Chris Hyzer (upenn.edu)
            gettes@ufl.edu Michael Gettes
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: