add threads to loader daemon like grouper loader to improve performance

From: Sachdeva, Vivek
Sent: Tuesday, November 04, 2014 5:29 PM
To: grouper-users
Subject: [grouper-users] Rule taking a long time

Hi,

I am running the following rule and it is taking very long time. It is been more than an hour and it is still running.

grouperSession = GrouperSession.startRootSession();

folder = StemFinder.findByName(grouperSession, "ucla:hierarchy");
readers = GroupFinder.findByName(grouperSession, "ucla:opus_roles:apo_director");
RuleApi.inheritGroupPrivileges(SubjectFinder.findRootSubject(), folder, Stem.Scope.SUB, readers.toSubject(), Privilege.getInstances("view, read"));

status = GrouperLoader.runOnceByJobName(grouperSession, GrouperLoaderType.GROUPER_RULES);

I am running grouper version 2.2.

When last time I ran it in 2.1.5, it ran pretty fast.

Are others seeing the same issue?

Thanks,
Vivek

From: Chris Hyzer
Sent: Tuesday, November 04, 2014 10:32 PM
To: 'Sachdeva, Vivek'; grouper-users
Subject: RE: Rule taking a long time

First of all if someone has any privilege, then they implicitly have view. So you don’t need to include view, just read. That will cut down the time by 50% . How many groups are under that folder? How many subjects in the group? Which database do you use? Try analyzing tables. Also, there are some performance improvements with 2.2.1, try that at some point (should be final soon)  the rule will take a while the first time you run it, then it is incremental from there so it will be faster. Maybe we can add threads to the rule daemon like I just did to the loader (added a jira for this)…

Get the group count like this:

select count(1) from grouper_groups where name like ' ucla:hierarchy:%';

Get the assigned count like this:

select count(1) from grouper_memberships_lw_v gmlv, grouper_groups gg
where gmlv.group_name like 'ucla:hierarchy:%'
and gmlv.list_name = 'readers'
and gmlv.subject_id = gg.id
and gmlv.subject_source = 'g:gsa'
and gg.name = 'ucla:opus_roles:apo_director';

Run that every minute for a few minutes and then you can see the progress…

Thanks,
Chris