loader looks up all subjects when it doesnt need to

From: grouper-core-request On Behalf Of Shilen Patel
Sent: Wednesday, February 18, 2015 7:46 AM
To: grouper-core
Subject: [grouper-core] GRP-1051 - grouper loader performance bulk subject queries

Hi Chris,

Regarding this feature you added in 2.2.1. It looks like in 2.2.0 and in 2.2.1 when this feature is turned off, the loader will only lookup subjects that are changing memberships. So if I have a single loader group with 50K members and only 5 members are changing, it’ll do approximately 5 subject lookups. With the feature in 2.2.1, it seems like it wants to resolve all 50K members regardless of whether there are any changes.

Not sure if this behavior was expected. The issue came up after NYU upgraded from 2.2.0 to 2.2.1. With the default settings, some of their loader jobs went from taking ~75 seconds to about ~75 minutes. After they turned off the feature, their performance was back to the way it was. So they should be good for now but just reporting what they were seeing.

Thanks!

• Shilen