Ticket #814 (assigned clarification)

Opened 9 months ago

Last modified 2 months ago

feeds are slow

Reported by: russ Assigned to: russ (accepted)
Priority: high Milestone: 0.9.1
Component: ambra Version: 0.8.2.1
Keywords: feed Cc:
Blocking: Blocked By:

Description

howdy!

the feeds are really slow, so slow that building a feed often sends the stack into an unrecoverable hang.

i'm not sure why the feeds should be so slow.

most of the feeds we run have only category and/or date parameters, so it should be possible to server this list from the browse cache, using the browse services.

that would remove the necessity of running each article through XACML, which may be one of the bottlenecks.

i'm also pretty sure that all the info necessary to pull for each article exists in the browse cache for the non-extended feed.

the only other parameter (not used at all in the production system) is author. in this case, we'd need to go to mulgara (or lucene!!!) instead of building from browse.

Dependency Graph

Change History

03/03/08 10:49:37 changed by russ

  • owner changed from ronald to rich.

03/03/08 10:49:50 changed by russ

  • component changed from topaz to publishing-app.

03/03/08 10:56:37 changed by amit

  • version changed from 0.8.2.1-SNAPSHOT to 0.8.2.1.

Wrong version number.

06/13/08 12:53:47 changed by rich

  • owner changed from rich to russ.
  • blocking changed.
  • blockedby changed.
  • milestone set to 0.9.0.

Confirm with 0.9

06/13/08 16:58:42 changed by amit

  • type changed from defect to clarification.

07/09/08 12:48:59 changed by russ

  • owner changed from russ to amit.
  • type changed from clarification to defect.

like fetchArticle, things seem to have gotten worse in 0.9

the uncached main feed went from avg. 33293ms to avg. 73083 ms.

the uncached subject area feeds got slightly faster (from avg. 3358 ms to avg. 1793 ms), however that may be because, with the passage of time since creating the mirror of production, there are fewer articles to return in the default time frame.

i also noticed that we seem to be adding keys to FeedCache? and ObjectCache? when building a feed in 0.9. in 0.8 we also added keys to article-state. i'm not sure if this means that feeds are ignoring article-state (i will test separately) or if we're missing some caching that would be helpful, or if this is just the new way of doing things.

07/09/08 14:54:39 changed by amit

  • owner changed from amit to russ.

Based on the email exchange on the mailing list we are not sure the numbers are. Handing it back to Russ (although we might still run some tests)

07/16/08 11:13:00 changed by amit

  • status changed from new to closed.
  • resolution set to fixed.

Based on latest emails from Russ.

08/20/08 14:48:59 changed by russ

  • status changed from closed to reopened.
  • resolution deleted.

really? this is absolutely not fixed. it takes about 14 seconds to build the main plosone feed.

building an extended feed for plosone with a date range takes around 8 minutes.

08/20/08 14:57:08 changed by amit

:) Well, this was closed based on your emails. I have no problems with reopening it if those are the new numbers you are seeing.

08/20/08 14:57:22 changed by amit

  • milestone changed from 0.9.0 to 0.9.1.

08/21/08 10:45:38 changed by russ

which emails? yes, things are faster (about twice as fast) 0.9 vs. 0.8.

but they are still too slow to be usable.

08/21/08 10:45:45 changed by russ

  • owner changed from russ to rich.
  • status changed from reopened to new.

09/01/08 11:49:06 changed by russ

  • owner changed from rich to russ.
  • status changed from new to assigned.

taking this back to test performance post r6368

09/03/08 12:19:09 changed by rich

  • keywords set to feed.

09/08/08 15:52:31 changed by amit

  • type changed from defect to clarification.