[$300] Initial Page Load

[rootelement]

Looks like when you go to https://discussions.topcoder.com, it takes ~9seconds for the page to load. After a couple page loads, i sorted the load time descending and see this:

[jmgasper]

Contest https://www.topcoder.com/challenges/30154851 has been created for this ticket.

This is an automated message for ghostar via Topcoder X

[jmgasper]

Contest https://www.topcoder.com/challenges/30154851 has been updated - it has been assigned to obog.

This is an automated message for ghostar via Topcoder X

[jmgasper]

@atelomycterus - Can we make speeding this up top priority please?

[atelomycterus]

@jmgasper Yes, sure

[atelomycterus]

@jmgasper There are 2 main issues.

Screens

Recent discussion page with a topcoder DB dump from my local env.
I've made the screens from DEV for statistics. We can compare performance after deploying.

Before any changes :

Controller - FIXED

Please apply PR -#249. Thanks!
Discussion Controller executed 31-45 sec my local env. The time of execution was count of different Users on the recent page * 0,5 sec. It executes about 3-5 secs in DEV.

Vanilla loads a recent discussion page and joins a list of users. UserModel loads users and checks Photo.
This piece of code works about 0,5 sec per user.

if ($column == 'Photo') {
  if ($value && !isUrl($value)) {
	$value = Gdn_Upload::url(changeBasename($value, 'n%s'));
  } elseif (!$value) {
	$value = UserModel::getDefaultAvatarUrl($user);
 }
}

After commenting it DiscussionController for recent page executes <1 sec.

Page rendering

Page rendering is slow due to our custom Topcoder link. Each discussion was rendered about 3,2-3,8 sec. In DEV i think about 0,5-1 sec per a discussion. Before rendering User link we request Topcoder API (Roles, Picture, Member rating) to apply proper css styles. We don't have cache for that.

Using default Vanilla function to render user link. The page was rendered 3 times faster.

In Dev Recent page is rendered about 20 sec, there are about 40 user links.

DEV

I did screens for statistics. So we can compare after fixing custom Topcoder link.

Recent page - about 40 user links:

https://vanilla.topcoder-dev.com/discussion/1715/home-challenges-forum-development-forums-sandy-forum-receiver-code-documents-new-discussio#latest - 1 user link:

https://vanilla.topcoder-dev.com/discussion/1633/review-time-extends#latest - about 25 user links:

https://vanilla.topcoder-dev.com/discussion/1838/test-editor-3333#latest 3 user links:

[atelomycterus]

@jmgasper I'm implementing cache in Topcoder plugin. Vanilla provides a cache layer.

[atelomycterus]

@jmgasper Keep you updated. I've implemented caching Topcoder User Details in Topcoder plugin. I started testing and enabled cache locally. Vanilla is full of surprises. After enabling cache, Topcoder auth isn't working because getByUserName returns always empty. It uses cached data and user wasn't found. I've fixed it. Then the same issues with permissions and roles. Vanilla couldn't start a new session because couldn't find permissions by UserID in cache. It throws errors, without any loading data from DB. Some Vanilla methods only use data from cache, others check cache and load DB.

I'm trying to fix these issues. I haven't used cache in my local env before. Sorry about that.
So don't enable cache.

$Configuration['Cache']['Enabled'] = getenv('CACHE_ENABLED'); 

DEV

I've checked cache parameters and I couldn't find 'Cache.Enabled' and 'Cache.Method' in DEV. MEMCACHED_SERVER has a value. If getenv('CACHE_ENABLED') is empty then cache is disabled.
In addition, I remember when I fixed the Guests permissions issue that the active cache (Gdn_Cache::activeCache()) returned false in DEV.

PROD

Could you check if getenv('CACHE_ENABLED') and getenv('CACHE_METHOD') are defined for PROD?

[atelomycterus]

@jmgasper Ready for your review.
PRs:
#251
topcoder-platform/forums-plugins#52
topcoder-platform/forums-groups-plugin#36

Configuration

Please check getenv('CACHE_ENABLED'), getenv('CACHE_METHOD') for Dev env. See my prev comment, these params looks like empty.

Example config with working memcached:

$Configuration['Cache']['Enabled'] = true; // getenv('CACHE_ENABLED');
$Configuration['Cache']['Method'] = 'memcached';//getenv('CACHE_METHOD');
$Configuration['Cache']['Memcached']['Store']=['memcached-local:11211']; // getenv('MEMCACHED_SERVER');

You can see cache info with debug bar. If Vanilla Cache is disabled then 'Active Cache' is false.

Topcoder plugins

PhotoUrl, IsAdmin and Rating are cached only. Gdn_Cache::FEATURE_EXPIRY is 3600. It allows items to auto-expire.

Results

Note that debugbar has little effect on page rendering.

No memcached

Page rendering was 85-86 sec for each request.

Memcached used, no changes in Topcoder plugin

Vanilla caches data (categories, users, roles) in memcached. Page rendering was 85-86 sec for each request.

Memcached used + changes in Topcoder plugin

Topcoder data (PhoroUrl, Rating, isAdmin) is cached.

1. Page rendering is 12-14 sec for the first request, no cached data in memcached due to re-build services locally.
   

2. Page rendering is < 1 sec (0.6-0.8 sec) for the next requests. Controller is about 2 sec.
   

Rendering per a discussion:

Topcoder Data in cache

TODO

There is still room for improvement.

• We can cache challenge roles per user (Topcoder Challenge API). It helps to render a discussion page/ a group page faster.
  Now Discussion page may load slowly if there are many comments on the page or a list of @userXXX (users mentioned).
  Group page may load slowly if there are many members because we request challenge roles and append it to handles.

[atelomycterus]

@jmgasper Please apply PR-#252. Thanks!

$Configuration['Cache']['Enabled'] must be boolean, getenv('CACHE_ENABLED') returns string value. I fixed it.

$Configuration['Cache']['Enabled'] = strtolower(getenv('CACHE_ENABLED')) === "true";

Sorry about that.

[jmgasper]

@atelomycterus - We set those config values in dev:

"CACHE_ENABLED": true,
  "CACHE_METHOD": "memcached"

I'm redeploying now, thanks.

[jmgasper]

@atelomycterus - Still seems a bit slow in dev - anything I can do to help debug what's still going on? Thanks!

[jmgasper]

@atelomycterus - One other thing - is there a key or keys we can check in the memcache to make sure that the caching is working? Thanks!

[atelomycterus]

@jmgasper I've checked DEV.

Memcached

The Vanilla cache is enabled and works.

Results in DEV

I've checked page rendering (#247 (comment)):

[atelomycterus]

@jmgasper

@atelomycterus - One other thing - is there a key or keys we can check in the memcache to make sure that the caching is working? Thanks!

You can connect to memcached with telnet and use 'stats' or 'get' ('get YOUR_KEY').
I used php -r '$c = new Memcached(); $c->addServer("memcached-local", 11211); var_dump( $c->getAllKeys() );'
This command should be run at Vanilla server because it uses php.

Displayed all keys:

[jmgasper]

Payment task has been updated: https://software.topcoder.com/review/actions/ViewProjectDetails?pid=30154851

This is an automated message for ghostar via Topcoder X