Question Forum loads fast but takes up three times the Memory now.

[Bryan Choo]

Please help, my forum loads much faster now but i'm having memory issues which i have never had before. Before VBoptimize my site loaded slowly but after it it loads fast but it keeps going down throughout the day whenever the memory limit is exceeded. My forum use to work perfect with a 500MB memory limit. I have now increased this to almost triple because of the resources used by VBoptiimse to 1,200MB a month and its still constantly hitting the limit and going down. My host says i need to scale it up to 1,600MB but that is way too expensive and the average is only 400MB!! Just that sometimes i suddenly have 1,600MB spikes over the day because of VBoptimize and im paying 80$USD a month instead of 30$ .

Some other info:
My server is Apache and everytime it hits the memory limit, the site goes down and restarts for 5 mins.
My host is dreamhost, im using a VPS and php 5.2 with xcache. myphpinfo() at phpinfo()
Ran the system test, everything seems to be working fine, including the flushing etc but i dont seem to be using the datastore correctly.
I did not edit my config.php file. Should i be editing something here? I dont know what to write if i need to edit it please help me edit it and i will copy/paste it in...

// ****** DATASTORE CACHE CONFIGURATION *****
// Here you can configure different methods for caching datastore items.
// vB_Datastore_Filecache - to use includes/datastore/datastore_cache.php
// vB_Datastore_APC - to use APC
// vB_Datastore_XCache - to use XCache
// vB_Datastore_Memcached - to use a Memcache server, more configuration below
// $config['Datastore']['class'] = 'vB_Datastore_Filecache';

// ******** DATASTORE PREFIX ******
// If you are using a PHP Caching system (APC, XCache, eAccelerator) with more
// than one set of forums installed on your host, you *may* need to use a prefix
// so that they do not try to use the same variable within the cache.
// This works in a similar manner to the database table prefix.
// $config['Datastore']['prefix'] = '';

// It is also necessary to specify the hostname or IP address and the port the server is listening on
/*
$config['Datastore']['class'] = 'vB_Datastore_Memcached';
$i = 0;
// First Server
$i++;
$config['Misc']['memcacheserver'][$i] = '127.0.0.1';
$config['Misc']['memcacheport'][$i] = 11211;
$config['Misc']['memcachepersistent'][$i] = true;
$config['Misc']['memcacheweight'][$i] = 1;
$config['Misc']['memcachetimeout'][$i] = 1;
$config['Misc']['memcacheretry_interval'][$i] = 15;

Please spend some time helping me, I have bought 2 DBtech products and if you help resolve my issue i will really appreciate it and continue buying many more.

 

[Fillip H.]

"Memcached" means "Memory Cache Daemon", so it's intended and fully expected that memory usage increases

My advice to you is to switch vBO to use the Filecache option, and comment out the memcached datastore class in the bottom there

 

[Bryan Choo]

1) Ty but im using xcache not memcache. or is it something different?

2) and does this mean comment it out like this? I copy n paste this inside my config?

// ****** DATASTORE CACHE CONFIGURATION *****
// Here you can configure different methods for caching datastore items.
// vB_Datastore_Filecache - to use includes/datastore/datastore_cache.php
// vB_Datastore_APC - to use APC
// vB_Datastore_XCache - to use XCache
vB_Datastore_Memcached - to use a Memcache server, more configuration below
// $config['Datastore']['class'] = 'vB_Datastore_Filecache';

// ******** DATASTORE PREFIX ******

$config['Datastore']['prefix'] = '';

$config['Datastore']['class'] = 'vB_Datastore_Memcached';
$i = 0;
// First Server
$i++;
$config['Misc']['memcacheserver'][$i] = '127.0.0.1';
$config['Misc']['memcacheport'][$i] = 11211;
$config['Misc']['memcachepersistent'][$i] = true;
$config['Misc']['memcacheweight'][$i] = 1;
$config['Misc']['memcachetimeout'][$i] = 1;
$config['Misc']['memcacheretry_interval'][$i] = 15;

 

[Fillip H.]

Ohh sorry I missed the /* that comments out the memcached there, my bad You don't have to make any changes to config.php.

I still recommend switching to Filecache, as XCache is also a memory cacher AFAIK.

 

[Bryan Choo]

Sorry now i am even more confused. I am a newbie please be patient and explain it clearly to me.

So what do i do now?

Step 1: Do I switch from xcache to filecache?

Step 2: For the config file (please read it) do i use the first one or second one?

First one(original)

// ****** DATASTORE CACHE CONFIGURATION *****
// Here you can configure different methods for caching datastore items.
// vB_Datastore_Filecache - to use includes/datastore/datastore_cache.php
// vB_Datastore_APC - to use APC
// vB_Datastore_XCache - to use XCache
// vB_Datastore_Memcached - to use a Memcache server, more configuration below
// $config['Datastore']['class'] = 'vB_Datastore_Filecache';

// ******** DATASTORE PREFIX ******
// If you are using a PHP Caching system (APC, XCache, eAccelerator) with more
// than one set of forums installed on your host, you *may* need to use a prefix
// so that they do not try to use the same variable within the cache.
// This works in a similar manner to the database table prefix.
// $config['Datastore']['prefix'] = '';

// It is also necessary to specify the hostname or IP address and the port the server is listening on
/*
$config['Datastore']['class'] = 'vB_Datastore_Memcached';
$i = 0;
// First Server
$i++;
$config['Misc']['memcacheserver'][$i] = '127.0.0.1';
$config['Misc']['memcacheport'][$i] = 11211;
$config['Misc']['memcachepersistent'][$i] = true;
$config['Misc']['memcacheweight'][$i] = 1;
$config['Misc']['memcachetimeout'][$i] = 1;
$config['Misc']['memcacheretry_interval'][$i] = 15;
*/

Second one (took out commented line as u suggested? is this right?)

// ****** DATASTORE CACHE CONFIGURATION *****
// Here you can configure different methods for caching datastore items.
// vB_Datastore_Filecache - to use includes/datastore/datastore_cache.php
// vB_Datastore_APC - to use APC
// vB_Datastore_XCache - to use XCache
vB_Datastore_Memcached - to use a Memcache server, more configuration below
// $config['Datastore']['class'] = 'vB_Datastore_Filecache';

// ******** DATASTORE PREFIX ******

$config['Datastore']['prefix'] = '';

$config['Datastore']['class'] = 'vB_Datastore_Memcached';
$i = 0;
// First Server
$i++;
$config['Misc']['memcacheserver'][$i] = '127.0.0.1';
$config['Misc']['memcacheport'][$i] = 11211;
$config['Misc']['memcachepersistent'][$i] = true;
$config['Misc']['memcacheweight'][$i] = 1;
$config['Misc']['memcachetimeout'][$i] = 1;
$config['Misc']['memcacheretry_interval'][$i] = 15;

It would really help if you explained it like this.

Example:

Step 1: Yes switch to filecache, its better. Do not use xcache.
Step 2: Use the second one exactly as how you typed it and paste it in the config.php file.

Thanks alot.

 

[Fillip H.]

Step 1: Yes switch to filecache, as it will reduce your memory usage.
Step 2: Use the first one exactly as how you typed it and paste it in the config.php file

 

[Bryan Choo]

Thanks for the reply. Switched to filecache but still getting memory limit hit!

The site averages 200mb, i've switched it to 900mb just to account for the memory spikes alone.

Is there a better way to configure this? I really need help because although my site loads fast which is great, the extra hosting is costing me 4 times as much now.

If you can have a look at my site www.sc2sea.com it would be fantastic.


Heres the log over the past 24 hours, once memory is hit the server restarts and site goes down for 5 mins

2011-09-12 04:22 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 05:06 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 06:41 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 07:39 - ps23278 exceeded memory ceiling, killed all processes in context

(Applied File Cache)

2011-09-12 18:26 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 18:57 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 19:39 - ps23278 exceeded memory ceiling, killed all processes in context

2011-09-12 22:38 - ps23278 exceeded memory ceiling, killed all processes in context

 

[Fillip H.]

Perhaps loading cache from files also takes up memory, I'm not entirely sure about that.

In either case, how much are you paying for the host? You can easily get a good dedicated server or VPS for $100/mo, and unless you were paying $5 for your host before that would probably be cheaper than whatever you've upgraded to

 

[Bryan Choo]

Was paying 15$ / month for a VPS, now im paying $60 / month and my site only gets 2kish hits a day so $60 per month is reallly steep.

i also dont feel its fair at all because i use on average throughout the day just 400MB memory its just the spikes that come about 10-12 times a day that go up to 1600MB and make the server shutdown. Never had this problem before VBOptimise, but then again before VBoptimise it was super slow so im not complaining. But what im not happy about is I have to pay for 1600/mb constantly and its making me really disappointed because its costing me so much more, i thoguht this mod would help save costs in the long run or want to know if theres any way i could do so.

im really hoping u would actually take 10 mins of your time to help me solve the problem rather than giving me generic answers and quickly moving on to the next customer. Quality support would make me a life long customer rather than leave me very disappointed with a bad taste in my mouth, getting the feeling you dont care and just want to "clear the tickets" as soon as you can and move on to the next one.

 

[Fillip H.]

2k unique? If so, that's fairly large and I'm surprised you lasted that long on an obviously oversold VPS plan like $15/mo. DBTech used to be hosted on a $20/mo VPS with 192 MB RAM... Yeah that didn't last very long

I'm sorry if you feel misled by the mod, as far as I can tell on our product info page we made no claims that it would save costs. What the mod does is reduce more CPU / HDD usage (I'll admit we could be more specific in that department) by shifting relatively slow operations (like asking the HDD for the contents of a database file, etc) into the much, much faster space of RAM.

For 99% of your forum's database, what's happening is that the information vBulletin requests when it sends a query is stored on the HDD, and the CPU then requests the information from the HDD into memory (RAM), which is then displayed on the page and subsequently discarded from memory.

Depending on the speed of your HDD, this can be a very slow process. Think of it like you reading a book, and someone asks you for the contents of a specific sentence. You remember having seen that sentence on page 120, so what you do is turn each page until you reach 120, and then sift through the lines until you find the sentence you want.
This is essentially what the HDD does every time vBulletin executes a query.

The more data you request (number of sentences / words), the more time it takes to send back (read it back to whoever asked you to find it).

Now what a cache system does, is one of two things, depending on the type of cache system.

#1 - "Filecache"
This is very similar to the "normal" process, except it takes the database engine out of the equation. This speeds things up, sure, but less so because it's still waiting for the relatively slow HDD to fetch the data in question.

#2 - "RAM Cache"
Memcache is a type of RAM cache, because it takes the database engine AND the HDD out of the equation. This is the maximum amount of speed up possible, because if you know exactly the entire sentence the person asked you to read back, you don't need to open the book and sift through the pages at all, you can just recite it right there and then.

The reason why these two systems use more memory is because of what I just described, they're attempting to make fetching the requested information faster.

There is no way around the notion of increased memory usage when using a cache system.

This is why I haven't tried to "solve the problem", because there is no problem to solve. Your hosting provider seems very poor, not offering any burstable RAM (i.e. RAM you can temporarily access for spikes such as these) for VPS services. I thought all VPS providers offered burstable RAM, apparently not though.

Unfortunately, there is nothing that can be done.

 

[Bryan Choo]

Thanks for explaining it that way to that detail!

That was a fantastic response and I'm really happy you took the time to write it.

Im a happy customer now and will be glad to continue buying DBTech products with support like that

 

[Bryan Choo]

One last question.

Turns out liquidweb (this had rave reviews!!) doesn't offer the burstable ram service as well.

Could you please recommend to me 1 or 2 VPS hosts which might be good?

Thanks!

 

[Fillip H.]

Hawk Host - VPS Hosting may be your best bet that I know of