Bug No Automatic Sitemap Generation after upgrade to 1.0.4

[bzcomputers]

We're not experiencing any issues with sitemap generation on this site: https://www.dropbox.com/s/70ny7ncgf02hytd/Screenshot 2014-02-26 14.35.21.png

I noticed your sitemap does not have a "custom" section which would be the first section run on my site and right where the automated process seems to hang (after deleting the previous sitemap sections). Would you mind trying with an automated custom urls file import test.

My problems began with 1.0.4 and continued into 1.0.5. The issue I'm having with the automated process coincidentally occurs right where a change was made to the class_sitemap.php file in these versions:
lines 252-259

		if ($first_page)
        {
            $sitemap_obj->remove_sitemaps();
            /*DBTECH_PRO_START*/
            $sitemap_obj->import_custom_urls();
            /*DBTECH_PRO_END*/
        }

P.S. Also it appears on your sitemap it took over 5 hours to complete all the sections? That seems rather excessive even for a large site.

 

[Fillip H.]

My local test board has custom URLs, and it does not experience any issues when I manually run the cron job.

I logged on to your forum, changed the sitemap generation time to 17:00 (the server time at the time of testing), then had to manually flush the cache due to XCache acting up. I was able to generate the sitemap with no issues. I observed the FTP, and all sitemap files were generated correctly.

Before I conducted my tests, your FTP (as well as the records in the ACP) showed that the sitemap had been successfully generated 21 hours prior, via the cron job.

 

[bzcomputers]

My local test board has custom URLs, and it does not experience any issues when I manually run the cron job.

I logged on to your forum, changed the sitemap generation time to 17:00 (the server time at the time of testing), then had to manually flush the cache due to XCache acting up. I was able to generate the sitemap with no issues. I observed the FTP, and all sitemap files were generated correctly.

Before I conducted my tests, your FTP (as well as the records in the ACP) showed that the sitemap had been successfully generated 21 hours prior, via the cron job.

Correct, after the last changes I haven't had issues manually running the cron job, it's just when it attempts to automatically run that the issues appear. When it attempts to automatically run it just clears all the old sitemap sections and hangs (doesn't create any sections at all, so it hangs right at the "custom" sitemap section). If I go in a few minutes after the automatic session attempted to run and did nothing (other than clear the old sitemaps) and click on the cron job manually all sections are created without issue.

I believe the ONLY way I was able to get the automatic sitemap to complete was when I went into the database and cleared the data (Text field) for "sitemapcronbuild", I'm about to test this again in just a couple minutes.

Edit: Well it just attempted to automatically run and again hung after clearing the old sitemaps. Interestingly as I was monitoring the adminutil table this time instead of updating the "sitemapcronbuild" (which is what I had been observing for the last 2 weeks trying to figure this out), it created a new field called "dbseositemapcronbuild" for the first time. At the same time just like I had been observing before when the automatic sitemap builds attempt to run not a single session was created in the table.

 

[Fillip H.]

Be advised I changed the field you're referring to to "dbseositemapcronbuild" on your site after completing my tests, to differentiate it from the vB4 sitemap in the rare instance someone leaves both of them enabled.

You can test automatic generation by following the above steps, and then also change the sitemap cron to run every minute temporarily, then simply refresh your forum once every 60 seconds while observing the FTP.

 

[bzcomputers]

You can test automatic generation by following the above steps, and then also change the sitemap cron to run every minute temporarily, then simply refresh your forum once every 60 seconds while observing the FTP.

I just attempted to test the automatic generation again by adjusting the cron job time. Same results - just deleting old sitemap sections and no new ones created..

One thing I did monitor is that the automatic cronjob new it needed to continue to run and is continuing to run for consecutive minutes (attempting to create each section) but not a single one is being created/saved.

 

[Fillip H.]

I discovered the issue, a function was being called in the sitemap builder file that is only available in the AdminCP. That explains why the manual cron job worked, why the front-end auto build didn't work and why it worked for me on my local board (other addons include the necessary file).

I have corrected this issue on your site, restored the scheduled task & configuration to its intended state and manually built the sitemap again.

 

[bzcomputers]

Thanks for taking the time on this. I felt like I was being a pain to keep bringing it up, but I knew something wasn't quite right. Thanks again.

 

[siddi]

Hi Fillip,

do you have a solution for this? I have the same problem.
Tonight was created only a part, and then nothing happened.

-rw-r--r--   1 www-data www-data 471K Feb 28 00:15 dbseo_sitemap_post_0.xml.gz
-rw-r--r--   1 www-data www-data 527K Feb 28 00:19 dbseo_sitemap_post_1.xml.gz
-rw-r--r--   1 www-data www-data 568K Feb 28 00:20 dbseo_sitemap_post_2.xml.gz
-rw-r--r--   1 www-data www-data 530K Feb 28 00:32 dbseo_sitemap_post_3.xml.gz
-rw-r--r--   1 www-data www-data 485K Feb 28 00:41 dbseo_sitemap_post_4.xml.gz
-rw-r--r--   1 www-data www-data 617K Feb 28 00:43 dbseo_sitemap_post_5.xml.gz
-rw-r--r--   1 www-data www-data 568K Feb 28 00:45 dbseo_sitemap_post_6.xml.gz
-rw-r--r--   1 www-data www-data 563K Feb 28 00:50 dbseo_sitemap_post_7.xml.gz
-rw-r--r--   1 www-data www-data 567K Feb 28 00:57 dbseo_sitemap_post_8.xml.gz

I hope you find a solution for this.
I can so far only manually create everything completely.

Thanks so lot!
Sid

 

[Fillip H.]

Yes, the release files have already been patched.

 

[siddi]

Oh cool. is it the same Version 1.0.5?

 

[bzcomputers]

I'm pretty sure if you redownload and install the current 1.0.5 this fix is already integrated.

 

[siddi]

Hi, sorry the same problem. The Sitemap was build incompletely
Only the "Build Sitemap" Function in the DBSEO Admincp works correct.

Can you check it please? I can send you Login Infos.

thanks

Siddi

 

[Fillip H.]

If you are having issues, please try reducing the number of items per page. Lower it to 10000 and see if that helps, and if not, go to 5000. From the looks of things, you have a very large forum and that may be causing timeouts on the cron job.

As the issue mentioned in the OP has been resolved, I'm closing this thread. Separate issues should be posted in a new thread. Thanks!